Prompt Engineering Paradigms for Medical Applications: Scoping Review.

BACKGROUND: Prompt engineering, focusing on crafting effective prompts to large language models (LLMs), has garnered attention for its capabilities at harnessing the potential of LLMs. This is even more crucial in the medical domain due to its specialized terminology and language technicity. Clinical natural language processing applications must navigate complex language and ensure privacy compliance. Prompt engineering offers a novel approach by designing tailored prompts to guide models in exploiting clinically relevant information from complex medical texts. Despite its promise, the efficacy of prompt engineering in the medical domain remains to be fully explored. OBJECTIVE: The aim of the study is to review research efforts and technical approaches in prompt engineering for medical applications as well as provide an overview of opportunities and challenges for clinical practice. METHODS: Databases indexing the fields of medicine, computer science, and medical informatics were queried in order to identify relevant published papers. Since prompt engineering is an emerging field, preprint databases were also considered. Multiple data were extracted, such as the prompt paradigm, the involved LLMs, the languages of the study, the domain of the topic, the baselines, and several learning, design, and architecture strategies specific to prompt engineering. We include studies that apply prompt engineering-based methods to the medical domain, published between 2022 and 2024, and covering multiple prompt paradigms such as prompt learning (PL), prompt tuning (PT), and prompt design (PD). RESULTS: We included 114 recent prompt engineering studies. Among the 3 prompt paradigms, we have observed that PD is the most prevalent (78 papers). In 12 papers, PD, PL, and PT terms were used interchangeably. While ChatGPT is the most commonly used LLM, we have identified 7 studies using this LLM on a sensitive clinical data set. Chain-of-thought, present in 17 studies, emerges as the most frequent PD technique. While PL and PT papers typically provide a baseline for evaluating prompt-based approaches, 61% (48/78) of the PD studies do not report any nonprompt-related baseline. Finally, we individually examine each of the key prompt engineering-specific information reported across papers and find that many studies neglect to explicitly mention them, posing a challenge for advancing prompt engineering research. CONCLUSIONS: In addition to reporting on trends and the scientific landscape of prompt engineering, we provide reporting guidelines for future studies to help advance research in the medical field. We also disclose tables and figures summarizing medical prompt engineering papers available and hope that future contributions will leverage these existing works to better advance the field.

Finding full texts in bulk: a comparison of EndNote 20 versus Zotero 6 using the University of York's subscriptions.

Objective: To understand the performance of EndNote 20 and Zotero 6's full text retrieval features. Methods: Using the University of York's subscriptions, we tested and compared EndNote and Zotero's full text retrieval. 1,000 records from four evidence synthesis projects were tested for the number of: full texts retrieved; available full texts retrieved; unique full texts (found by one program only); and differences in versions of full texts for the same record. We also tested the time taken and accuracy of retrieved full texts. One dataset was tested multiple times to confirm if the number of full texts retrieved was consistent. We also investigated the available full texts missed by EndNote or Zotero by: reference type; whether full texts were available open access or via subscription; and the content provider. Results: EndNote retrieved 47% of available full texts versus 52% by Zotero. Zotero was faster by 2 minutes 15 seconds. Each program found unique full texts. There were differences in full text versions retrieved between programs. For both programs, 99% of the retrieved full texts were accurate. Zotero was less consistent in the number of full texts it retrieved. Conclusion: EndNote and Zotero do not find all available full texts. Users should not assume full texts are correct; are the version of record; or that records without full texts cannot be retrieved manually. Repeating the full text retrieval process multiple times could yield additional full texts. Users with access to EndNote and Zotero could use both for full text retrieval.

Automated System to Capture Patient Symptoms From Multitype Japanese Clinical Texts: Retrospective Study.

BACKGROUND: Natural language processing (NLP) techniques can be used to analyze large amounts of electronic health record texts, which encompasses various types of patient information such as quality of life, effectiveness of treatments, and adverse drug event (ADE) signals. As different aspects of a patient's status are stored in different types of documents, we propose an NLP system capable of processing 6 types of documents: physician progress notes, discharge summaries, radiology reports, radioisotope reports, nursing records, and pharmacist progress notes. OBJECTIVE: This study aimed to investigate the system's performance in detecting ADEs by evaluating the results from multitype texts. The main objective is to detect adverse events accurately using an NLP system. METHODS: We used data written in Japanese from 2289 patients with breast cancer, including medication data, physician progress notes, discharge summaries, radiology reports, radioisotope reports, nursing records, and pharmacist progress notes. Our system performs 3 processes: named entity recognition, normalization of symptoms, and aggregation of multiple types of documents from multiple patients. Among all patients with breast cancer, 103 and 112 with peripheral neuropathy (PN) received paclitaxel or docetaxel, respectively. We evaluate the utility of using multiple types of documents by correlation coefficient and regression analysis to compare their performance with each single type of document. All evaluations of detection rates with our system are performed 30 days after drug administration. RESULTS: Our system underestimates by 13.3 percentage points (74.0%-60.7%), as the incidence of paclitaxel-induced PN was 60.7%, compared with 74.0% in the previous research based on manual extraction. The Pearson correlation coefficient between the manual extraction and system results was 0.87 Although the pharmacist progress notes had the highest detection rate among each type of document, the rate did not match the performance using all documents. The estimated median duration of PN with paclitaxel was 92 days, whereas the previously reported median duration of PN with paclitaxel was 727 days. The number of events detected in each document was highest in the physician's progress notes, followed by the pharmacist's and nursing records. CONCLUSIONS: Considering the inherent cost that requires constant monitoring of the patient's condition, such as the treatment of PN, our system has a significant advantage in that it can immediately estimate the treatment duration without fine-tuning a new NLP model. Leveraging multitype documents is better than using single-type documents to improve detection performance. Although the onset time estimation was relatively accurate, the duration might have been influenced by the length of the data follow-up period. The results suggest that our method using various types of data can detect more ADEs from clinical documents.

Zero-Shot LLMs for Named Entity Recognition: Targeting Cardiac Function Indicators in German Clinical Texts.

INTRODUCTION: Large Language Models (LLMs) like ChatGPT have become increasingly prevalent. In medicine, many potential areas arise where LLMs may offer added value. Our research focuses on the use of open-source LLM alternatives like Llama 3, Gemma, Mistral, and Mixtral to extract medical parameters from German clinical texts. We concentrate on German due to an observed gap in research for non-English tasks. OBJECTIVE: To evaluate the effectiveness of open-source LLMs in extracting medical parameters from German clinical texts, specially focusing on cardiovascular function indicators from cardiac MRI reports. METHODS: We extracted 14 cardiovascular function indicators, including left and right ventricular ejection fraction (LV-EF and RV-EF), from 497 variously formulated cardiac magnetic resonance imaging (MRI) reports. Our systematic analysis involved assessing the performance of Llama 3, Gemma, Mistral, and Mixtral models in terms of right annotation and named entity recognition (NER) accuracy. RESULTS: The analysis confirms strong performance with up to 95.4% right annotation and 99.8% NER accuracy across different architectures, despite the fact that these models were not explicitly fine-tuned for data extraction and the German language. CONCLUSION: The results strongly recommend using open-source LLMs for extracting medical parameters from clinical texts, including those in German, due to their high accuracy and effectiveness even without specific fine-tuning.

Plants and their uses in dermatological recipes of the Receptarium of Burkhard III von Hallwyl from 16th century Switzerland - Data mining a historical text and preliminary in vitro screening.

ETHNOPHARMACOLOGICAL RELEVANCE: Historical texts on materia medica can be an attractive source of ethnopharmacological information. Various research groups have investigated corresponding resources from Europe and the Mediterranean region, pursuing different objectives. Regardless of the method used, the indexing of textual information and its conversion into data sets useful for further investigations represents a significant challenge. AIM OF THE STUDY: First, this study aims to systematically catalogue pharmaco-botanical information in the Receptarium of Burkhard von Hallwyl (RBH) in order to identify candidate plants in a targeted manner. Secondly, the potential of RBH as a resource for pharmacological investigations will be assessed by means of a preliminary in vitro screening. MATERIALS AND METHODS: We developed a relational database for the systematic recording of parameters composing the medical recipes contained in the historical text. Focusing on dermatological recipes, we explored the mentioned plants and their uses by drawing on specific literature. The botanical identities (candidate species) suggested in the literature for the historical plant names were rated based on their plausibility of being the correct attribution. The historical uses were interpreted by consulting medical-historical and modern clinical literature. For the subsequent in vitro screening, we selected candidate species used in recipes directed at the treatment of inflammatory or infectious skin disorders and wounds. Plants were collected in Switzerland and their hydroethanolic crude extracts tested for possible cytotoxic effects and for their potential to modulate the release of IL-6 and TNF in PS-stimulated whole blood and PBMCs. RESULTS: The historical text analysis points up the challenges associated with the assessment of historical plant names. Often two or more plant species are available as candidates for each of the 161 historical plant names counted in the 200 dermatological recipes in RBH. On the other hand, our method enabled to draw conclusions about the diseases underlying the 56 medical applications mentioned in the text. On this basis, 11 candidate species were selected for in vitro screening, four of which were used in RBH in herbal simple recipes and seven in a herbal compound formulation. None of the extracts tested showed a noteworthy effect on cell viability except for the sample of Sanicula europaea L. Extracts were tested at 50 µg/mL in the whole blood assay, where especially Vincetoxicum hirundinaria Medik. or Solanum nigrum L. showed inhibitory or stimulatory activities. In the PBMC assay, the root of Vincetoxicum hirundinaria revealed a distinct inhibitory effect on IL-6 release (IC50 of 3.6 µg/mL). CONCLUSIONS: Using the example of RBH, this study illustrates a possible ethnopharmacological path from unlocking the historical text and its subsequent analysis, through the selection and collection of plant candidates to their in vitro investigation. Fully documenting our approach to the analysis of historical texts, we hope to contribute to the discussion on solutions for the digital indexing of premodern information on the use of plants or other natural products.

Comparing NER Approaches on French Clinical Text, with Easy-to-Reuse Pipelines.

The task of Named Entity Recognition (NER) is central for leveraging the content of clinical texts in observational studies. Indeed, texts contain a large part of the information available in Electronic Health Records (EHRs). However, clinical texts are highly heterogeneous between healthcare services and institutions, between countries and languages, making it hard to predict how existing tools may perform on a particular corpus. We compared four NER approaches on three French corpora and share our benchmarking pipeline in an open and easy-to-reuse manner, using the medkit Python library. We include in our pipelines fine-tuning operations with either one or several of the considered corpora. Our results illustrate the expected superiority of language models over a dictionary-based approach, and question the necessity of refining models already trained on biomedical texts. Beyond benchmarking, we believe sharing reusable and customizable pipelines for comparing fast-evolving Natural Language Processing (NLP) tools is a valuable contribution, since clinical texts themselves can hardly be shared for privacy concerns.

Gilberti project and the creation of a linguistic dataset: Conservation and linguistic study of the P'urhépecha language.

This document provides a dataset transcription and translation of unpublished texts in the P'urhépecha language. The preserved texts are of a religious nature, reflecting the evangelizing efforts of missionaries during the 17th to 19th centuries, with a specific emphasis on the initiatives undertaken by the Gilberti Project at the Center for the Study of Traditions of El Colegio de Michoacán. The investigation introduces innovative digital tools and editable resources, opening new avenues for the study and preservation of the P'urhépecha language, ensuring its relevance and accessibility for future generations. The Gilberti Project has been active for over two decades, dedicating itself to the analysis of P'urhépecha texts. Beyond its academic role, the project significantly contributes to the conservation and promotion of the p'urhépecha language in several indigenous communities in the state of Michoacán, Mexico, where the language is still alive.

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study.

BACKGROUND: Although uncertainties exist regarding implementation, artificial intelligence-driven generative language models (GLMs) have enormous potential in medicine. Deployment of GLMs could improve patient comprehension of clinical texts and improve low health literacy. OBJECTIVE: The goal of this study is to evaluate the potential of ChatGPT-3.5 and GPT-4 to tailor the complexity of medical information to patient-specific input education level, which is crucial if it is to serve as a tool in addressing low health literacy. METHODS: Input templates related to 2 prevalent chronic diseases-type II diabetes and hypertension-were designed. Each clinical vignette was adjusted for hypothetical patient education levels to evaluate output personalization. To assess the success of a GLM (GPT-3.5 and GPT-4) in tailoring output writing, the readability of pre- and posttransformation outputs were quantified using the Flesch reading ease score (FKRE) and the Flesch-Kincaid grade level (FKGL). RESULTS: Responses (n=80) were generated using GPT-3.5 and GPT-4 across 2 clinical vignettes. For GPT-3.5, FKRE means were 57.75 (SD 4.75), 51.28 (SD 5.14), 32.28 (SD 4.52), and 28.31 (SD 5.22) for 6th grade, 8th grade, high school, and bachelor's, respectively; FKGL mean scores were 9.08 (SD 0.90), 10.27 (SD 1.06), 13.4 (SD 0.80), and 13.74 (SD 1.18). GPT-3.5 only aligned with the prespecified education levels at the bachelor's degree. Conversely, GPT-4's FKRE mean scores were 74.54 (SD 2.6), 71.25 (SD 4.96), 47.61 (SD 6.13), and 13.71 (SD 5.77), with FKGL mean scores of 6.3 (SD 0.73), 6.7 (SD 1.11), 11.09 (SD 1.26), and 17.03 (SD 1.11) for the same respective education levels. GPT-4 met the target readability for all groups except the 6th-grade FKRE average. Both GLMs produced outputs with statistically significant differences (P<.001; 8th grade P<.001; high school P<.001; bachelors P=.003; FKGL: 6th grade P=.001; 8th grade P<.001; high school P<.001; bachelors P<.001) between mean FKRE and FKGL across input education levels. CONCLUSIONS: GLMs can change the structure and readability of medical text outputs according to input-specified education. However, GLMs categorize input education designation into 3 broad tiers of output readability: easy (6th and 8th grade), medium (high school), and difficult (bachelor's degree). This is the first result to suggest that there are broader boundaries in the success of GLMs in output text simplification. Future research must establish how GLMs can reliably personalize medical texts to prespecified education levels to enable a broader impact on health care literacy.

The Ability of ChatGPT in Paraphrasing Texts and Reducing Plagiarism: A Descriptive Analysis.

Background: The introduction of ChatGPT by OpenAI has garnered significant attention. Among its capabilities, paraphrasing stands out. Objective: This study aims to investigate the satisfactory levels of plagiarism in the paraphrased text produced by this chatbot. Methods: Three texts of varying lengths were presented to ChatGPT. ChatGPT was then instructed to paraphrase the provided texts using five different prompts. In the subsequent stage of the study, the texts were divided into separate paragraphs, and ChatGPT was requested to paraphrase each paragraph individually. Lastly, in the third stage, ChatGPT was asked to paraphrase the texts it had previously generated. Results: The average plagiarism rate in the texts generated by ChatGPT was 45% (SD 10%). ChatGPT exhibited a substantial reduction in plagiarism for the provided texts (mean difference -0.51, 95% CI -0.54 to -0.48; P<.001). Furthermore, when comparing the second attempt with the initial attempt, a significant decrease in the plagiarism rate was observed (mean difference -0.06, 95% CI -0.08 to -0.03; P<.001). The number of paragraphs in the texts demonstrated a noteworthy association with the percentage of plagiarism, with texts consisting of a single paragraph exhibiting the lowest plagiarism rate (P<.001). Conclusions: Although ChatGPT demonstrates a notable reduction of plagiarism within texts, the existing levels of plagiarism remain relatively high. This underscores a crucial caution for researchers when incorporating this chatbot into their work.

The conversational AI "ChatGPT" outperforms medical students on a physiology university examination.

Artificial intelligence (AI) has gained massive interest with the public release of the conversational AI "ChatGPT," but it also has become a matter of concern for academia as it can easily be misused. We performed a quantitative evaluation of the performance of ChatGPT on a medical physiology university examination. Forty-one answers were obtained with ChatGPT and compared to the results of 24 students. The results of ChatGPT were significantly better than those of the students; the median (IQR) score was 75% (66-84%) for the AI compared to 56% (43-65%) for students (P < 0.001). The exam success rate was 100% for ChatGPT, whereas 29% (n = 7) of students failed. ChatGPT could promote plagiarism and intellectual laziness among students and could represent a new and easy way to cheat, especially when evaluations are performed online. Considering that these powerful AI tools are now freely available, scholars should take great care to construct assessments that really evaluate student reflection skills and prevent AI-assisted cheating.NEW & NOTEWORTHY The release of the conversational artificial intelligence (AI) ChatGPT has become a matter of concern for academia as it can easily be misused by students for cheating purposes. We performed a quantitative evaluation of the performance of ChatGPT on a medical physiology university examination and observed that ChatGPT outperforms medical students obtaining significantly better grades. Scholars should therefore take great care to construct assessments crafted to really evaluate the student reflection skills and prevent AI-assisted cheating.

Pattern transitions in diary data of MDD patients: a mixed-methods multiple case study of psychotherapy dynamics.

Aim: Mixed-methods approaches promise a deep understanding of psychotherapeutic processes. This study uses qualitative and quantitative data from daily diary entries and daily self-assessments during inpatient treatment. The aim of the study is to get an insight into the similarities and differences between both types of data and how they represent self-organized pattern transitions in psychotherapy. While a complete correlation of results is not expected, we anticipate observing amplifying and subsidiary patterns from both perspectives. Materials and methods: Daily, five MDD patients wrote diaries and completed self-assessments using the Therapy Process Questionnaire, a questionnaire for monitoring the change dynamics of psychotherapy. The data were collected using the Synergetic Navigation System, an online tool for real-time monitoring. Diary entries of the patients described their experiences in everyday life. The qualitative text analysis was conducted using Mixed Grounded Theory, which provided categories representing the patients' ongoing experiences of transformation and stagnation. The time series data was analyzed using the dynamic complexity algorithm and the pattern transition detection algorithm. Results from qualitative and quantitative analyses were combined and compared. Following the process of data triangulation, the leading perspective came from the theory of self-organization. In addition to presenting the overall results for all five patients, we delve into two specific case examples in greater detail. Results: Specific and highly diversified diary entries of 5 patients were classified into the categories of perceived pattern stability, noticing improvement, broadening the perspective, critical instability, and experiencing moments of Kairos. Patients reported problems not only related to their disorder (e.g., lack of energy and hopelessness) but also to phases and steps of change, which could be related to the theory of self-organization (e.g., problem attractors, critical fluctuations, pattern transitions, and Kairos). Qualitative and quantitative analysis provide important supplementary results without being redundant or identical. Conclusion: Data triangulation allows for a comprehensive and multi-perspective understanding of therapeutic change dynamics. The different topics expressed in the diary entries especially help to follow micro-psychological processes, which are far from being a simple reaction to interventions. The way patients experience themselves being in stability or instability and stagnation or transformation is surprisingly close to the general features of self-organizing processes in complex systems.

Hattie and Sammler: Saul Bellow's older woman and older man.

Hattie in Saul Bellow's "Leaving the Yellow House" and Sammler in Bellow's Mr. Sammler's Planet are both elderly characters. This article intends to compare the two characters from a gender perspective, to illustrate how these characters appear to experience and respond to old age and how other characters in these two fictions respond to the old age of their respective elderly characters. The comparison of these two characters in the fiction of Saul Bellow gives rise to the observation that old age is not merely a phase of negative changes but also of positive ones; ageism claims victims among both men and women whose suffering is aggravated by other kinds of injustice, such as racism and sexism.

Pediatrics, Pediatricians and Child Care : Based on 16th Dr. K. C. Chaudhuri and Dr. Amala Chaudhuri Oration, on 15th October 2023.

Medical problems of children and their differences from adults have been mentioned in the ancient texts. Important contributions to medicine, including treatment of diseases of children were made by Greek and many scholars from middle East countries in 10th century. Pediatrics became widely recognized in Europe and USA during early 19th century and a number of children's hospitals were established in major cities. With technological advances, pediatric subspecialties also developed. In India, pediatrics was recognized around 1950s and thereafter, gradually progressed. Pediatric specialties came up in 1970s and became well established during 2020s. Pediatricians are regarded as doctors treating sick children. Pediatric specialists have the responsibility of providing tertiary care to patients with complex systemic diseases and critical care. In our country having a huge underprivileged population, pediatricians need to play a wider role and aim to provide comprehensive care that would lead to optimum development for every child. They should be aware of child rights, widely prevalent child abuse and exploitation and legal protective mechanisms, and attempt to tackle these issues in association with other agencies and organizations working for child welfare.

Investigating topic modeling techniques through evaluation of topics discovered in short texts data across diverse domains.

The online channel has affected many facets of an individual's identity, commercial, social policy, and culture, among others. It implies that discovering the topics on which these brief writings are focused, as well as examining the qualities of these short texts is critical. Another key issue that has been identified is the evaluation of newly discovered topics in terms of topic quality, which includes topic separation and coherence. A topic modeling method has been shown to be an outstanding aid in the linguistic interpretation of quite tiny texts. Based on the underlying strategy, topic models are divided into two categories: probabilistic methods and non-probabilistic methods. In this research, short texts are analyzed using topic models, including latent Dirichlet allocation (LDA) for probabilistic topic modeling and non-negative matrix factorization (NMF) for non-probabilistic topic modeling. A novel approach for topic evaluation is used, such as clustering methods and silhouette analysis on both models, to investigate performance in terms of quality. The experiment results indicate that the proposed evaluation method outperforms on both LDA and NMF.

Dynamic source credibility and its impacts on knowledge revision.

Corrections to readers' misconceptions should result in higher belief when information sources are of high credibility. However, evaluations of credibility may be malleable, and we do not yet fully understand how changes to a source's credibility influence readers' credibility evaluations and knowledge revision outcomes. Thus, in two experiments, we examined how updating a source's credibility (Experiment 1: initially neutral sources later updated to be high-, low-, or neutral-credibility sources; Experiment 2: initially high- or low-credibility sources later updated to be low- or high-credibility sources) influenced knowledge revision and source credibility evaluations after readers engaged with refutation and non-refutation texts. Results showed that readers revised their credibility judgments from neutral-, high-, and low-credibility initial evaluations, indicating that source judgments are malleable rather than fixed. In addition, refutations from sources that are later revealed to be of high credibility can facilitate revision of both knowledge and initial source credibility evaluations.

Egyptian Dynasties to Biologics: A Historical Review of Nasal Polyposis.

The history of nasal polyposis originates even before Hippocrates described a nasal mass that he likened to a sea polyp. References to sinonasal disease and treatment can be found in ancient texts, such as the Ebers Papyrus and the Edwin Smith Papyrus of Ancient Egypt, as well as in the foundational texts of Ayurvedic medicine. Greek philosophers marked a significant shift away from the belief that illness was a result of divine intervention and embraced medical theory. Over the subsequent millennia, the understanding of nasal polyposis expanded, resulting in notable progress in surgical procedures and medical treatments. However, the complex pathophysiology of this condition remained enigmatic until breakthroughs in basic science and immunology. This historical journey takes us from the tomb of the first rhinologist in 2500 BC to the development of immune-modulating biologics.

Simplified COVID-19 guidance for adults with intellectual and developmental disabilities.

BACKGROUND: During the COVID-19 pandemic, the United States' Centers for Disease Control and Prevention (CDC) created guidance documents that were too complex to be read and understood by the majority of adults with intellectual and developmental disabilities who often read at or below a third-grade reading level. This study explored the extent to which these adults could read and understand CDC documents simplified using Minimised Text Complexity Guidelines. METHOD: This study involved 20 participants, 18-48 years of age. Participants read texts and responded to multiple-choice items and open-ended questions to gather information about how they interacted with and understood the texts. RESULTS: The results provide initial evidence that the Minimised Text Complexity Guidelines resulted in texts that participants could read and understand. CONCLUSION: Implications for increasing the accessibility of public health information so that it can be read and understood by adults with extremely low literacy skills are discussed.

The Temperature Feature of ChatGPT: Modifying Creativity for Clinical Research.

More clinicians and researchers are exploring uses for large language model chatbots, such as ChatGPT, for research, dissemination, and educational purposes. Therefore, it becomes increasingly relevant to consider the full potential of this tool, including the special features that are currently available through the application programming interface. One of these features is a variable called temperature, which changes the degree to which randomness is involved in the model's generated output. This is of particular interest to clinicians and researchers. By lowering this variable, one can generate more consistent outputs; by increasing it, one can receive more creative responses. For clinicians and researchers who are exploring these tools for a variety of tasks, the ability to tailor outputs to be less creative may be beneficial for work that demands consistency. Additionally, access to more creative text generation may enable scientific authors to describe their research in more general language and potentially connect with a broader public through social media. In this viewpoint, we present the temperature feature, discuss potential uses, and provide some examples.

A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study.

BACKGROUND: Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration. OBJECTIVE: This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification. METHODS: This study used a data set of 861 PSE reports from a large academic hospital's maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier's predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems. RESULTS: The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95/126) compared to an accuracy of 66.7% (84/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight. CONCLUSIONS: This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.

Rule-Based Text Classification of Dental Diagnosis.

Unstructured medical records boast an abundance of information that could greatly facilitate medical decision-making and improve patient care. With the development of Natural Language Processing (NLP) methodology, the free-text medical data starts to attract more and more research attention. Most existing studies try to leverage the power of such unstructured data using Machine Learning algorithms, which would usually require a relatively large training set, and high computational capacity. However, when faced with a smaller-scale project, opting for an alternative approach may be more effective and practical. This project proposes an efficient and light-weight rule-based approach to categorize dental diagnosis data. It not only fills the void of dental records in the medical free-text processing area, but also demonstrates that with expertly designed research structure and proper implementation, simple method could achieve our study goal very competently.