Development and Validation of a Machine Learning Model for Automated Assessment of Resident Clinical Reasoning Documentation.

BACKGROUND: Residents receive infrequent feedback on their clinical reasoning (CR) documentation. While machine learning (ML) and natural language processing (NLP) have been used to assess CR documentation in standardized cases, no studies have described similar use in the clinical environment. OBJECTIVE: The authors developed and validated using Kane's framework a ML model for automated assessment of CR documentation quality in residents' admission notes. DESIGN, PARTICIPANTS, MAIN MEASURES: Internal medicine residents' and subspecialty fellows' admission notes at one medical center from July 2014 to March 2020 were extracted from the electronic health record. Using a validated CR documentation rubric, the authors rated 414 notes for the ML development dataset. Notes were truncated to isolate the relevant portion; an NLP software (cTAKES) extracted disease/disorder named entities and human review generated CR terms. The final model had three input variables and classified notes as demonstrating low- or high-quality CR documentation. The ML model was applied to a retrospective dataset (9591 notes) for human validation and data analysis. Reliability between human and ML ratings was assessed on 205 of these notes with Cohen's kappa. CR documentation quality by post-graduate year (PGY) was evaluated by the Mantel-Haenszel test of trend. KEY RESULTS: The top-performing logistic regression model had an area under the receiver operating characteristic curve of 0.88, a positive predictive value of 0.68, and an accuracy of 0.79. Cohen's kappa was 0.67. Of the 9591 notes, 31.1% demonstrated high-quality CR documentation; quality increased from 27.0% (PGY1) to 31.0% (PGY2) to 39.0% (PGY3) (p < .001 for trend). Validity evidence was collected in each domain of Kane's framework (scoring, generalization, extrapolation, and implications). CONCLUSIONS: The authors developed and validated a high-performing ML model that classifies CR documentation quality in resident admission notes in the clinical environment-a novel application of ML and NLP with many potential use cases.

Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback.

BACKGROUND: Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability. OBJECTIVE: Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool. DESIGN, PARTICIPANTS, AND MAIN MEASURES: The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting. KEY RESULTS: The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality. CONCLUSIONS: The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.

Lessons in clinical reasoning ‒ pitfalls, myths and pearls: a case of recurrent pancreatitis.

OBJECTIVES: Cognitive biases can result in clinical reasoning failures that can lead to diagnostic errors. Autobrewery syndrome is a rare, but likely underdiagnosed, condition in which gut flora ferment glucose, producing ethanol. It most frequently presents with unexplained episodes of inebriation, though more case studies are necessary to better characterize the syndrome. CASE PRESENTATION: This is a case of a 41-year old male with a past medical history notable only for frequent sinus infections, who presented with recurrent episodes of acute pancreatitis. In the week prior to his first episode of pancreatitis, he consumed four beers, an increase from his baseline of 1-2 drinks per month. At home, he had several episodes of confusion, which he attributed to fatigue. He underwent laparoscopic cholecystectomy and testing for genetic and autoimmune causes of pancreatitis, which were non-revealing. He was hospitalized 10 more times during that 9-month period for acute pancreatitis with elevated transaminases. During these admissions, he had elevated triglycerides requiring an insulin drip and elevated alcohol level despite abstaining from alcohol for the prior eight months. His alcohol level increased after consumption of complex carbohydrates, confirming the diagnosis of autobrewery syndrome. CONCLUSIONS: Through integrated commentary on the diagnostic reasoning process, this case underscores how overconfidence can lead to premature closure and anchoring resulting in diagnostic error. Using a metacognitive overview, case discussants describe the importance of structured reflection and a standardized approach to early hypothesis generation to navigate these cognitive biases.

Hickam's dictum, Occam's razor, and Crabtree's bludgeon: a case of renal failure and a clavicular mass.

OBJECTIVES: Our discussant's thoughtful consideration of the patient's case allows for review of three maxims of medicine: Occam's razor (the simplest diagnosis is the most likely to be correct), Hickam's dictum (multiple disease entities are more likely than one), and Crabtree's bludgeon (the tendency to make data fit to an explanation we hold dear). CASE PRESENTATION: A 66-year-old woman with a history of hypertension presented to our hospital one day after arrival to the United States from Guinea with chronic daily vomiting, unintentional weight loss and progressive shoulder pain. Her labs are notable for renal failure, nephrotic range proteinuria and normocytic anemia while her shoulder X-ray shows osseous resorption in the lateral right clavicle. Multiple myeloma became the team's working diagnosis; however, a subsequent shoulder biopsy was consistent with follicular thyroid carcinoma. Imaging suggested the patient's renal failure was more likely a result of a chronic, unrelated process. CONCLUSIONS: It is tempting to bludgeon diagnostic possibilities into Occam's razor. Presumption that a patient's signs and symptoms are connected by one disease process often puts us at a cognitive advantage. However, atypical presentations, multiple disease processes, and unique populations often lend themselves more to Hickam's dictum than to Occam's razor. Diagnostic aids include performing a metacognitive checklist, engaging analytic thinking, and acknowledging the imperfections of these axioms.

An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department.

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients.

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department.

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3,661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

Carboplatin (every 21 days) and divided-dose paclitaxel (days 1, 11): rationale and tolerance in chemotherapy naïve women with high-grade epithelial cancers of Mullerian origin.

PURPOSE: We report here on the tolerance of a carboplatin-'divided dose' paclitaxel (given on days 1 and 11) regimen in chemotherapy-naïve patients with resected and staged endometrial epithelial neoplasms deemed at high-risk of recurrence or early stage epithelial high-grade serous tubo-ovarian adenocarcinomas after risk-reducing surgery. More recently, we applied this regimen as neoadjuvant chemotherapy for advanced ovarian cancer presentations. METHODS: A retrospective chart review of patients receiving this day 1, 11 paclitaxel regimens in combination with carboplatin at AUC 6 every 3 weeks since 2004 was carried out by the second author with subsequent updates by the first and third authors. Tolerance over the first three cycles was analyzed. RESULTS: A total of 27 women were treated with at least three cycles of this paclitaxel 'divided dose' schedule combined with carboplatin: 6 had endometrial adenocarcinoma, 9 had early stage ovarian cancer, and 12 received it as part of neoadjuvant therapy prior to undergoing cytoreductive surgery. Only 14 of 27 patients required dose reductions to complete the first three cycles of treatment. CONCLUSIONS: A median of three cycles of divided dose paclitaxel (D1, D11) concurrent with carboplatin dosed every 3 weeks was found to be safe and feasible as adjuvant to surgery in early endometrial and ovarian cancers or as neoadjuvant treatment in chemotherapy-naive women with ovarian cancer.

Clinical and laboratory characteristics associated with a high optical density anti-platelet factor 4 ELISA test.

PURPOSE: Diagnosing heparin-induced thrombocytopenia, a potentially catastrophic immune-mediated disorder, continues to pose significant challenges for clinicians, as both clinical and laboratory tools lack specificity. There is mounting evidence supporting a positive correlation between definitive heparin-induced thrombocytopenia and optical density (OD) positivity from the widely available anti-platelet factor 4 enzyme-linked immunosorbent assays (PF4 ELISAs). However, the clinical features distinguishing these patients remain poorly understood. PATIENTS AND METHODS: To better characterize this group, we conducted a case-controlled, retrospective chart review of patients from two large, urban academic institutions who underwent a PF4 ELISA at a central laboratory. Associations between OD and 18 clinical characteristics were calculated using the Fisher's exact test for categorical variables and Wilcoxon rank-sum test for continuous variables. RESULTS: In total, 184 negative patients (OD <0.7), and 121 positive patients (OD >0.7), including 74 low-positive patients (0.7< OD <1.4) and 47 high-positive patients (OD >1.4) were identified. Several clinical variables were significantly different in the negative group compared with the positive group, including hospital day (P<0.001), previous admission within the past 3 months (P<0.001), and the presence of a new thrombus (P=0.003). However, many of these variables were not different between the negative and low-positive group, and were only distinct between the negative and high-positive group. When the low-positive and high-positive groups were compared, only the 4T score was significantly different (P=0.003). CONCLUSION: These data indicate that those with OD >1.4 form a distinct clinical group and support the clinical utility of the 4T score.

Novel taxanes.

Since their addition to paclitaxel in the oncologists' armamentarium in the early 1990s, several new taxane formulations have been developed. Besides docetaxel and nab-paclitaxel, new analogs with better therapeutic profiles are being investigated. The goals of this next generation of taxanes are to improve the toxicity profile and efficacy, and to overcome resistance patterns. Several new taxanes, including cabazitaxel, paclitaxel poliglumex, paclitaxel+endotag, and polymeric-micellar paclitaxel, have shown clinical efficacy. These chemotherapeutics are part of many ongoing phase II and III studies on various cancer types. In addition, there are immunotoxins that link key antibodies to mitotic spindle inhibitors (trastuzumab emtansine and brentuximab vedotin). Through this mechanism, novel formulations increase cytotoxicity, improve specificity, and create possibilities for drug enhancement.

Clinical features of taxane neuropathy.

Sensory neuropathy is the dose-limiting toxicity of paclitaxel and also impacts on the use of docetaxel and other taxanes. The cause of this adverse effect has to do with their mechanism of action against microtubules and its interaction with neuronal cytoskeletal components. The variability of this toxicity is defined by several factors including disease type, taxane class, schedule and dose of the specific drug, patient demographics, and use of taxanes in combination regimens (especially with the platinums that are also neurotoxic). Prevention of life-long neuropathy is only produced if the causative drug is halted--treatments to reverse toxicity have shown only minimal improvement. This review investigates trials defining the clinical factors that determine the therapeutic window of taxanes and the enhanced susceptibility to this toxicity. In addition, case vignettes illustrate the range of clinical manifestations of this toxicity during taxane administration.

Defining risks of taxane neuropathy: insights from randomized clinical trials.

Sensory neuropathy is a common but difficult to quantify complication encountered during treatment of various cancers with taxane-containing regimens. Docetaxel, paclitaxel, and its nanoparticle albumin-bound formulation have been extensively studied in randomized clinical trials comparing various dose and schedules for the treatment of breast, lung, and ovarian cancers. This review highlights differences in extent of severe neuropathies encountered in such randomized trials and seeks to draw conclusions in terms of known pharmacologic factors that may lead to neuropathy. This basic knowledge provides an essential background for exploring pharmacogenomic differences among patients in relation to their susceptibility of developing severe manifestations. In addition, the differences highlighted may lead to greater insight into drug and basic host factors (such as age, sex, and ethnicity) contributing to axonal injury from taxanes.

Early recognition of renal toxicity of high-dose methotrexate therapy: a case report.

A 10-year-old boy with osteosarcoma and normal renal function manifested laboratory evidence of impending renal toxicity and extreme elevation of aspartate aminotrasferase and alanine aminotransferase within 2 hours after the completion of a 4-hour infusion of high-dose methotrexate (MTX) (12 g/m2), and went on to develop acute renal failure with life-threatening hyperkalemia 29 hours later. Although his renal function recovered completely with high-dose leucovorin, hemodialysis, charcoal hemoperfusion, and carboxypeptidase G2, we present this case to emphasize that signs of renal toxicity may be present as early as 2 hours after the completion of a 4-hour MTX infusion, and to suggest that monitoring for MTX toxicity should perhaps begin within a few hours after the completion of 4-hour MTX infusion.