Prediction of episode of hemodynamic instability using an electrocardiogram based analytic: a retrospective cohort study.

BACKGROUND: Predicting the onset of hemodynamic instability before it occurs remains a sought-after goal in acute and critical care medicine. Technologies that allow for this may assist clinicians in preventing episodes of hemodynamic instability (EHI). We tested a novel noninvasive technology, the Analytic for Hemodynamic Instability-Predictive Indicator (AHI-PI), which analyzes a single lead of electrocardiogram (ECG) and extracts heart rate variability and morphologic waveform features to predict an EHI prior to its occurrence. METHODS: Retrospective cohort study at a quaternary care academic health system using data from hospitalized adult patients between August 2019 and April 2020 undergoing continuous ECG monitoring with intermittent noninvasive blood pressure (NIBP) or with continuous intraarterial pressure (IAP) monitoring. RESULTS: AHI-PI's low and high-risk indications were compared with the presence of EHI in the future as indicated by vital signs (heart rate > 100 beats/min with a systolic blood pressure < 90 mmHg or a mean arterial blood pressure of < 70 mmHg). 4,633 patients were analyzed (3,961 undergoing NIBP monitoring, 672 with continuous IAP monitoring). 692 patients had an EHI (380 undergoing NIBP, 312 undergoing IAP). For IAP patients, the sensitivity and specificity of AHI-PI to predict EHI was 89.7% and 78.3% with a positive and negative predictive value of 33.7% and 98.4% respectively. For NIBP patients, AHI-PI had a sensitivity and specificity of 86.3% and 80.5% with a positive and negative predictive value of 11.7% and 99.5% respectively. Both groups performed with an AUC of 0.87. AHI-PI predicted EHI in both groups with a median lead time of 1.1 h (average lead time of 3.7 h for IAP group, 2.9 h for NIBP group). CONCLUSIONS: AHI-PI predicted EHIs with high sensitivity and specificity and within clinically significant time windows that may allow for intervention. Performance was similar in patients undergoing NIBP and IAP monitoring.

External Validation and Comparison of a General Ward Deterioration Index Between Diversely Different Health Systems.

OBJECTIVES: Implementing a predictive analytic model in a new clinical environment is fraught with challenges. Dataset shifts such as differences in clinical practice, new data acquisition devices, or changes in the electronic health record (EHR) implementation mean that the input data seen by a model can differ significantly from the data it was trained on. Validating models at multiple institutions is therefore critical. Here, using retrospective data, we demonstrate how Predicting Intensive Care Transfers and other UnfoReseen Events (PICTURE), a deterioration index developed at a single academic medical center, generalizes to a second institution with significantly different patient population. DESIGN: PICTURE is a deterioration index designed for the general ward, which uses structured EHR data such as laboratory values and vital signs. SETTING: The general wards of two large hospitals, one an academic medical center and the other a community hospital. SUBJECTS: The model has previously been trained and validated on a cohort of 165,018 general ward encounters from a large academic medical center. Here, we apply this model to 11,083 encounters from a separate community hospital. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The hospitals were found to have significant differences in missingness rates (> 5% difference in 9/52 features), deterioration rate (4.5% vs 2.5%), and racial makeup (20% non-White vs 49% non-White). Despite these differences, PICTURE's performance was consistent (area under the receiver operating characteristic curve [AUROC], 0.870; 95% CI, 0.861-0.878), area under the precision-recall curve (AUPRC, 0.298; 95% CI, 0.275-0.320) at the first hospital; AUROC 0.875 (0.851-0.902), AUPRC 0.339 (0.281-0.398) at the second. AUPRC was standardized to a 2.5% event rate. PICTURE also outperformed both the Epic Deterioration Index and the National Early Warning Score at both institutions. CONCLUSIONS: Important differences were observed between the two institutions, including data availability and demographic makeup. PICTURE was able to identify general ward patients at risk of deterioration at both hospitals with consistent performance (AUROC and AUPRC) and compared favorably to existing metrics.

Machine Learning Model of Emergency Department Use for Patients Undergoing Treatment for Head and Neck Cancer Using Comprehensive Multifactor Electronic Health Records.

PURPOSE: To use a hybrid method, combining statistical profiling, machine learning (ML), and clinical evaluation to predict emergency department (ED) visits among patients with head and neck cancer undergoing radiotherapy. MATERIALS AND METHODS: Patients with head and neck cancer treated with radiation therapy from 2015 to 2019 were identified using electronic health record data. Records from 60 days before 90 days after treatment were analyzed. Statistical profiling and ML were used to create a predictive model for ED visits during or after radiation therapy. A comprehensive set of variables were studied. Multiple ML models were developed including extreme gradient-boosted decision tree and generalized logistic regression with comparison of multiple predictive performance metrics. RESULTS: Of the 1,355 patients studied, 13% had an ED visit during or after treatment. Our hybrid methodology enabled evidence-based winnowing of candidate features from 141 to 11 with clinically applicable, evidence-based thresholds. Extreme gradient boosting had the highest area under the curve (0.81 ± 0.06) with a sensitivity of 0.89 ± 0.10 and exceeded generalized logistic regression (area under the curve 0.64 ± 0.02). Significant predictors of ED visits during treatment included increasingly complex opioid use, number of prior ED visits, tumor volume, rate of change of blood urea nitrogen, total bilirubin, body mass index, and distance from hospital. CONCLUSION: Our approach combining bootstrapped statistical profiling and ML importance analysis supported integration of clinician input to identify a distilled set of phenotypical characteristics for developing ML models predicting which patients undergoing head and neck cancer radiation therapy were at risk for ED visits.

Cost-effectiveness of an Emergency Department-Based Intensive Care Unit.

Importance: Value in health care is quality per unit cost (V = Q/C), and an emergency department-based intensive care unit (ED-ICU) model has been associated with improved quality. To assess the value of this care delivery model, it is essential to determine the incremental direct cost of care. Objective: To determine the association of an ED-ICU with inflation-adjusted change in mean direct cost of care, net revenue, and direct margin per ED patient encounter. Design, Setting, and Participants: This retrospective economic analysis evaluated the cost of care delivery to patients in the ED before and after deployment of the Joyce and Don Massey Family Foundation Emergency Critical Care Center, an ED-ICU, on February 16, 2015, at a large academic medical center in the US with approximately 75 000 adult ED visits per year. The pre-ED-ICU cohort was defined as all documented ED visits by patients 18 years or older with a complete financial record from September 8, 2012, through June 30, 2014 (660 days); the post-ED-ICU cohort, all visits from July 1, 2015, through April 21, 2017 (660 days). Fiscal year 2015 was excluded from analysis to phase in the new care model. Statistical analysis was performed March 1 through December 30, 2021. Exposures: Implementation of an ED-ICU. Main Outcomes and Measures: Inflation-adjusted direct cost of care, net revenue, and direct margin per patient encounter in the ED. Results: A total of 234 884 ED visits during the study period were analyzed, with 115 052 patients (54.7% women) in the pre-ED-ICU cohort and 119 832 patients (54.5% women) in the post-ED-ICU cohort. The post-ED-ICU cohort was older (mean [SD] age, 49.1 [19.9] vs 47.8 [19.6] years; P < .001), required more intensive respiratory support (2.2% vs 1.1%; P < .001) and more vasopressor use (0.5% vs 0.2%; P < .001), and had a higher overall case mix index (mean [SD], 1.7 [2.0] vs 1.5 [1.7]; P < .001). Implementation of the ED-ICU was associated with similar inflation-adjusted total direct cost per ED encounter (pre-ED-ICU, mean [SD], $4875 [$15 175]; post-ED-ICU, $4877 [$17 400]; P = .98). Inflation-adjusted net revenue per encounter increased by 7.0% (95% CI, 3.4%-10.6%; P < .001), and inflation-adjusted direct margin per encounter increased by 46.6% (95% CI, 32.1%-61.2%; P < .001). Conclusions and Relevance: Implementation of an ED-ICU was associated with no significant change in inflation-adjusted total direct cost per ED encounter. Holding delivery costs constant while improving quality demonstrates improved value via the ED-ICU model of care.

Emergency department length of stay and outcomes of emergency department-based intensive care unit patients.

OBJECTIVE: Emergency department (ED) boarding of patients who are critically ill is associated with poor outcomes. ED-based intensive care units (ED-ICUs) may mitigate the risks of ED boarding. We sought to analyze the impact of ED length of stay (LOS) before transfer to an ED-ICU on patient outcomes. METHODS: We retrospectively analyzed adult ED patients managed in the ED-ICU at a US medical center. Bivariate and multivariable linear regressions tested ED LOS as a predictor of inpatient ICU and hospital LOS, and separate bivariate and multivariable logistic regressions tested ED LOS as a predictor of inpatient ICU admission, 48-hour mortality, and hospital mortality. Multivariable analyses' covariates were age, sex, Charlson Comorbidity Index (CCI), Emergency Severity Index, and eSimplified Acute Physiology Score (eSAPS3). RESULTS: We included 5859 ED visits with subsequent care in the ED-ICU. Median age, CCI, eSAPS3, ED LOS, and ED-ICU LOS were 62 years (interquartile range [IQR], 48-73 years), 5 (IQR, 2-8), 46 (IQR, 36-56), 3.6 hours (IQR, 2.5-5.3 hours), and 8.5 hours (IQR, 5.3-13.4 hours), respectively, and 46.3% were women. Bivariate analyses showed negative associations of ED LOS with hospital LOS (ß = -3.4; 95% confidence interval [CI], -5.9 to -1.0), inpatient ICU admission (odds ratio [OR], 0.86, 95% CI, 0.84-0.88), 48-hour mortality (OR, 0.89; 95% CI, 0.82-0.98), and hospital mortality (OR, 0.89; 95% CI, 0.85-0.92), but no association with inpatient ICU LOS. Multivariable analyses showed a negative association of ED LOS with inpatient ICU admission (OR, 0.91; 95% CI, 0.88-0.93), but no associations with other outcomes. CONCLUSIONS: We observed no significant associations between ED LOS before ED-ICU transfer and worsened outcomes, suggesting an ED-ICU may mitigate the risks of ED boarding of patients who are critically ill.

Developing an electronic health record-derived health equity dashboard to improve learner access to data and metrics.

OBJECTIVES: It is essential to engage learners in efforts aimed at dismantling racism and other contributors to health care disparities. Barriers to their involvement include limited access to data. The objective of our study was to create a data dashboard using an existing quality improvement (QI) infrastructure and provide resident access to data to facilitate exploratory analysis on disparities in emergency department (ED) patient care. METHODS: Focusing on patient populations that have previously been shown in the literature to suffer significant disparities in the ED, we extracted outcomes across a variety of metrics already collected as part of routine ED operations. Using data visualization software, we developed an interactive dashboard for visual exploratory analyses. RESULTS: We designed a dashboard for our resident learners with views that are flexible and allow user selected filters to view clinical outcomes by patient age, treatment area, and chief complaint. Learners were also allowed to select grouping and outcomes of interest to investigate questions and form new hypotheses of their choosing. Available dashboard views included summary counts view to assess ED visits over time by selectable group, a rooming and triage acuity view, time-to-event survival curve view, histogram and box plot views for continuous variables, a view to assess outcome variables by time of day of ED arrival, customizable contingency table views, and correspondence analysis. CONCLUSIONS: Utilizing an existing QI infrastructure, we developed a dashboard that provides a new perspective into commonly collected ED operations data to allow for the exploration of disparities in ED care that is accessible to learners. Future directions include using these data to refine hypotheses on ED disparities, understand root causes, develop interventions, and measure their impact.

An emergency department-based intensive care unit is associated with decreased hospital length of stay for upper gastrointestinal bleeding.

INTRODUCTION: Upper gastrointestinal bleeding (UGIB) is associated with substantial morbidity, mortality, and intensive care unit (ICU) utilization. Initial risk stratification and disposition from the Emergency Department (ED) can prove challenging due to limited data points during a short period of observation. An ED-based ICU (ED-ICU) may allow more rapid delivery of ICU-level care, though its impact on patients with UGIB is unknown. METHODS: A retrospective observational study was conducted at a tertiary U.S. academic medical center. An ED-ICU (the Emergency Critical Care Center [EC3]) opened in February 2015. Patients presenting to the ED with UGIB undergoing esophagogastroduodenoscopy within 72 h were identified and analyzed. The Pre- and Post-EC3 cohorts included patients from 9/2/2012-2/15/2015 and 2/16/2015-6/30/2019. RESULTS: We identified 3788 ED visits; 1033 Pre-EC3 and 2755 Post-EC3. Of Pre-EC3 visits, 200 were critically ill and admitted to ICU [Cohort A]. Of Post-EC3 visits, 682 were critically ill and managed in EC3 [Cohort B], whereas 61 were critically ill and admitted directly to ICU without care in EC3 [Cohort C]. The mean interval from ED presentation to ICU level care was shorter in Cohort B than A or C (3.8 vs 6.3 vs 7.7 h, p < 0.05). More patients in Cohort B received ICU level care within six hours of ED arrival (85.3 vs 52.0 vs 57.4%, p < 0.05). Mean hospital length of stay (LOS) was shorter in Cohort B than A or C (6.2 vs 7.3 vs 10.0 days, p < 0.05). In the Post-EC3 cohort, fewer patients were admitted to an ICU (9.3 vs 19.4%, p < 0.001). The rate of floor admission with transfer to ICU within 24 h was similar. No differences in absolute or risk-adjusted mortality were observed. CONCLUSION: For critically ill ED patients with UGIB, implementation of an ED-ICU was associated with reductions in rate of ICU admission and hospital LOS, with no differences in safety outcomes.

Clinical Informatics Training During Emergency Medicine Residency: The University of Michigan Experience.

Clinical informatics (CI) is a rich field with longstanding ties to resident education in many clinical specialties, although a historic gap persists in emergency medicine. To address this gap, we developed a CI track to facilitate advanced training for senior residents at our 4-year emergency medicine residency. We piloted an affordable project-based approach with strong ties to operational leadership at our institution and describe specific projects and their outcomes. Given the relatively low cost, departmental benefit, and unique educational value, we believe that our model is generalizable to many emergency medicine residencies. We present a pathway to defining a formal curriculum using Kern's framework.

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods.

BACKGROUND: COVID-19 has led to an unprecedented strain on health care facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here, we present the results of an analytical model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to identify patients at high risk for imminent intensive care unit transfer, respiratory failure, or death, with the intention to improve the prediction of deterioration due to COVID-19. OBJECTIVE: This study aims to validate the PICTURE model's ability to predict unexpected deterioration in general ward and COVID-19 patients, and to compare its performance with the Epic Deterioration Index (EDI), an existing model that has recently been assessed for use in patients with COVID-19. METHODS: The PICTURE model was trained and validated on a cohort of hospitalized non-COVID-19 patients using electronic health record data from 2014 to 2018. It was then applied to two holdout test sets: non-COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to EDI and NEWS scores for head-to-head comparison via area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve. We compared the models' ability to predict an adverse event (defined as intensive care unit transfer, mechanical ventilation use, or death). Shapley values were used to provide explanations for PICTURE predictions. RESULTS: In non-COVID-19 general ward patients, PICTURE achieved an AUROC of 0.819 (95% CI 0.805-0.834) per observation, compared to the EDI's AUROC of 0.763 (95% CI 0.746-0.781; n=21,740; P<.001). In patients testing positive for COVID-19, PICTURE again outperformed the EDI with an AUROC of 0.849 (95% CI 0.820-0.878) compared to the EDI's AUROC of 0.803 (95% CI 0.772-0.838; n=607; P<.001). The most important variables influencing PICTURE predictions in the COVID-19 cohort were a rapid respiratory rate, a high level of oxygen support, low oxygen saturation, and impaired mental status (Glasgow Coma Scale). CONCLUSIONS: The PICTURE model is more accurate in predicting adverse patient outcomes for both general ward patients and COVID-19 positive patients in our cohorts compared to the EDI. The ability to consistently anticipate these events may be especially valuable when considering potential incipient waves of COVID-19 infections. The generalizability of the model will require testing in other health care systems for validation.

Identifying trigger concepts to screen emergency department visits for diagnostic errors.

OBJECTIVES: The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm. METHODS: We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED. RESULTS: Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings. CONCLUSIONS: We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Demonstrating the consequences of learning missingness patterns in early warning systems for preventative health care: A novel simulation and solution.

When using tree-based methods to develop predictive analytics and early warning systems for preventive healthcare, it is important to use an appropriate imputation method to prevent learning the missingness pattern. To demonstrate this, we developed a novel simulation that generated synthetic electronic health record data using a variational autoencoder with a custom loss function, which took into account the high missing rate of electronic health data. We showed that when tree-based methods learn missingness patterns (correlated with adverse events) in electronic health record data, this leads to decreased performance if the system is used in a new setting that has different missingness patterns. Performance is worst in this scenario when the missing rate between those with and without an adverse event is the greatest. We found that randomized and Bayesian regression imputation methods mitigate the issue of learning the missingness pattern for tree-based methods. We used this information to build a novel early warning system for predicting patient deterioration in general wards and telemetry units: PICTURE (Predicting Intensive Care Transfers and other UnfoReseen Events). To develop, tune, and test PICTURE, we used labs and vital signs from electronic health records of adult patients over four years (n = 133,089 encounters). We analyzed primary outcomes of unplanned intensive care unit transfer, emergency vasoactive medication administration, cardiac arrest, and death. We compared PICTURE with existing early warning systems and logistic regression at multiple levels of granularity. When analyzing PICTURE on the testing set using all observations within a hospital encounter (event rate = 3.4%), PICTURE had an area under the receiver operating characteristic curve (AUROC) of 0.83 and an adjusted (event rate = 4%) area under the precision-recall curve (AUPR) of 0.27, while the next best tested method-regularized logistic regression-had an AUROC of 0.80 and an adjusted AUPR of 0.22. To ensure system interpretability, we applied a state-of-the-art prediction explainer that provided a ranked list of features contributing most to the prediction. Though it is currently difficult to compare machine learning-based early warning systems, a rudimentary comparison with published scores demonstrated that PICTURE is on par with state-of-the-art machine learning systems. To facilitate more robust comparisons and development of early warning systems in the future, we have released our variational autoencoder's code and weights so researchers can (a) test their models on data similar to our institution and (b) make their own synthetic datasets.

Association of an Emergency Department-Based Intensive Care Unit With Survival and Inpatient Intensive Care Unit Admissions.

Importance: Increased patient acuity, decreased intensive care unit (ICU) bed availability, and a shortage of intensivist physicians have led to strained ICU capacity. The resulting increase in emergency department (ED) boarding time for patients requiring ICU-level care has been associated with worse outcomes. Objective: To determine the association of a novel ED-based ICU, the Emergency Critical Care Center (EC3), with 30-day mortality and inpatient ICU admission. Design, Setting, and Participants: This retrospective cohort study used electronic health records of all ED visits between September 1, 2012, and July 31, 2017, with a documented clinician encounter at a large academic medical center in the United States with approximately 75 000 adult ED visits per year. The pre-EC3 cohort included ED patients from September 2, 2012, to February 15, 2015, when the EC3 opened, and the post-EC3 cohort included ED patients from February 16, 2015, to July 31, 2017. Data analyses were conducted from March 2, 2018, to May 28, 2019. Exposures: Implementation of EC3, an ED-based ICU designed to provide rapid initiation of ICU-level care in the ED setting and seamless transition to inpatient ICUs. Main Outcomes and Measures: The main outcomes were 30-day mortality among ED patients and rate of ED to ICU admission. Results: A total of 349 310 visits from a consecutive sample of ED patients (mean [SD] age, 48.5 [19.7] years; 189 709 [54.3%] women) were examined; the pre-EC3 cohort included 168 877 visits and the post-EC3 cohort included 180 433 visits. Implementation of EC3 was associated with a statistically significant reduction in risk-adjusted 30-day mortality among all ED patients (pre-EC3, 2.13%; post-EC3, 1.83%; adjusted odds ratio, 0.85; 95% CI, 0.80-0.90; number needed to treat, 333 patient encounters; 95% CI, 256-476). The risk-adjusted rate of ED admission to ICU decreased with implementation of EC3 (pre-EC3, 3.2%; post-EC3, 2.7%; adjusted odds ratio, 0.80; 95% CI, 0.76-0.83; number needed to treat, 179 patient encounters; 95% CI, 149-217). Conclusions and Relevance: Implementation of a novel ED-based ICU was associated with improved 30-day survival and reduced inpatient ICU admission. Additional research is warranted to further explore the value of this novel care delivery model in various health care systems.

Willingness and Ability of Older Adults in the Emergency Department to Provide Clinical Information Using a Tablet Computer.

OBJECTIVES: To estimate the proportion of older adults in the emergency department (ED) who are willing and able to use a tablet computer to answer questions. DESIGN: Prospective, ED-based cross-sectional study. SETTING: Two U.S. academic EDs. PARTICIPANTS: Individuals aged 65 and older. MEASUREMENTS: As part of screening for another study, potential study participants were asked whether they would be willing to use a tablet computer to answer eight questions instead of answering questions orally. A custom user interface optimized for older adults was used. Trained research assistants observed study participants as they used the tablets. Ability to use the tablet was assessed based on need for assistance and number of questions answered correctly. RESULTS: Of 365 individuals approached, 248 (68%) were willing to answer screening questions, 121 of these (49%) were willing to use a tablet computer; of these, 91 (75%) were able to answer at least six questions correctly, and 35 (29%) did not require assistance. Only 14 (12%) were able to answer all eight questions correctly without assistance. Individuals aged 65 to 74 and those reporting use of a touchscreen device at least weekly were more likely to be willing and able to use the tablet computer. Of individuals with no or mild cognitive impairment, the percentage willing to use the tablet was 45%, and the percentage answering all questions correctly was 32%. CONCLUSION: Approximately half of this sample of older adults in the ED was willing to provide information using a tablet computer, but only a small minority of these were able to enter all information correctly without assistance. Tablet computers may provide an efficient means of collecting clinical information from some older adults in the ED, but at present, it will be ineffective for a significant portion of this population.

Formative evaluation of a patient-specific clinical knowledge summarization tool.

OBJECTIVE: To iteratively design a prototype of a computerized clinical knowledge summarization (CKS) tool aimed at helping clinicians finding answers to their clinical questions; and to conduct a formative assessment of the usability, usefulness, efficiency, and impact of the CKS prototype on physicians' perceived decision quality compared with standard search of UpToDate and PubMed. MATERIALS AND METHODS: Mixed-methods observations of the interactions of 10 physicians with the CKS prototype vs. standard search in an effort to solve clinical problems posed as case vignettes. RESULTS: The CKS tool automatically summarizes patient-specific and actionable clinical recommendations from PubMed (high quality randomized controlled trials and systematic reviews) and UpToDate. Two thirds of the study participants completed 15 out of 17 usability tasks. The median time to task completion was less than 10s for 12 of the 17 tasks. The difference in search time between the CKS and standard search was not significant (median=4.9 vs. 4.5m in). Physician's perceived decision quality was significantly higher with the CKS than with manual search (mean=16.6 vs. 14.4; p=0.036). CONCLUSIONS: The CKS prototype was well-accepted by physicians both in terms of usability and usefulness. Physicians perceived better decision quality with the CKS prototype compared to standard search of PubMed and UpToDate within a similar search time. Due to the formative nature of this study and a small sample size, conclusions regarding efficiency and efficacy are exploratory.

Automatic identification of comparative effectiveness research from medline citations to support clinicians' treatment information needs.

UNLABELLED: Online knowledge resources such as Medline can address most clinicians' patient care information needs. Yet, significant barriers, notably lack of time, limit the use of these sources at the point of care. The most common information needs raised by clinicians are treatment-related. Comparative effectiveness studies allow clinicians to consider multiple treatment alternatives for a particular problem. Still, solutions are needed to enable efficient and effective consumption of comparative effectiveness research at the point of care. OBJECTIVE: Design and assess an algorithm for automatically identifying comparative effectiveness studies and extracting the interventions investigated in these studies. METHODS: The algorithm combines semantic natural language processing, Medline citation metadata, and machine learning techniques. We assessed the algorithm in a case study of treatment alternatives for depression. RESULTS: Both precision and recall for identifying comparative studies was 0.83. A total of 86% of the interventions extracted perfectly or partially matched the gold standard. CONCLUSION: Overall, the algorithm achieved reasonable performance. The method provides building blocks for the automatic summarization of comparative effectiveness research to inform point of care decision-making.

Computer-facilitated review of electronic medical records reliably identifies emergency department interventions in older adults.

OBJECTIVES: An estimated 14% to 25% of all scientific studies in peer-reviewed emergency medicine (EM) journals are medical records reviews. The majority of the chart reviews in these studies are performed manually, a process that is both time-consuming and error-prone. Computer-based text search engines have the potential to enhance chart reviews of electronic emergency department (ED) medical records. The authors compared the efficiency and accuracy of a computer-facilitated medical record review of ED clinical records of geriatric patients with a traditional manual review of the same data and describe the process by which this computer-facilitated review was completed. METHODS: Clinical data from consecutive ED patients age 65 years or older were collected retrospectively by manual and computer-facilitated medical record review. The frequency of three significant ED interventions in older adults was determined using each method. Performance characteristics of each search method, including sensitivity and positive predictive value, were determined, and the overall sensitivities of the two search methods were compared using McNemar's test. RESULTS: For 665 patient visits, there were 49 (7.4%) Foley catheters placed, 36 (5.4%) sedative medications administered, and 15 (2.3%) patients who received positive pressure ventilation. The computer-facilitated review identified more of the targeted procedures (99 of 100, 99%), compared to manual review (74 of 100 procedures, 74%; p < 0.0001). CONCLUSIONS: A practical, non-resource-intensive, computer-facilitated free-text medical record review was completed and was more efficient and accurate than manually reviewing ED records.

Automatically extracting sentences from Medline citations to support clinicians' information needs.

OBJECTIVE: Online health knowledge resources contain answers to most of the information needs raised by clinicians in the course of care. However, significant barriers limit the use of these resources for decision-making, especially clinicians' lack of time. In this study we assessed the feasibility of automatically generating knowledge summaries for a particular clinical topic composed of relevant sentences extracted from Medline citations. METHODS: The proposed approach combines information retrieval and semantic information extraction techniques to identify relevant sentences from Medline abstracts. We assessed this approach in two case studies on the treatment alternatives for depression and Alzheimer's disease. RESULTS: A total of 515 of 564 (91.3%) sentences retrieved in the two case studies were relevant to the topic of interest. About one-third of the relevant sentences described factual knowledge or a study conclusion that can be used for supporting information needs at the point of care. CONCLUSIONS: The high rate of relevant sentences is desirable, given that clinicians' lack of time is one of the main barriers to using knowledge resources at the point of care. Sentence rank was not significantly associated with relevancy, possibly due to most sentences being highly relevant. Sentences located closer to the end of the abstract and sentences with treatment and comparative predications were likely to be conclusive sentences. Our proposed technical approach to helping clinicians meet their information needs is promising. The approach can be extended for other knowledge resources and information need types.