Understanding the Information Needs and Context of Trauma Handoffs to Design Automated Sensing Clinical Documentation Technologies: Qualitative Mixed-Method Study of Military and Civilian Cases.

BACKGROUND: Current methods of communication between the point of injury and receiving medical facilities rely on verbal communication, supported by brief notes and the memory of the field medic. This communication can be made more complete and reliable with technologies that automatically document the actions of field medics. However, designing state-of-the-art technology for military field personnel and civilian first responders is challenging due to the barriers researchers face in accessing the environment and understanding situated actions and cognitive models employed in the field. OBJECTIVE: To identify design insights for an automated sensing clinical documentation (ASCD) system, we sought to understand what information is transferred in trauma cases between prehospital and hospital personnel, and what contextual factors influence the collection, management, and handover of information in trauma cases, in both military and civilian cases. METHODS: Using a multi-method approach including video review and focus groups, we developed an understanding of the information needs of trauma handoffs and the context of field documentation to inform the design of an automated sensing documentation system that uses wearables, cameras, and environmental sensors to passively infer clinical activity and automatically produce documentation. RESULTS: Comparing military and civilian trauma documentation and handoff, we found similarities in the types of data collected and the prioritization of information. We found that military environments involved many more contextual factors that have implications for design, such as the physical environment (eg, heat, lack of lighting, lack of power) and the potential for active combat and triage, creating additional complexity. CONCLUSIONS: An ineffectiveness of communication is evident in both the civilian and military worlds. We used multiple methods of inquiry to study the information needs of trauma care and handoff, and the context of medical work in the field. Our findings informed the design and evaluation of an automated documentation tool. The data illustrated the need for more accurate recordkeeping, specifically temporal aspects, during transportation, and characterized the environment in which field testing of the developed tool will take place. The employment of a systems perspective in this project produced design insights that our team would not have identified otherwise. These insights created exciting and interesting challenges for the technical team to resolve.

Crowdsourcing to delineate skin affected by chronic graft-vs-host disease.

BACKGROUND: Estimating the extent of affected skin is an important unmet clinical need both for research and practical management in many diseases. In particular, cutaneous burden of chronic graft-vs-host disease (cGVHD) is a primary outcome in many trials. Despite advances in artificial intelligence and 3D photography, progress toward reliable automated techniques is hindered by limited expert time to delineate cGVHD patient images. Crowdsourcing may have potential to provide the requisite expert-level data. MATERIALS AND METHODS: Forty-one three-dimensional photographs of three cutaneous cGVHD patients were delineated by a board-certified dermatologist. 410 two-dimensional projections of the raw photos were each annotated by seven crowd workers, whose consensus performance was compared to the expert. RESULTS: The consensus delineation by four of seven crowd workers achieved the highest agreement with the expert, measured by a median Dice index of 0.7551 across all 410 images, outperforming even the best worker from the crowd (Dice index 0.7216). For their internal agreement, crowd workers achieved a median Fleiss's kappa of 0.4140 across the images. The time a worker spent marking an image had only weak correlation with the surface area marked, and very low correlation with accuracy. Percent of pixels selected by the consensus exhibited good correlation (Pearson R = 0.81) with the patient's affected surface area. CONCLUSION: Crowdsourcing may be an efficient method for obtaining demarcations of affected skin, on par with expert performance. Crowdsourced data generally agreed with the current clinical standard of percent body surface area to assess cGVHD severity in the skin.

Differences in gastric carcinoma microenvironment stratify according to EBV infection intensity: implications for possible immune adjuvant therapy.

Epstein-Barr virus (EBV) is associated with roughly 10% of gastric carcinomas worldwide (EBVaGC). Although previous investigations provide a strong link between EBV and gastric carcinomas, these studies were performed using selected EBV gene probes. Using a cohort of gastric carcinoma RNA-seq data sets from The Cancer Genome Atlas (TCGA), we performed a quantitative and global assessment of EBV gene expression in gastric carcinomas and assessed EBV associated cellular pathway alterations. EBV transcripts were detected in 17% of samples but these samples varied significantly in EBV coverage depth. In four samples with the highest EBV coverage (hiEBVaGC - high EBV associated gastric carcinoma), transcripts from the BamHI A region comprised the majority of EBV reads. Expression of LMP2, and to a lesser extent, LMP1 were also observed as was evidence of abortive lytic replication. Analysis of cellular gene expression indicated significant immune cell infiltration and a predominant IFNG response in samples expressing high levels of EBV transcripts relative to samples expressing low or no EBV transcripts. Despite the apparent immune cell infiltration, high levels of the cytotoxic T-cell (CTL) and natural killer (NK) cell inhibitor, IDO1, was observed in the hiEBVaGCs samples suggesting an active tolerance inducing pathway in this subgroup. These results were confirmed in a separate cohort of 21 Vietnamese gastric carcinoma samples using qRT-PCR and on tissue samples using in situ hybridization and immunohistochemistry. Lastly, a panel of tumor suppressors and candidate oncogenes were expressed at lower levels in hiEBVaGC versus EBV-low and EBV-negative gastric cancers suggesting the direct regulation of tumor pathways by EBV.

Detection of murine leukemia virus in the Epstein-Barr virus-positive human B-cell line JY, using a computational RNA-Seq-based exogenous agent detection pipeline, PARSES.

Many cell lines commonly used for biological studies have been found to harbor exogenous agents such as the human tumor viruses Epstein-Barr virus (EBV) and human papillomavirus. Nevertheless, broad-based, unbiased approaches to globally assess the presence of ectopic organisms within cell model systems have not previously been available. We reasoned that high-throughput sequencing should provide unparalleled insights into the microbiomes of tissue culture cell systems. Here we have used our RNA-seq analysis pipeline, PARSES (Pipeline for Analysis of RNA-Seq Exogenous Sequences), to investigate the presence of ectopic organisms within two EBV-positive B-cell lines commonly used by EBV researchers. Sequencing data sets from both the Akata and JY B-cell lines were found to contain reads for EBV, and the JY data set was found to also contain reads from the murine leukemia virus (MuLV). Further investigation revealed that MuLV transcription in JY cells is highly active. We also identified a number of MuLV alternative splicing events, and we uncovered evidence of APOBEC3G (apolipoprotein B mRNA-editing enzyme, catalytic polypeptide-like 3G)-dependent DNA editing. Finally, reverse transcription-PCR analysis showed the presence of MuLV in three other human B-cell lines (DG75, Ramos, and P3HR1 Cl.13) commonly used by investigators in the Epstein-Barr virus field. We believe that a thorough examination of tissue culture microbiomes using RNA-seq/PARSES-like approaches is critical for the appropriate utilization of these systems in biological studies.

Crowdsourcing Skin Demarcations of Chronic Graft-Versus-Host Disease in Patient Photographs: Training Versus Performance Study.

BACKGROUND: Chronic graft-versus-host disease (cGVHD) is a significant cause of long-term morbidity and mortality in patients after allogeneic hematopoietic cell transplantation. Skin is the most commonly affected organ, and visual assessment of cGVHD can have low reliability. Crowdsourcing data from nonexpert participants has been used for numerous medical applications, including image labeling and segmentation tasks. OBJECTIVE: This study aimed to assess the ability of crowds of nonexpert raters-individuals without any prior training for identifying or marking cGHVD-to demarcate photos of cGVHD-affected skin. We also studied the effect of training and feedback on crowd performance. METHODS: Using a Canfield Vectra H1 3D camera, 360 photographs of the skin of 36 patients with cGVHD were taken. Ground truth demarcations were provided in 3D by a trained expert and reviewed by a board-certified dermatologist. In total, 3000 2D images (projections from various angles) were created for crowd demarcation through the DiagnosUs mobile app. Raters were split into high and low feedback groups. The performances of 4 different crowds of nonexperts were analyzed, including 17 raters per image for the low and high feedback groups, 32-35 raters per image for the low feedback group, and the top 5 performers for each image from the low feedback group. RESULTS: Across 8 demarcation competitions, 130 raters were recruited to the high feedback group and 161 to the low feedback group. This resulted in a total of 54,887 individual demarcations from the high feedback group and 78,967 from the low feedback group. The nonexpert crowds achieved good overall performance for segmenting cGVHD-affected skin with minimal training, achieving a median surface area error of less than 12% of skin pixels for all crowds in both the high and low feedback groups. The low feedback crowds performed slightly poorer than the high feedback crowd, even when a larger crowd was used. Tracking the 5 most reliable raters from the low feedback group for each image recovered a performance similar to that of the high feedback crowd. Higher variability between raters for a given image was not found to correlate with lower performance of the crowd consensus demarcation and cannot therefore be used as a measure of reliability. No significant learning was observed during the task as more photos and feedback were seen. CONCLUSIONS: Crowds of nonexpert raters can demarcate cGVHD images with good overall performance. Tracking the top 5 most reliable raters provided optimal results, obtaining the best performance with the lowest number of expert demarcations required for adequate training. However, the agreement amongst individual nonexperts does not help predict whether the crowd has provided an accurate result. Future work should explore the performance of crowdsourcing in standard clinical photos and further methods to estimate the reliability of consensus demarcations.

Developing Ethics and Equity Principles, Terms, and Engagement Tools to Advance Health Equity and Researcher Diversity in AI and Machine Learning: Modified Delphi Approach.

BACKGROUND: Artificial intelligence (AI) and machine learning (ML) technology design and development continues to be rapid, despite major limitations in its current form as a practice and discipline to address all sociohumanitarian issues and complexities. From these limitations emerges an imperative to strengthen AI and ML literacy in underserved communities and build a more diverse AI and ML design and development workforce engaged in health research. OBJECTIVE: AI and ML has the potential to account for and assess a variety of factors that contribute to health and disease and to improve prevention, diagnosis, and therapy. Here, we describe recent activities within the Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) Ethics and Equity Workgroup (EEWG) that led to the development of deliverables that will help put ethics and fairness at the forefront of AI and ML applications to build equity in biomedical research, education, and health care. METHODS: The AIM-AHEAD EEWG was created in 2021 with 3 cochairs and 51 members in year 1 and 2 cochairs and ~40 members in year 2. Members in both years included AIM-AHEAD principal investigators, coinvestigators, leadership fellows, and research fellows. The EEWG used a modified Delphi approach using polling, ranking, and other exercises to facilitate discussions around tangible steps, key terms, and definitions needed to ensure that ethics and fairness are at the forefront of AI and ML applications to build equity in biomedical research, education, and health care. RESULTS: The EEWG developed a set of ethics and equity principles, a glossary, and an interview guide. The ethics and equity principles comprise 5 core principles, each with subparts, which articulate best practices for working with stakeholders from historically and presently underrepresented communities. The glossary contains 12 terms and definitions, with particular emphasis on optimal development, refinement, and implementation of AI and ML in health equity research. To accompany the glossary, the EEWG developed a concept relationship diagram that describes the logical flow of and relationship between the definitional concepts. Lastly, the interview guide provides questions that can be used or adapted to garner stakeholder and community perspectives on the principles and glossary. CONCLUSIONS: Ongoing engagement is needed around our principles and glossary to identify and predict potential limitations in their uses in AI and ML research settings, especially for institutions with limited resources. This requires time, careful consideration, and honest discussions around what classifies an engagement incentive as meaningful to support and sustain their full engagement. By slowing down to meet historically and presently underresourced institutions and communities where they are and where they are capable of engaging and competing, there is higher potential to achieve needed diversity, ethics, and equity in AI and ML implementation in health research.

Feasibility and Short-Term Effects of a Multi-Component Emergency Department Blood Pressure Intervention: A Pilot Randomized Trial.

Background Emergency department (ED) visits can be opportunities to address uncontrolled hypertension. We sought to compare short-term blood pressure measures between the Vanderbilt Emergency Room Bundle (VERB) intervention and usual care plus education. Methods and Results We conducted a randomized trial of 206 adult patients with hypertension and elevated systolic blood pressure (SBP) presenting to 2 urban emergency departments in Tennessee, USA. The VERB intervention included educational materials, a brief motivational interview, pillbox, primary care engagement letter, pharmacy resources, and 45 days of informational and reminder text messages. The education arm received a hypertension pamphlet. After 78 participants were enrolled, text messages requested confirmation of receipt. The primary clinical outcome was 30-day SBP. The median 30-day SBP was 122 and 126 mm Hg in the VERB and education arms, respectively. We estimated the mean 30-day SBP to be 3.98 mm Hg lower in the VERB arm (95% CI, -2.44 to 10.4; P=0.22). Among participants enrolled after text messages were adapted, the respective median SBPs were 121 and 130 mm Hg, and we estimated the mean 30-day SBP to be 8.57 mm Hg lower in the VERB arm (95% CI, 0.98‒16.2; P=0.027). In this subgroup, the median response rate to VERB text messages was 56% (interquartile range, [26%‒80%]). Conclusions This pilot study demonstrated feasibility and found an improvement in SBP for the subgroup for whom interactive messages were featured. Future studies should evaluate the role of interactive text messaging as part of a comprehensive emergency department intervention to improve blood pressure control. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT02672787.

User preferences for and engagement with text messages to support antihypertensive medication adherence: Findings from a pilot study evaluating an emergency department-based behavioral intervention.

OBJECTIVE: We examined users' preferences for and engagement with text messages delivered as part of an emergency department (ED)-based intervention to improve antihypertensive medication adherence. METHODS: We recruited ED patients with elevated blood pressure for a pilot randomized trial evaluating a medication adherence intervention with text messages. Intervention participants chose text content and frequency, received texts for 45 days, and completed a feedback survey. We defined engagement via responses to texts. We examined participant characteristics associated with text preferences, engagement, and feedback. RESULTS: Participants (N = 101) were 57% female and 46% non-White. Most participants (71%) chose to receive both reminder and informational texts; 94% chose reminder texts once per day and 97% chose informational texts three times per week. Median text message response rate was 56% (IQR 26-80%). Participants who were Black (p < 0.01), had lower income (p = 0.03), or had lower medication adherence (p < 0.01) rated the program as more helpful and wanted additional functionalities for adherence support. CONCLUSIONS AND PRACTICE IMPLICATIONS: While overall engagement was modest, participants at risk of worse health outcomes expressed more value and interest in the program. Findings inform the design of text messaging interventions for antihypertensive medication adherence and support targeting vulnerable patients to reduce health disparities. CLINICAL TRIALS REGISTRATION: NCT02672787.

Extracting Medical Information from Paper COVID-19 Assessment Forms.

OBJECTIVE: This study examines the validity of optical mark recognition, a novel user interface, and crowdsourced data validation to rapidly digitize and extract data from paper COVID-19 assessment forms at a large medical center. METHODS: An optical mark recognition/optical character recognition (OMR/OCR) system was developed to identify fields that were selected on 2,814 paper assessment forms, each with 141 fields which were used to assess potential COVID-19 infections. A novel user interface (UI) displayed mirrored forms showing the scanned assessment forms with OMR results superimposed on the left and an editable web form on the right to improve ease of data validation. Crowdsourced participants validated the results of the OMR system. Overall error rate and time taken to validate were calculated. A subset of forms was validated by multiple participants to calculate agreement between participants. RESULTS: The OMR/OCR tools correctly extracted data from scanned forms fields with an average accuracy of 70% and median accuracy of 78% when the OMR/OCR results were compared with the results from crowd validation. Scanned forms were crowd-validated at a mean rate of 157 seconds per document and a volume of approximately 108 documents per day. A randomly selected subset of documents was reviewed by multiple participants, producing an interobserver agreement of 97% for documents when narrative-text fields were included and 98% when only Boolean and multiple-choice fields were considered. CONCLUSION: Due to the COVID-19 pandemic, it may be challenging for health care workers wearing personal protective equipment to interact with electronic health records. The combination of OMR/OCR technology, a novel UI, and crowdsourcing data-validation processes allowed for the efficient extraction of a large volume of paper medical documents produced during the COVID-19 pandemic.

Visualization of aggregate perioperative data improves anesthesia case planning: A randomized, cross-over trial.

STUDY OBJECTIVE: A challenge in reducing unwanted care variation is effectively managing the wide variety of performed surgical procedures. While an organization may perform thousands of types of cases, privacy and logistical constraints prevent review of previous cases to learn about prior practices. To bridge this gap, we developed a system for extracting key data from anesthesia records. Our objective was to determine whether usage of the system would improve case planning performance for anesthesia residents. DESIGN: Randomized, cross-over trial. SETTING: Vanderbilt University Medical Center. MEASUREMENTS: We developed a web-based, data visualization tool for reviewing de-identified anesthesia records. First year anesthesia residents were recruited and performed simulated case planning tasks (e.g., selecting an anesthetic type) across six case scenarios using a randomized, cross-over design after a baseline assessment. An algorithm scored case planning performance based on care components selected by residents occurring frequently among prior anesthetics, which was scored on a 0-4 point scale. Linear mixed effects regression quantified the tool effect on the average performance score, adjusting for potential confounders. MAIN RESULTS: We analyzed 516 survey questionnaires from 19 residents. The mean performance score was 2.55 ± SD 0.32. Utilization of the tool was associated with an average score improvement of 0.120 points (95% CI 0.060 to 0.179; p < 0.001). Additionally, a 0.055 point improvement due to the "learning effect" was observed from each assessment to the next (95% CI 0.034 to 0.077; p < 0.001). Assessment score was also significantly associated with specific case scenarios (p < 0.001). CONCLUSIONS: This study demonstrated the feasibility of developing of a clinical data visualization system that aggregated key anesthetic information and found that the usage of tools modestly improved residents' performance in simulated case planning.

Emergency Clinical Procedure Detection With Deep Learning.

Information about a patient's state is critical for hospitals to provide timely care and treatment. Prior work on improving the information flow from emergency medical services (EMS) to hospitals demonstrated the potential of using automated algorithms to detect clinical procedures. However, prior work has not made effective use of video sources that might be available during patient care. In this paper we explore the use convolutional neural networks (CNNs) on raw video data to determine how well video data alone can automatically identify clinical procedures. We apply multiple deep learning models to this problem, with significant variation in results. Our findings indicate performance improvements compared to prior work, but also indicate a need for more training data to reach clinically deployable levels of success.

Association of Elevated Blood Pressure in the Emergency Department With Chronically Elevated Blood Pressure.

Background Emergency department (ED) visits for hypertension are rising, but the importance of elevated blood pressure (BP) measured during the ED visit is controversial. We evaluated the relationship between ED BP and mean BP over the subsequent year. Methods and Results We performed a retrospective cohort study from January 1, 2010 to December 31, 2013 of 8105 adult patients who made 1 visit to an academic medical center ED with ≥2 ED BPs and ≥2 BPs measured in the subsequent year. The primary exposure was lowest ED systolic BP. The primary outcome was mean systolic BP ≥140 mm Hg over the year following the index ED visit. Diastolic BP was examined as a secondary exposure and outcome. Multiple logistic regression was performed adjusting for several covariates, with interaction terms for hypertension diagnosis, ED disposition, pain-related ED chief complaint, and sex. Patients whose lowest ED systolic BP was 140 to 159 mm Hg had an adjusted odds ratio of having a mean SBP ≥140 mm Hg in the subsequent year of 10.9 (95% CI, 7.6-15.6). Patients without diagnosed hypertension and ED BP 140/90 to 159/99 mm Hg were more likely to have elevated BP in the following year. Hospitalization increased the likelihood of persistently elevated systolic BP but not diastolic BP. There was no effect modification by pain-related ED complaint. Conclusions When ED BP is consistently elevated, BP is highly likely to remain elevated in the subsequent year, regardless of pain, and particularly among patients without diagnosed hypertension. Further research is needed to determine the optimal management of elevated ED BP.

CYP2C19 Phenotype and Risk of Proton Pump Inhibitor-Associated Infections.

OBJECTIVES: Proton pump inhibitors (PPIs) are often used in pediatrics to treat common gastrointestinal disorders, and there are growing concerns for infectious adverse events. Because CYP2C19 inactivates PPIs, genetic variants that increase CYP2C19 function may decrease PPI exposure and infections. We tested the hypothesis that CYP2C19 metabolizer phenotypes are associated with infection event rates in children exposed to PPIs. METHODS: This retrospective biorepository cohort study included individuals aged 0 to 36 months at the time of PPI exposure. Respiratory tract and gastrointestinal tract infection events were identified by using International Classification of Diseases codes in the year after the first PPI mention. Variants defining CYP2C19 *2, *3, *4, *8, *9, and *17 were genotyped, and all individuals were classified as CYP2C19 poor or intermediate, normal metabolizers (NMs), or rapid or ultrarapid metabolizers (RM/UMs). Infection rates were compared by using univariate and multivariate analyses. RESULTS: In all, 670 individuals were included (median age 7 months; 44% girls). CYP2C19 NMs (n = 267; 40%) had a higher infection rate than RM/UMs (n = 220; 33%; median 2 vs 1 infections per person per year; P = .03). There was no difference between poor or intermediate (n = 183; 27%) and NMs. In multivariable analysis of NMs and RM/UMs adjusting for age, sex, PPI dose, and comorbidities, CYP2C19 metabolizer status remained a significant risk factor for infection events (odds ratio 0.70 [95% confidence interval 0.50-0.97] for RM/UMs versus NMs). CONCLUSIONS: PPI therapy is associated with higher infection rates in children with normal CYP2C19 function than in those with increased CYP2C19 function, highlighting this adverse effect of PPI therapy and the relevance of CYP2C19 genotypes to PPI therapeutic decision-making.

Automatic Clinical Procedure Detection for Emergency Services.

Understanding a patient's state is critical to providing optimal care. However, information loss occurs during patient hand-offs (e.g., emergency services (EMS) transferring patient care to a receiving hospital), which hinders care quality. Augmenting the information flow from an EMS vehicle to a receiving hospital may reduce information loss and improve patient outcomes. Such augmentation requires a noninvasive system that can automatically recognize clinical procedures being performed and send near real-time information to a receiving hospital. An automatic clinical procedure detection system that uses wearable sensors, video, and machine-learning to recognize clinical procedures within a controlled environment is presented. The system demonstrated how contextual information and a majority vote method can substantially improve procedure recognition accuracy. Future work concerning computer vision techniques and deep learning are discussed.

Copy-and-Paste in Medical Student Notes: Extent, Temporal Trends, and Relationship to Scholastic Performance.

BACKGROUND: Medical students may observe and subsequently perpetuate redundancy in clinical documentation, but the degree of redundancy in student notes and whether there is an association with scholastic performance are unknown. OBJECTIVES: This study sought to quantify redundancy, defined generally as the proportion of similar text between two strings, in medical student notes and evaluate the relationship between note redundancy and objective indicators of student performance. METHODS: Notes generated by medical students rotating through their medicine clerkship during a single academic year at our institution were analyzed. A student-patient interaction (SPI) was defined as a history and physical and at least two contiguous progress notes authored by the same student during a single patient's hospitalization. For some students, SPI pairs were available from early and late in the clerkship. Redundancy between analogous sections of consecutive notes was calculated on a 0 to 100% scale and was derived from edit distance, the number of changes needed to transform one text string into another. Indicators of student performance included United States Medical Licensing Exam (USMLE) scores. RESULTS: Ninety-four single SPIs and 58 SPI pairs were analyzed. Redundancy in the assessment/plan section was high (40%) and increased within individual SPIs (to 60%; p < 0.001) and between SPI pairs over the course of the clerkship (by 30-40%; p < 0.001). Students in the lowest tertile of USMLE step II clinical knowledge scores had higher redundancy in the assessment/plan section than their classmates (67 ± 24% vs. 38 ± 22%; p = 0.002). CONCLUSION: During the medicine clerkship, the assessment/plan section of medical student notes became more redundant over a patient's hospital course and as students gained clinical experience. These trends may be indicative of deficiencies in clinical knowledge or reasoning, as evidenced by performance on some standardized evaluations.

Feasibility Assessment of a Pre-Hospital Automated Sensing Clinical Documentation System.

Clinical documentation in the pre-hospital setting is challenged by limited resources and fast-paced, high-acuity. Military and civilian medics are responsible for performing procedures and treatments to stabilize the patient, while transporting the injured to a trauma facility. Upon arrival, medics typically give a verbal report from memory or informal source of documentation such as a glove or piece of tape. The development of an automated documentation system would increase the accuracy and amount of information that is relayed to the receiving physicians. This paper discusses the 12-week deployment of an Automated Sensing Clinical Documentation (ASCD) system among the Nashville Fire Department EMS paramedics. The paper examines the data collection methods, operational challenges, and perceptions surrounding real-life deployment of the system. Our preliminary results suggest that the ASCD system is feasible for use in the pre-hospital setting, and it revealed several barriers and their solutions.

Advancing In-Hospital Clinical Deterioration Prediction Models.

BACKGROUND: Early warning systems lack robust evidence that they improve patients' outcomes, possibly because of their limitation of predicting binary rather than time-to-event outcomes. OBJECTIVES: To compare the prediction accuracy of 2 statistical modeling strategies (logistic regression and Cox proportional hazards regression) and 2 machine learning strategies (random forest and random survival forest) for in-hospital cardiopulmonary arrest. METHODS: Retrospective cohort study with prediction model development from deidentified electronic health records at an urban academic medical center. RESULTS: The classification models (logistic regression and random forest) had statistical recall and precision similar to or greater than those of the time-to-event models (Cox proportional hazards regression and random survival forest). However, the time-to-event models provided predictions that could potentially better indicate to clinicians whether and when a patient is likely to experience cardiopulmonary arrest. CONCLUSIONS: As early warning scoring systems are refined, they must use the best analytical methods that both model the underlying phenomenon and provide an understandable prediction.

A Crowdsourcing Framework for Medical Data Sets.

Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to various concerns. Major challenges in building and deploying a crowdsourcing system for medical data include, but are not limited to: managing access rights to sensitive data and ensuring data privacy controls are enforced; identifying workers with the necessary expertise to analyze complex information; and efficiently retrieving relevant information in massive data sets. In this paper, we introduce a crowdsourcing framework to support the annotation of medical data sets. We further demonstrate a workflow for crowdsourcing clinical chart reviews including (1) the design and decomposition of research questions; (2) the architecture for storing and displaying sensitive data; and (3) the development of tools to support crowd workers in quickly analyzing information from complex data sets.