An "All-Data-on-Hand" Deep Learning Model to Predict Hospitalization for Diabetic Ketoacidosis in Youth With Type 1 Diabetes: Development and Validation Study.

BACKGROUND: Although prior research has identified multiple risk factors for diabetic ketoacidosis (DKA), clinicians continue to lack clinic-ready models to predict dangerous and costly episodes of DKA. We asked whether we could apply deep learning, specifically the use of a long short-term memory (LSTM) model, to accurately predict the 180-day risk of DKA-related hospitalization for youth with type 1 diabetes (T1D). OBJECTIVE: We aimed to describe the development of an LSTM model to predict the 180-day risk of DKA-related hospitalization for youth with T1D. METHODS: We used 17 consecutive calendar quarters of clinical data (January 10, 2016, to March 18, 2020) for 1745 youths aged 8 to 18 years with T1D from a pediatric diabetes clinic network in the Midwestern United States. The input data included demographics, discrete clinical observations (laboratory results, vital signs, anthropometric measures, diagnosis, and procedure codes), medications, visit counts by type of encounter, number of historic DKA episodes, number of days since last DKA admission, patient-reported outcomes (answers to clinic intake questions), and data features derived from diabetes- and nondiabetes-related clinical notes via natural language processing. We trained the model using input data from quarters 1 to 7 (n=1377), validated it using input from quarters 3 to 9 in a partial out-of-sample (OOS-P; n=1505) cohort, and further validated it in a full out-of-sample (OOS-F; n=354) cohort with input from quarters 10 to 15. RESULTS: DKA admissions occurred at a rate of 5% per 180-days in both out-of-sample cohorts. In the OOS-P and OOS-F cohorts, the median age was 13.7 (IQR 11.3-15.8) years and 13.1 (IQR 10.7-15.5) years; median glycated hemoglobin levels at enrollment were 8.6% (IQR 7.6%-9.8%) and 8.1% (IQR 6.9%-9.5%); recall was 33% (26/80) and 50% (9/18) for the top-ranked 5% of youth with T1D; and 14.15% (213/1505) and 12.7% (45/354) had prior DKA admissions (after the T1D diagnosis), respectively. For lists rank ordered by the probability of hospitalization, precision increased from 33% to 56% to 100% for positions 1 to 80, 1 to 25, and 1 to 10 in the OOS-P cohort and from 50% to 60% to 80% for positions 1 to 18, 1 to 10, and 1 to 5 in the OOS-F cohort, respectively. CONCLUSIONS: The proposed LSTM model for predicting 180-day DKA-related hospitalization was valid in this sample. Future research should evaluate model validity in multiple populations and settings to account for health inequities that may be present in different segments of the population (eg, racially or socioeconomically diverse cohorts). Rank ordering youth by probability of DKA-related hospitalization will allow clinics to identify the most at-risk youth. The clinical implication of this is that clinics may then create and evaluate novel preventive interventions based on available resources.

Blue Circle Health: A Novel Patient-Centered Model of Health Care Delivery for Low-Income Patients With Type 1 Diabetes.

Analog insulins, insulin pumps, and continuous glucose monitors (CGM) have revolutionized type 1 diabetes (T1D) treatment over the last 50 years. Nevertheless, less than 20% of patients in the United States reach guideline-based HbA1c targets. The dysfunctional delivery of U.S. health care has further worsened glycemic outcomes among structurally disadvantaged groups such as non-Hispanic Black and low-income populations. Administrative complexities resulting from mixed insurance coverage and delivery systems, incongruity between effective policies and reimbursement, structural racism, and implicit biases have led to high diabetes care-related costs, provider scarcity and burnout, and patient diabetes distress. The Extension for Community Healthcare Outcomes (ECHO) Diabetes tele-education outreach model was created to increase self-efficacy among primary care providers through a combination of weekly didactic sessions led by a team of diabetes experts and access to community-based peer coaches. As an evolution of ECHO Diabetes, Blue Circle Health has been established as a philanthropically funded health care delivery system, using a whole-person, individualized approach to T1D care for adults living in underserved communities. The program will provide direct-to-patient telehealth services, including diabetes education, management, and related psychological care regardless of ability to pay. Community-based diabetes support coaches will serve as the primary point of contact, or guide on the "Blue Circle Health Member Journey." Access to needed insulins, supplies, and CGMs will be provided at no cost to the individual. Through a continuous learning and improvement model, a person-centered, equitable, accessible, and effective health care delivery model will be built for people living with T1D.

Institutional variation in surgical care for early-stage breast cancer at community hospitals.

BACKGROUND: There is significant institutional variation in the surgical care of breast cancer, and this may reflect access to services and resultant physician practice patterns. In previous studies, specialty care has been associated with variation in the operative treatment of breast cancer but has not been evaluated in a community setting. This study investigates these issues in a cohort of 59 community hospitals in the United States. MATERIALS AND METHODS: Data on patients receiving an operation for breast cancer (2006-2009) in a large, geographically diverse cohort of hospitals were obtained. Administrative data, autoabstracted cancer-specific variables from free text, and multiple other data sets were combined. Polymotous logistic regression with multilevel outcomes identified associations between these variables and surgical treatment. RESULTS: At 59 community hospitals, 4766 patients underwent breast conserving surgery (BCS), mastectomy, or mastectomy with reconstruction. The older patients were most likely to receive mastectomy alone, whereas the younger age group underwent more reconstruction (age <50), and BCS was most likely in patients aged 50-65. Surgical procedure also varied according to tumor characteristics. BCS was more likely at smaller hospitals, those with ambulatory surgery centers, and those located in nonmetropolitan areas. The likelihood of reconstruction doubled when there were more reconstructive surgeons in the health services area (P = 0.02). BCS was more likely when radiation oncology services were available within the hospital or network (P = 0.04). CONCLUSIONS: Interpretation of these results for practice redesign is not straightforward. Although access to specialty care is statistically associated with type of breast surgical procedure, clinical impact is limited. It may be more effective to target other aspects of care to ensure each patient receives treatment consistent with her individual preferences.

Feasibility of Automating Patient Acuity Measurement Using a Machine Learning Algorithm.

BACKGROUND AND PURPOSE: One method of determining nurse staffing is to match patient demand for nursing care (patient acuity) with available nursing staff. This pilot study explored the feasibility of automating acuity measurement using a machine learning algorithm. METHODS: Natural language processing combined with a machine learning algorithm was used to predict acuity levels based on electronic health record data. RESULTS: The algorithm was able to predict acuity relatively well. A main challenge was discordance among nurse raters of acuity in generating a gold standard of acuity before applying the machine learning algorithm. CONCLUSIONS: This pilot study tested applying machine learning techniques to acuity measurement and yielded a moderate level of performance. Higher agreement among the gold standard may yield higher performance in future studies.

Case misclassification in studies of spinal manipulation and arterial dissection.

BACKGROUND: Spinal manipulation has been associated with cervical arterial dissection and stroke but a causal relationship has been questioned by population-based studies. Earlier studies identified cases using International Classification of Diseases Ninth Revision (ICD-9) codes specific to anatomic stroke location rather than stroke etiology. We hypothesize that case misclassification occurred in these previous studies and an underestimation of the strength of the association. We also predicted that case misclassification would differ by patient age. METHODS: We identified cases in the Veterans Health Administration database using the same strategy as the prior studies. The electronic medical record was then screened for the word "dissection." The presence of atraumatic dissection was determined by medical record review by a neurologist. RESULTS: Of 3690 patients found by ICD-9 codes over a 30-month period, 414 (11.2%) had confirmed cervical artery dissection with a positive predictive value of 10.5% (95% confidence interval [CI] 9.6%-11.5%). The positive predictive value was higher in patients less than 45 years of age vs 45 years of age or older (41% vs 9%, P < .001). We reanalyzed a previous study, which reported no association between spinal manipulation and cervical artery dissection (odds ratio [OR] = 1.12, 95% CI .77-1.63) and recalculated an odds ratio of 2.15 (95% CI .98-4.69). For patients less than 45 years of age, the OR was 6.91 (95% CI 2.59-13.74). CONCLUSIONS: Prior studies grossly misclassified cases of cervical dissection and mistakenly dismissed a causal association with manipulation. Our study indicates that the OR for spinal manipulation exposure in cervical artery dissection is higher than previously reported.

Using the electronic medical record to identify community-acquired pneumonia: toward a replicable automated strategy.

BACKGROUND: Timely information about disease severity can be central to the detection and management of outbreaks of acute respiratory infections (ARI), including influenza. We asked if two resources: 1) free text, and 2) structured data from an electronic medical record (EMR) could complement each other to identify patients with pneumonia, an ARI severity landmark. METHODS: A manual EMR review of 2747 outpatient ARI visits with associated chest imaging identified x-ray reports that could support the diagnosis of pneumonia (kappa score  = 0.88 (95% CI 0.82∶0.93)), along with attendant cases with Possible Pneumonia (adds either cough, sputum, fever/chills/night sweats, dyspnea or pleuritic chest pain) or with Pneumonia-in-Plan (adds pneumonia stated as a likely diagnosis by the provider). The x-ray reports served as a reference to develop a text classifier using machine-learning software that did not require custom coding. To identify pneumonia cases, the classifier was combined with EMR-based structured data and with text analyses aimed at ARI symptoms in clinical notes. RESULTS: 370 reference cases with Possible Pneumonia and 250 with Pneumonia-in-Plan were identified. The x-ray report text classifier increased the positive predictive value of otherwise identical EMR-based case-detection algorithms by 20-70%, while retaining sensitivities of 58-75%. These performance gains were independent of the case definitions and of whether patients were admitted to the hospital or sent home. Text analyses seeking ARI symptoms in clinical notes did not add further value. CONCLUSION: Specialized software development is not required for automated text analyses to help identify pneumonia patients. These results begin to map an efficient, replicable strategy through which EMR data can be used to stratify ARI severity.

Measuring use of evidence based psychotherapy for posttraumatic stress disorder.

To improve methods of estimating use of evidence-based psychotherapy for posttraumatic stress disorder in the Veteran's health administration, we evaluated administrative data and note text for patients newly enrolling in six VHA outpatient PTSD clinics in New England during the 2010 fiscal year (n = 1,924). Using natural language processing, we developed machine learning algorithms that mimic human raters in classifying note text. We met our targets for algorithm performance as measured by precision, recall, and F-measure. We found that 6.3 % of our study population received at least one session of evidence-based psychotherapy during the initial 6 months of treatment. Evidence-based psychotherapies appear to be infrequently utilized in VHA outpatient PTSD clinics in New England. Our method could support efforts to improve use of these treatments.

Automated identification of surveillance colonoscopy in inflammatory bowel disease using natural language processing.

BACKGROUND: Differentiating surveillance from non-surveillance colonoscopy for colorectal cancer in patients with inflammatory bowel disease (IBD) using electronic medical records (EMR) is important for practice improvement and research purposes, but diagnosis code algorithms are lacking. The automated retrieval console (ARC) is natural language processing (NLP)-based software that allows text-based document-level classification. AIMS: The purpose of this study was to test the feasibility and accuracy of ARC in identifying surveillance and non-surveillance colonoscopy in IBD using EMR. METHODS: We performed a split validation study of electronic reports of colonoscopy pathology for patients with IBD from the Michael E. DeBakey VA Medical Center. A gastroenterologist manually classified pathology reports as either derived from surveillance or non-surveillance colonoscopy. Pathology reports were randomly split into two sets: 70 % for algorithm derivation and 30 % for validation. An ARC generated classification model was applied to the validation set of pathology reports. The performance of the model was compared with manual classification for surveillance and non-surveillance colonoscopy. RESULTS: A total of 575 colonoscopy pathology reports were available on 195 IBD patients, of which 400 reports were designated as training and 175 as testing sets. Within the testing set, a total of 69 pathology reports were classified as surveillance by manual review, whereas the ARC model classified 66 reports as surveillance for a recall of 0.77, precision of 0.80, and specificity of 0.88. CONCLUSIONS: ARC was able to identify surveillance colonoscopy for IBD without customized software programming. NLP-based document-level classification may be used to differentiate surveillance from non-surveillance colonoscopy in IBD.

Using natural language processing on the free text of clinical documents to screen for evidence of homelessness among US veterans.

Information retrieval algorithms based on natural language processing (NLP) of the free text of medical records have been used to find documents of interest from databases. Homelessness is a high priority non-medical diagnosis that is noted in electronic medical records of Veterans in Veterans Affairs (VA) facilities. Using a human-reviewed reference standard corpus of clinical documents of Veterans with evidence of homelessness and those without, an open-source NLP tool (Automated Retrieval Console v2.0, ARC) was trained to classify documents. The best performing model based on document level work-flow performed well on a test set (Precision 94%, Recall 97%, F-Measure 96). Processing of a naïve set of 10,000 randomly selected documents from the VA using this best performing model yielded 463 documents flagged as positive, indicating a 4.7% prevalence of homelessness. Human review noted a precision of 70% for these flags resulting in an adjusted prevalence of homelessness of 3.3% which matches current VA estimates. Further refinements are underway to improve the performance. We demonstrate an effective and rapid lifecycle of using an off-the-shelf NLP tool for screening targets of interest from medical records.

Implementation of the Department of Veterans Affairs' first point-of-care clinical trial.

OBJECTIVES: The Massachusetts Veterans Epidemiology Research and Information Center in collaboration with the Stanford Center for Innovative Study Design set out to test the feasibility of a new method of evidence generation. The first pilot of a point-of-care clinical trial (POCCT), adding randomization and other study processes to an electronic medical record (EMR) system, was launched to compare the effectiveness of two insulin regimens. MATERIALS AND METHODS: Existing functionalities of the Veterans Affairs (VA) computerized patient record system (CPRS)/veterans health information systems and technology architecture (VISTA) were modified to support the activities of a randomized controlled trial including enrolment, randomization, and longitudinal data collection. RESULTS: The VA's CPRS/VISTA was successfully adapted to support the processes of a clinical trial and longitudinal study data are being collected from the medical record automatically. As of 30 June 2011, 55 of the 67 eligible patients approached received a randomized intervention. DISCUSSION: The design of CPRS/VISTA made integration of study workflows and data collection possible. Institutions and investigators considering similar designs must carefully map clinical workflows and clinical trial workflows to EMR capabilities. POCCT study teams are necessarily interdisciplinary and interdepartmental. As a result, executive sponsorship is critical. CONCLUSION: POCCT represent a promising new method for conducting clinical science. Much work is needed to understand better the optimal uses and designs for this new approach. Next steps include focus groups to measure patient and clinician perceptions, multisite deployment of the current pilot, and implementation of additional studies.

The association between height and prostate cancer grade in the Early Stage Prostate Cancer Cohort Study.

OBJECTIVE: We examined the relationship between height and prostate cancer grade. METHODS: The Early Stage Prostate Cancer Cohort Study is an observational cohort of 1,037 men diagnosed with early-stage prostate cancer, T(0-3)N(x)M(0). High-grade prostate cancer was defined as a biopsy Gleason score ≥ 7 (4 + 3). Logistic regression models were created to calculate odds ratios (OR) and 95% confidence intervals (CI) for the cross-sectional relationship between height and prostate cancer grade in the overall cohort and subpopulations. RESULTS: We identified 939 participants with a biopsy Gleason score. High-grade prostate cancer was diagnosed in 138 participants. Overall, participants in the highest quartile of height were more than twice as likely to have a Gleason score ≥ 7 (4 + 3) than participants in the lowest quartile of height, OR 2.14 (95% CI 1.11, 4.14), after multivariate adjustment. Participants in the highest quartile of height were more likely to be diagnosed with high-grade prostate cancer than participants in the lowest quartile of height among participants who were black, OR 8.00 (95% CI 1.99, 32.18), and participants who had diabetes mellitus, OR 5.09 (95% CI 1.30, 19.98). CONCLUSIONS: Height is associated with increased risk of high-grade prostate cancer overall and perhaps among certain subpopulations.

Automated concept-level information extraction to reduce the need for custom software and rules development.

OBJECTIVE: Despite at least 40 years of promising empirical performance, very few clinical natural language processing (NLP) or information extraction systems currently contribute to medical science or care. The authors address this gap by reducing the need for custom software and rules development with a graphical user interface-driven, highly generalizable approach to concept-level retrieval. MATERIALS AND METHODS: A 'learn by example' approach combines features derived from open-source NLP pipelines with open-source machine learning classifiers to automatically and iteratively evaluate top-performing configurations. The Fourth i2b2/VA Shared Task Challenge's concept extraction task provided the data sets and metrics used to evaluate performance. RESULTS: Top F-measure scores for each of the tasks were medical problems (0.83), treatments (0.82), and tests (0.83). Recall lagged precision in all experiments. Precision was near or above 0.90 in all tasks. Discussion With no customization for the tasks and less than 5 min of end-user time to configure and launch each experiment, the average F-measure was 0.83, one point behind the mean F-measure of the 22 entrants in the competition. Strong precision scores indicate the potential of applying the approach for more specific clinical information extraction tasks. There was not one best configuration, supporting an iterative approach to model creation. CONCLUSION: Acceptable levels of performance can be achieved using fully automated and generalizable approaches to concept-level information extraction. The described implementation and related documentation is available for download.

Statins and prostate cancer diagnosis and grade in a veterans population.

BACKGROUND: Although prostate cancer is commonly diagnosed, few risk factors for high-grade prostate cancer are known and few prevention strategies exist. Statins have been proposed as a possible treatment to prevent prostate cancer. METHODS: Using electronic and administrative files from the Veterans Affairs New England Healthcare System, we identified 55,875 men taking either a statin or antihypertensive medication. We used age- and multivariable-adjusted Cox proportional hazard models to calculate hazard ratios (HRs) and 95% confidence intervals (CIs) for prostate cancer incidence among patients taking statins (n = 41,078) compared with patients taking antihypertensive medications (n = 14,797). We performed similar analyses for all lipid parameters including total cholesterol examining each lipid parameter as a continuous variable and by quartiles. All statistical tests were two-sided. RESULTS: Compared with men taking an antihypertensive medication, statin users were 31% less likely (HR = 0.69, 95% CI = 0.52 to 0.90) to be diagnosed with prostate cancer. Furthermore, statin users were 14% less likely (HR = 0.86, 95% CI = 0.62 to 1.20) to be diagnosed with low-grade prostate cancer and 60% less likely (HR = 0.40, 95% CI = 0.24 to 0.65) to be diagnosed with high-grade prostate cancer compared with antihypertensive medication users. Increased levels of total cholesterol were also associated with both total (HR = 1.02, 95% CI = 1.00 to 1.05) and high-grade (HR = 1.06, 95% CI = 1.02 to 1.10) prostate cancer incidence but not with low-grade prostate cancer incidence (HR = 1.01, 95% CI = 0.98 to 1.04). CONCLUSIONS: Statin use is associated with statistically significantly reduced risk for total and high-grade prostate cancer, and increased levels of serum cholesterol are associated with higher risk for total and high-grade prostate cancer. These findings indicate that clinical trials of statins for prostate cancer prevention are warranted.

A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen.

BACKGROUND: Clinical trials are widely considered the gold standard in comparative effectiveness research (CER) but the high cost and complexity of traditional trials and concerns about generalizability to broad patient populations and general clinical practice limit their appeal. Unsuccessful implementation of CER results limits the value of even the highest quality trials. Planning for a trial comparing two standard strategies of insulin administration for hospitalized patients led us to develop a new method for a clinical trial designed to be embedded directly into the clinical care setting thereby lowering the cost, increasing the pragmatic nature of the overall trial, strengthening implementation, and creating an integrated environment of research-based care. PURPOSE: We describe a novel randomized clinical trial that uses the informatics and statistics infrastructure of the Veterans Affairs Healthcare System (VA) to illustrate one key component (called the point-of-care clinical trial - POC-CT) of a 'learning healthcare system,' and settles a clinical question of interest to the VA. METHODS: This study is an open-label, randomized trial comparing sliding scale regular insulin to a weight-based regimen for control of hyperglycemia, using the primary outcome length of stay, in non-ICU inpatients within the northeast region of the VA. All non-ICU patients who require in-hospital insulin therapy are eligible for the trial, and the VA's automated systems will be used to assess eligibility and present the possibility of randomization to the clinician at the point of care. Clinicians will indicate their approval for informed consent to be obtained by study staff. Adaptive randomization will assign up to 3000 patients, preferentially to the currently 'winning' strategy, and all care will proceed according to usual practices. Based on a Bayesian stopping rule, the study has acceptable frequentist operating characteristics (Type I error 6%, power 86%) against a 12% reduction of median length of stay from 5 to 4.4 days. The adaptive stopping rule promotes implementation of a successful treatment strategy. LIMITATIONS: Despite clinical equipoise, individual healthcare providers may have strong treatment preferences that jeopardize the success and implementation of the trial design, leading to low rates of randomization. Unblinded treatment assignment may bias results. In addition, generalization of clinical results to other healthcare systems may be limited by differences in patient population. Generalizability of the POC-CT method depends on the level of informatics and statistics infrastructure available to a healthcare system. CONCLUSIONS: The methods proposed will demonstrate outcome-based evaluation of control of hyperglycemia in hospitalized veterans. By institutionalizing a process of statistically sound and efficient learning, and by integrating that learning with automatic implementation of best practice, the participating VA Healthcare Systems will accelerate improvements in the effectiveness of care.

Comparative effectiveness research and medical informatics.

As is the case for environmental, ecological, astronomical, and other sciences, medical practice and research finds itself in a tsunami of data. This data deluge, due primarily to the introduction of digitalization in routine medical care and medical research, affords the opportunity for improved patient care and scientific discovery. Medical informatics is the subdiscipline of medicine created to make greater use of information in order to improve healthcare. The 4 areas of medical informatics research (information access, structure, analysis, and interaction) are used as a framework to discuss the overlap in information needs of comparative effectiveness research and potential contributions of medical informatics. Examples of progress from the medical informatics literature and the Veterans Affairs Healthcare System are provided.