Evaluation of a Large Language Model to Identify Confidential Content in Adolescent Encounter Notes.

This cross-sectional study assesses the ability of a language learning model to classify whether a progress note contains confidential information and to identify the specific confidential content in the note.

The Prevalence of Confidential Content in Adolescent Progress Notes Prior to the 21st Century Cures Act Information Blocking Mandate.

BACKGROUND: The 21st Century Cures Act information blocking final rule mandated the immediate and electronic release of health care data in 2020. There is anecdotal concern that a significant amount of information is documented in notes that would breach adolescent confidentiality if released electronically to a guardian. OBJECTIVES: The purpose of this study was to quantify the prevalence of confidential information, based on California laws, within progress notes for adolescent patients that would be released electronically and assess differences in prevalence across patient demographics. METHODS: This is a single-center retrospective chart review of outpatient progress notes written between January 1, 2016, and December 31, 2019, at a large suburban academic pediatric network. Notes were labeled into one of three confidential domains by five expert reviewers trained on a rubric defining confidential information for adolescents derived from California state law. Participants included a random sampling of eligible patients aged 12 to 17 years old at the time of note creation. Secondary analysis included prevalence of confidentiality across age, gender, language spoken, and patient race. RESULTS: Of 1,200 manually reviewed notes, 255 notes (21.3%) (95% confidence interval: 19-24%) contained confidential information. There was a similar distribution among gender and age and a majority of English speaking (83.9%) and white or Caucasian patients (41.2%) in the cohort. Confidential information was more likely to be found in notes for females (p < 0.05) as well as for English-speaking patients (p < 0.05). Older patients had a higher probability of notes containing confidential information (p < 0.05). CONCLUSION: This study demonstrates that there is a significant risk to breach adolescent confidentiality if historical progress notes are released electronically to proxies without further review or redaction. With increased sharing of health care data, there is a need to protect the privacy of the adolescents and prevent potential breaches of confidentiality.

Association between Electronic Health Record Implementations and Hospital-Acquired Conditions in Pediatric Hospitals.

BACKGROUND: Implementing an electronic health record (EHR) is one of the most disruptive operational tasks a health system can undergo. Despite anecdotal reports of adverse events around the time of EHR implementations, there is limited corroborating research, particularly in pediatrics. We utilized data from Solutions for Patient Safety (SPS), a network of 145+ children's hospitals that share data and protocols to reduce harm in pediatric care delivery, to study the impact of EHR implementations on patient safety. OBJECTIVE: Determine if there is an association between the time immediately surrounding an EHR implementation and hospital-acquired conditions (HACs) rates in pediatrics. METHODS: A survey of information technology leaders at pediatric institutions identified EHR implementations occurring between 2012 and 2022. This list was cross-referenced with the SPS database to create an anonymized dataset of 27 sites comprising monthly HAC and care bundle compliance rates in the 7 months preceding and succeeding the transition. Six HACs were analyzed: central-line associated bloodstream infection (CLABSI), catheter-associated urinary tract infection (CAUTI), adverse drug events, surgical site infections (SSIs), pressure injuries (PIs), and falls, in addition to four associated care bundle compliance rates: CLABSI and CAUTI maintenance bundles, SSI bundle, and PI bundle. To determine if there was a statistically significant association with EHR implementation, the observation period was divided into three eras: "before" (months -7 to -3), "during" (months -2 to +2), and "after" go-live (months +3 to +7). Average monthly HAC and bundle compliance rates were calculated across eras. Paired t-tests were performed to compare rates between the eras. RESULTS: No statistically significant increase in HAC rates or decrease in bundle compliance rates was observed across the EHR implementation eras. CONCLUSION: This multisite study detected no significant increase in HACs and no decrease in preventive care bundle compliance in the months surrounding an EHR implementation.

Pseudorandomized Testing of a Discharge Medication Alert to Reduce Free-Text Prescribing.

BACKGROUND: Pseudorandomized testing can be applied to perform rigorous yet practical evaluations of clinical decision support tools. We apply this methodology to an interruptive alert aimed at reducing free-text prescriptions. Using free-text instead of structured computerized provider order entry elements can cause medication errors and inequity in care by bypassing medication-based clinical decision support tools and hindering automated translation of prescription instructions. OBJECTIVE: The objective of this study is to evaluate the effectiveness of an interruptive alert at reducing free-text prescriptions via pseudorandomized testing using native electronic health records (EHR) functionality. METHODS: Two versions of an EHR alert triggered when a provider attempted to sign a discharge free-text prescription. The visible version displayed an interruptive alert to the user, and a silent version triggered in the background, serving as a control. Providers were assigned to the visible and silent arms based on even/odd EHR provider IDs. The proportion of encounters with a free-text prescription was calculated across the groups. Alert trigger rates were compared in process control charts. Free-text prescriptions were analyzed to identify prescribing patterns. RESULTS: Over the 28-week study period, 143 providers triggered 695 alerts (345 visible and 350 silent). The proportions of encounters with free-text prescriptions were 83% (266/320) and 90% (273/303) in the intervention and control groups, respectively (p = 0.01). For the active alert, median time to action was 31 seconds. Alert trigger rates between groups were similar over time. Ibuprofen, oxycodone, steroid tapers, and oncology-related prescriptions accounted for most free-text prescriptions. A majority of these prescriptions originated from user preference lists. CONCLUSION: An interruptive alert was associated with a modest reduction in free-text prescriptions. Furthermore, the majority of these prescriptions could have been reproduced using structured order entry fields. Targeting user preference lists shows promise for future intervention.

A Natural Language Processing Model to Identify Confidential Content in Adolescent Clinical Notes.

BACKGROUND: The 21st Century Cures Act mandates the immediate, electronic release of health information to patients. However, in the case of adolescents, special consideration is required to ensure that confidentiality is maintained. The detection of confidential content in clinical notes may support operational efforts to preserve adolescent confidentiality while implementing information sharing. OBJECTIVES: This study aimed to determine if a natural language processing (NLP) algorithm can identify confidential content in adolescent clinical progress notes. METHODS: A total of 1,200 outpatient adolescent progress notes written between 2016 and 2019 were manually annotated to identify confidential content. Labeled sentences from this corpus were featurized and used to train a two-part logistic regression model, which provides both sentence-level and note-level probability estimates that a given text contains confidential content. This model was prospectively validated on a set of 240 progress notes written in May 2022. It was subsequently deployed in a pilot intervention to augment an ongoing operational effort to identify confidential content in progress notes. Note-level probability estimates were used to triage notes for review and sentence-level probability estimates were used to highlight high-risk portions of those notes to aid the manual reviewer. RESULTS: The prevalence of notes containing confidential content was 21% (255/1,200) and 22% (53/240) in the train/test and validation cohorts, respectively. The ensemble logistic regression model achieved an area under the receiver operating characteristic of 90 and 88% in the test and validation cohorts, respectively. Its use in a pilot intervention identified outlier documentation practices and demonstrated efficiency gains over completely manual note review. CONCLUSION: An NLP algorithm can identify confidential content in progress notes with high accuracy. Its human-in-the-loop deployment in clinical operations augmented an ongoing operational effort to identify confidential content in adolescent progress notes. These findings suggest NLP may be used to support efforts to preserve adolescent confidentiality in the wake of the information blocking mandate.

Targeting repetitive laboratory testing with electronic health records-embedded predictive decision support: A pre-implementation study.

INTRODUCTION: Unnecessary laboratory testing contributes to patient morbidity and healthcare waste. Despite prior attempts at curbing such overutilization, there remains opportunity for improvement using novel data-driven approaches. This study presents the development and early evaluation of a clinical decision support tool that uses a predictive model to help providers reduce low-yield, repetitive laboratory testing in hospitalized patients. METHODS: We developed an EHR-embedded SMART on FHIR application that utilizes a laboratory test result prediction model based on historical laboratory data. A combination of semi-structured physician interviews, usability testing, and quantitative analysis on retrospective laboratory data were used to inform the tool's development and evaluate its acceptability and potential clinical impact. KEY RESULTS: Physicians identified culture and lack of awareness of repeat orders as key drivers for overuse of inpatient blood testing. Users expressed an openness to a lab prediction model and 13/15 physicians believed the tool would alter their ordering practices. The application received a median System Usability Scale score of 75, corresponding to the 75th percentile of software tools. On average, physicians desired a prediction certainty of 85% before discontinuing a routine recurring laboratory order and a higher certainty of 90% before being alerted. Simulation on historical lab data indicates that filtering based on accepted thresholds could have reduced âˆ¼22% of repeat chemistry panels. CONCLUSIONS: The use of a predictive algorithm as a means to calculate the utility of a diagnostic test is a promising paradigm for curbing laboratory test overutilization. An EHR-embedded clinical decision support tool employing such a model is a novel and acceptable intervention with the potential to reduce low-yield, repetitive laboratory testing.

Investigating real-world consequences of biases in commonly used clinical calculators.

OBJECTIVES: To evaluate whether one summary metric of calculator performance sufficiently conveys equity across different demographic subgroups, as well as to evaluate how calculator predictive performance affects downstream health outcomes. STUDY DESIGN: We evaluate 3 commonly used clinical calculators-Model for End-Stage Liver Disease (MELD), CHA2DS2-VASc, and simplified Pulmonary Embolism Severity Index (sPESI)-on the cohort extracted from the Stanford Medicine Research Data Repository, following the cohort selection process as described in respective calculator derivation papers. METHODS: We quantified the predictive performance of the 3 clinical calculators across sex and race. Then, using the clinical guidelines that guide care based on these calculators' output, we quantified potential disparities in subsequent health outcomes. RESULTS: Across the examined subgroups, the MELD calculator exhibited worse performance for female and White populations, CHA2DS2-VASc calculator for the male population, and sPESI for the Black population. The extent to which such performance differences translated into differential health outcomes depended on the distribution of the calculators' scores around the thresholds used to trigger a care action via the corresponding guidelines. In particular, under the old guideline for CHA2DS2-VASc, among those who would not have been offered anticoagulant therapy, the Hispanic subgroup exhibited the highest rate of stroke. CONCLUSIONS: Clinical calculators, even when they do not include variables such as sex and race as inputs, can have very different care consequences across those subgroups. These differences in health care outcomes across subgroups can be explained by examining the distribution of scores and their calibration around the thresholds encoded in the accompanying care guidelines.

National Trends in Pediatric Ambulatory Telehealth Utilization and Follow-Up Care.

Introduction: As telemedicine becomes standard in pediatrics, further research is required to ensure optimal adoption. This study seeks to characterize visits best suited for telemedicine by analyzing usage trends and encounter attributes associated with immediate in-person follow-up. Methods: Analysis of ambulatory pediatric encounters from the first quarter of 2021 in a nationwide insurance claims database. Results: Telemedicine comprised 9.5% (138,346) of ambulatory encounters. Among telemedicine visits, 7.5% (10,304) yielded in-person follow-up within 3 days. Encounters involving infants and diagnoses of the perinatal period were most frequently followed by in-person visits (11% and 20%, respectively). Mental health visits were least likely to have in-person follow-up. Conclusions: In 2021, telemedicine remained a common modality of care in pediatrics. Varying medical needs still require in-person evaluation, whereas other diagnoses may be conducive to even greater expansion. Insights from this study inform further research into optimization of pediatric telemedicine utilization and development of guidelines.

Association Between Cytomegalovirus Serostatus, Antiviral Therapy, and Allograft Survival in Pediatric Heart Transplantation.

Background: Cytomegalovirus (CMV) is an important complication of heart transplantation and has been associated with graft loss in adults. The data in pediatric transplantation, however, is limited and conflicting. We conducted a large-scale cohort study to better characterize the relationship between CMV serostatus, CMV antiviral use, and graft survival in pediatric heart transplantation. Methods: 4,968 pediatric recipients of solitary heart transplants from the Scientific Registry of Transplant Recipients were stratified into three groups based on donor or recipient seropositivity and antiviral use: CMV seronegative (CMV-) transplants, CMV seropositive (CMV+) transplants without antiviral therapy, and CMV+ transplants with antiviral therapy. The primary endpoint was retransplantation or death. Results: CMV+ transplants without antiviral therapy experienced worse graft survival than CMV+ transplants with antiviral therapy (10-year: 57 vs 65%). CMV+ transplants with antiviral therapy experienced similar survival as CMV- transplants. Compared to CMV seronegativity, CMV seropositivity without antiviral therapy had a hazard ratio of 1.21 (1.07-1.37 95% CI, p-value = .003). Amongst CMV+ transplants, antiviral therapy had a hazard ratio of .82 (0.74-.92 95% CI, p-value < .001). During the first year after transplantation, these hazard ratios were 1.32 (1.06-1.64 95% CI, p-value .014) and .59 (.48-.73 95% CI, p-value < .001), respectively. Conclusions: CMV seropositivity is associated with an increased risk of graft loss in pediatric heart transplant recipients, which occurs early after transplantation and may be mitigated by antiviral therapy.

Applications of machine learning in routine laboratory medicine: Current state and future directions.

Machine learning is able to leverage large amounts of data to infer complex patterns that are otherwise beyond the capabilities of rule-based systems and human experts. Its application to laboratory medicine is particularly exciting, as laboratory testing provides much of the foundation for clinical decision making. In this article, we provide a brief introduction to machine learning for the medical professional in addition to a comprehensive literature review outlining the current state of machine learning as it has been applied to routine laboratory medicine. Although still in its early stages, machine learning has been used to automate laboratory tasks, optimize utilization, and provide personalized reference ranges and test interpretation. The published literature leads us to believe that machine learning will be an area of increasing importance for the laboratory practitioner. We envision the laboratory of the future will utilize these methods to make significant improvements in efficiency and diagnostic precision.

Tracking clinical status for heart failure patients using ballistocardiography and electrocardiography signal features.

Heart failure (HF) is an escalating public health problem, with few effective methods for home monitoring. In HF management, the important clinical factors to monitor include symptoms, fluid status, cardiac output, and blood pressure--based on these factors, inotrope and diuretic dosages are adjusted day-by-day to control the disorder and improve the patient's status towards a successful discharge. Previously, the ballistocardiogram (BCG) measured on a weighing scale has been shown to be capable of detecting changes in cardiac output and contractility for healthy subjects. In this study, we investigated whether the BCG and electrocardiogram (ECG) signals measured on a wireless modified scale could accurately track the clinical status of HF patients during their hospital stay. Using logistic regression, we found that the root-mean-square (RMS) power of the BCG provided a good fit for clinical status, as determined based on clinical measurements and symptoms, for the 85 patient days studied from 10 patients (p < 0.01). These results provide a promising foundation for future studies aimed at using the BCG/ECG scale at home to track HF patient status remotely.

Physical constraints on the establishment of intracellular spatial gradients in bacteria.

BACKGROUND: Bacteria dynamically regulate their intricate intracellular organization involving proteins that facilitate cell division, motility, and numerous other processes. Consistent with this sophisticated organization, bacteria are able to create asymmetries and spatial gradients of proteins by localizing signaling pathway components. We use mathematical modeling to investigate the biochemical and physical constraints on the generation of intracellular gradients by the asymmetric localization of a source and a sink. RESULTS: We present a systematic computational analysis of the effects of other regulatory mechanisms, such as synthesis, degradation, saturation, and cell growth. We also demonstrate that gradients can be established in a variety of bacterial morphologies such as rods, crescents, spheres, branched and constricted cells. CONCLUSIONS: Taken together, these results suggest that gradients are a robust and potentially common mechanism for providing intracellular spatial cues.