Evaluation of a Large Language Model to Identify Confidential Content in Adolescent Encounter Notes.

This cross-sectional study assesses the ability of a language learning model to classify whether a progress note contains confidential information and to identify the specific confidential content in the note.

Variation in pediatric and adolescent electronic health data sharing practices under the 21st Century Cures Act.

OBJECTIVE: To describe real-world practices and variation in implementation of the Information Blocking provisions amongst healthcare organizations caring for pediatric patients. MATERIALS AND METHODS: An online survey regarding implementation practices was distributed to representatives from 10 participating US healthcare organizations located in 6 different states. The survey was followed by structured interviews conducted through video conference. Information was gathered about implementation practices at each organization, with a focus on patient and proxy portal access to, and segmentation capabilities of, certain data classes listed in the United States Core Data for Interoperability Version 1. RESULTS: All organizations had implemented the information blocking provisions at their institution. All organizations utilized different portal account types for proxies and users. All organizations reported the capability of sharing labs, medications, problem lists, imaging, and notes with the parent/guardian of the non-adolescent minor user with differences in how sensitive elements within the data classes were protected. Variability existed in how data was shared with the remaining user types. DISCUSSION: Significant variability exists in how organizations have implemented the information blocking rules. Variation in data sharing and data access between institutions can result in privacy breaches and create confusion about completeness of data for patients and families. CONCLUSION: Healthcare organizations have utilized varying strategies to comply with the information blocking provisions of the 21st Century Cures Act. Increased clarity from the Office of the National Coordinator for Health Information Technology on minor, adolescent, and caregiver privacy and improved segmentation capabilities from Electronic Health Record vendors is needed.

The Prevalence of Confidential Content in Adolescent Progress Notes Prior to the 21st Century Cures Act Information Blocking Mandate.

BACKGROUND: The 21st Century Cures Act information blocking final rule mandated the immediate and electronic release of health care data in 2020. There is anecdotal concern that a significant amount of information is documented in notes that would breach adolescent confidentiality if released electronically to a guardian. OBJECTIVES: The purpose of this study was to quantify the prevalence of confidential information, based on California laws, within progress notes for adolescent patients that would be released electronically and assess differences in prevalence across patient demographics. METHODS: This is a single-center retrospective chart review of outpatient progress notes written between January 1, 2016, and December 31, 2019, at a large suburban academic pediatric network. Notes were labeled into one of three confidential domains by five expert reviewers trained on a rubric defining confidential information for adolescents derived from California state law. Participants included a random sampling of eligible patients aged 12 to 17 years old at the time of note creation. Secondary analysis included prevalence of confidentiality across age, gender, language spoken, and patient race. RESULTS: Of 1,200 manually reviewed notes, 255 notes (21.3%) (95% confidence interval: 19-24%) contained confidential information. There was a similar distribution among gender and age and a majority of English speaking (83.9%) and white or Caucasian patients (41.2%) in the cohort. Confidential information was more likely to be found in notes for females (p < 0.05) as well as for English-speaking patients (p < 0.05). Older patients had a higher probability of notes containing confidential information (p < 0.05). CONCLUSION: This study demonstrates that there is a significant risk to breach adolescent confidentiality if historical progress notes are released electronically to proxies without further review or redaction. With increased sharing of health care data, there is a need to protect the privacy of the adolescents and prevent potential breaches of confidentiality.

A Natural Language Processing Model to Identify Confidential Content in Adolescent Clinical Notes.

BACKGROUND: The 21st Century Cures Act mandates the immediate, electronic release of health information to patients. However, in the case of adolescents, special consideration is required to ensure that confidentiality is maintained. The detection of confidential content in clinical notes may support operational efforts to preserve adolescent confidentiality while implementing information sharing. OBJECTIVES: This study aimed to determine if a natural language processing (NLP) algorithm can identify confidential content in adolescent clinical progress notes. METHODS: A total of 1,200 outpatient adolescent progress notes written between 2016 and 2019 were manually annotated to identify confidential content. Labeled sentences from this corpus were featurized and used to train a two-part logistic regression model, which provides both sentence-level and note-level probability estimates that a given text contains confidential content. This model was prospectively validated on a set of 240 progress notes written in May 2022. It was subsequently deployed in a pilot intervention to augment an ongoing operational effort to identify confidential content in progress notes. Note-level probability estimates were used to triage notes for review and sentence-level probability estimates were used to highlight high-risk portions of those notes to aid the manual reviewer. RESULTS: The prevalence of notes containing confidential content was 21% (255/1,200) and 22% (53/240) in the train/test and validation cohorts, respectively. The ensemble logistic regression model achieved an area under the receiver operating characteristic of 90 and 88% in the test and validation cohorts, respectively. Its use in a pilot intervention identified outlier documentation practices and demonstrated efficiency gains over completely manual note review. CONCLUSION: An NLP algorithm can identify confidential content in progress notes with high accuracy. Its human-in-the-loop deployment in clinical operations augmented an ongoing operational effort to identify confidential content in adolescent progress notes. These findings suggest NLP may be used to support efforts to preserve adolescent confidentiality in the wake of the information blocking mandate.