Natural language processing augments comorbidity documentation in neurosurgical inpatient admissions.

OBJECTIVE: To establish whether or not a natural language processing technique could identify two common inpatient neurosurgical comorbidities using only text reports of inpatient head imaging. MATERIALS AND METHODS: A training and testing dataset of reports of 979 CT or MRI scans of the brain for patients admitted to the neurosurgery service of a single hospital in June 2021 or to the Emergency Department between July 1-8, 2021, was identified. A variety of machine learning and deep learning algorithms utilizing natural language processing were trained on the training set (84% of the total cohort) and tested on the remaining images. A subset comparison cohort (n = 76) was then assessed to compare output of the best algorithm against real-life inpatient documentation. RESULTS: For "brain compression", a random forest classifier outperformed other candidate algorithms with an accuracy of 0.81 and area under the curve of 0.90 in the testing dataset. For "brain edema", a random forest classifier again outperformed other candidate algorithms with an accuracy of 0.92 and AUC of 0.94 in the testing dataset. In the provider comparison dataset, for "brain compression," the random forest algorithm demonstrated better accuracy (0.76 vs 0.70) and sensitivity (0.73 vs 0.43) than provider documentation. For "brain edema," the algorithm again demonstrated better accuracy (0.92 vs 0.84) and AUC (0.45 vs 0.09) than provider documentation. DISCUSSION: A natural language processing-based machine learning algorithm can reliably and reproducibly identify selected common neurosurgical comorbidities from radiology reports. CONCLUSION: This result may justify the use of machine learning-based decision support to augment provider documentation.

Immigration documentation statuses evoke racialized faceism in mental representations.

U.S. immigration discourse has spurred interest in characterizing who illegalized immigrants are or perceived to be. What are the associated visual representations of migrant illegality? Across two studies with undergraduate and online samples (N = 686), we used face-based reverse correlation and similarity sorting to capture and compare mental representations of illegalized immigrants, native-born U.S. citizens, and documented immigrants. Documentation statuses evoked racialized imagery. Immigrant representations were dark-skinned and perceived as non-white, while citizen representations were light-skinned, evaluated positively, and perceived as white. Legality further differentiated immigrant representations: documentation conjured trustworthy representations, illegality conjured threatening representations. Participants spontaneously sorted unlabeled faces by documentation status in a spatial arrangement task. Faces' spatial similarity correlated with their similarity in pixel luminance and "American" ratings, confirming racialized distinctions. Representations of illegalized immigrants were uniquely racialized as dark-skinned un-American threats, reflecting how U.S. imperialism and colorism set conditions of possibility for existing representations of migrant illegalization.

Evaluating feedback reports to support documentation of veterans' care preferences in home based primary care.

BACKGROUND: To evaluate the effectiveness of delivering feedback reports to increase completion of LST notes among VA Home Based Primary Care (HBPC) teams. The Life Sustaining Treatment Decisions Initiative (LSTDI) was implemented throughout the Veterans Health Administration (VHA) in the United States in 2017 to ensure that seriously ill Veterans have care goals and LST decisions elicited and documented. METHODS: We distributed monthly feedback reports summarizing LST template completion rates to 13 HBPC intervention sites between October 2018 and February 2020 as the sole implementation strategy. We used principal component analyses to match intervention to 26 comparison sites and used interrupted time series/segmented regression analyses to evaluate the differences in LST template completion rates between intervention and comparison sites. Data were extracted from national databases for VA HBPC in addition to interviews and surveys in a mixed methods process evaluation. RESULTS: LST template completion rose from 6.3 to 41.9% across both intervention and comparison HBPC teams between March 1, 2018, and February 26, 2020. There were no statistically significant differences for intervention sites that received feedback reports. CONCLUSIONS: Feedback reports did not increase documentation of LST preferences for Veterans at intervention compared with comparison sites. Observed increases in completion rates across intervention and comparison sites can likely be attributed to implementation strategies used nationally as part of the national roll-out of the LSTDI. Our results suggest that feedback reports alone were not an effective implementation strategy to augment national implementation strategies in HBPC teams.

General Surgery Resuscitation Preference Documentation: A Quality Improvement Initiative.

BACKGROUND/PURPOSE: Documentation of resuscitation preferences is crucial for patients undergoing surgery. Unfortunately, this remains an area for improvement at many institutions. We conducted a quality improvement initiative to enhance documentation percentages by integrating perioperative resuscitation checks into the surgical workflow. Specifically, we aimed to increase the percentage of general surgery patients with documented resuscitation statuses from 82% to 90% within a 1-year period. METHODS: Three key change ideas were developed. First, surgical consent forms were modified to include the patient's resuscitation status. Second, the resuscitation status was added to the routinely used perioperative surgical checklist. Finally, patient resources on resuscitation processes and options were updated with support from patient partners. An audit survey was distributed mid-way through the interventions to evaluate process measures. RESULTS: The initiatives were successful in reaching our study aim of 90% documentation rate for all general surgery patients. The audit revealed a high uptake of the new consent forms, moderate use of the surgical checklist, and only a few patients for whom additional resuscitation details were added to their clinical note. CONCLUSIONS: We successfully increased the documentation percentage of resuscitation statuses within our large tertiary care center by incorporating checks into routine forms to prompt the conversation with patients early.

Variable Impact of Medical Scribes on Physician Electronic Health Record Documentation Practices: A Quantitative Analysis Across a Large, Integrated Health-System.

BACKGROUND: Medical scribes have been utilized to reduce electronic health record (EHR) associated documentation burden. Although evidence suggests benefits to scribes, no large-scale studies have quantitatively evaluated scribe impact on physician documentation across clinical settings. This study aimed to evaluate the effect of scribes on physician EHR documentation behaviors and performance. METHODS: This retrospective cohort study used EHR audit log data from a large academic health system to evaluate clinical documentation for all ambulatory encounters between January 2014 and December 2019 to evaluate the effect of scribes on physician documentation behaviors. Scribe services were provided on a first-come, first-served basis on physician request. Based on a physician's scribe use, encounters were grouped into 3 categories: never using a scribe, prescribe (before scribe use), or using a scribe. Outcomes included chart closure time, the proportion of delinquent charts, and charts closed after-hours. RESULTS: Three hundred ninety-five physicians (23% scribe users) across 29 medical subspecialties, encompassing 1,132,487 encounters, were included in the analysis. At baseline, scribe users had higher chart closure time, delinquent charts, and after-hours documentation than physicians who never used scribes. Among scribe users, the difference in outcome measures postscribe compared with baseline varied, and using a scribe rarely resulted in outcome measures approaching a range similar to the performance levels of nonusing physicians. In addition, there was variability in outcome measures across medical specialties and within similar subspecialties. CONCLUSION: Although scribes may improve documentation efficiency among some physicians, not all will improve EHR-related documentation practices. Different strategies may help to optimize documentation behaviors of physician-scribe dyads and maximize outcomes of scribe implementation.

Use of natural language processing to uncover racial bias in obstetrical documentation.

Natural Language Processing (NLP), a form of Artificial Intelligence, allows free-text based clinical documentation to be integrated in ways that facilitate data analysis, data interpretation and formation of individualized medical and obstetrical care. In this cross-sectional study, we identified all births during the study period carrying the radiology-confirmed diagnosis of fibroid uterus in pregnancy (defined as size of largest diameter of >5 cm) by using an NLP platform and compared it to non-NLP derived data using ICD10 codes of the same diagnosis. We then compared the two sets of data and stratified documentation gaps by race. Using fibroid uterus in pregnancy as a marker, we found that Black patients were more likely to have the diagnosis entered late into the patient's chart or had missing documentation of the diagnosis. With appropriate algorithm definitions, cross referencing and thorough validation steps, NLP can contribute to identifying areas of documentation gaps and improve quality of care.

Assessing the Usability and Feasibility of Digital Assistant Tools for Direct Support Professionals: Participatory Design and Pilot-Testing.

BACKGROUND: The United States is experiencing a direct support professional (DSP) crisis, with demand far exceeding supply. Although generating documentation is a critical responsibility, it is one of the most wearisome aspects of DSPs' jobs. Technology that enables DSPs to log informal time-stamped notes throughout their shift could help reduce the burden of end-of-shift documentation and increase job satisfaction, which in turn could improve the quality of life of the individuals with intellectual and developmental disabilities (IDDs) whom DSPs support. However, DSPs, with varied ages, levels of education, and comfort using technology, are not likely to adopt tools that detract from caregiving responsibilities or increase workload; therefore, technological tools for them must be relatively simple, extremely intuitive, and provide highly valued capabilities. OBJECTIVE: This paper describes the development and pilot-testing of a digital assistant tool (DAT) that enables DSPs to create informal notes throughout their shifts and use these notes to facilitate end-of-shift documentation. The purpose of the pilot study was to assess the usability and feasibility of the DAT. METHODS: The research team applied an established user-centered participatory design process to design, develop, and test the DAT prototypes between May 2020 and April 2023. Pilot-testing entailed having 14 DSPs who support adults with IDDs use the first full implementation of the DAT prototypes during 2 or 3 successive work shifts and fill out demographic and usability questionnaires. RESULTS: Participants used the DAT prototypes to create notes and help generate end-of-shift reports. The System Usability Scale score of 81.79 indicates that they found the prototypes easy to use. Survey responses imply that using the DAT made it easier for participants to produce required documentation and suggest that they would adopt the DAT if this tool were available for daily use. CONCLUSIONS: Simple technologies such as the DAT prototypes, which enable DSPs to use mobile devices to log time-stamped notes throughout their shift with minimal effort and use the notes to help write reports, have the potential to both reduce the burden associated with producing documentation and enhance the quality (level of detail and accuracy) of this documentation. This could help to increase job satisfaction and reduce turnover in DSPs, both of which would help improve the quality of life of the individuals with IDDs whom they support. The pilot test results indicate that DSPs found the DAT easy to use. Next steps include (1) producing more robust versions of the DAT with additional capabilities, such as storing data locally on mobile devices when Wi-Fi is not available; and (2) eliciting input from agency directors, families, and others who use data about adults with IDDs to help care for them to ensure that data produced by DSPs are relevant and useful.

Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative Study.

BACKGROUND: Medical documentation plays a crucial role in clinical practice, facilitating accurate patient management and communication among health care professionals. However, inaccuracies in medical notes can lead to miscommunication and diagnostic errors. Additionally, the demands of documentation contribute to physician burnout. Although intermediaries like medical scribes and speech recognition software have been used to ease this burden, they have limitations in terms of accuracy and addressing provider-specific metrics. The integration of ambient artificial intelligence (AI)-powered solutions offers a promising way to improve documentation while fitting seamlessly into existing workflows. OBJECTIVE: This study aims to assess the accuracy and quality of Subjective, Objective, Assessment, and Plan (SOAP) notes generated by ChatGPT-4, an AI model, using established transcripts of History and Physical Examination as the gold standard. We seek to identify potential errors and evaluate the model's performance across different categories. METHODS: We conducted simulated patient-provider encounters representing various ambulatory specialties and transcribed the audio files. Key reportable elements were identified, and ChatGPT-4 was used to generate SOAP notes based on these transcripts. Three versions of each note were created and compared to the gold standard via chart review; errors generated from the comparison were categorized as omissions, incorrect information, or additions. We compared the accuracy of data elements across versions, transcript length, and data categories. Additionally, we assessed note quality using the Physician Documentation Quality Instrument (PDQI) scoring system. RESULTS: Although ChatGPT-4 consistently generated SOAP-style notes, there were, on average, 23.6 errors per clinical case, with errors of omission (86%) being the most common, followed by addition errors (10.5%) and inclusion of incorrect facts (3.2%). There was significant variance between replicates of the same case, with only 52.9% of data elements reported correctly across all 3 replicates. The accuracy of data elements varied across cases, with the highest accuracy observed in the "Objective" section. Consequently, the measure of note quality, assessed by PDQI, demonstrated intra- and intercase variance. Finally, the accuracy of ChatGPT-4 was inversely correlated to both the transcript length (P=.05) and the number of scorable data elements (P=.05). CONCLUSIONS: Our study reveals substantial variability in errors, accuracy, and note quality generated by ChatGPT-4. Errors were not limited to specific sections, and the inconsistency in error types across replicates complicated predictability. Transcript length and data complexity were inversely correlated with note accuracy, raising concerns about the model's effectiveness in handling complex medical cases. The quality and reliability of clinical notes produced by ChatGPT-4 do not meet the standards required for clinical use. Although AI holds promise in health care, caution should be exercised before widespread adoption. Further research is needed to address accuracy, variability, and potential errors. ChatGPT-4, while valuable in various applications, should not be considered a safe alternative to human-generated clinical documentation at this time.

Factors impacting clinical data and documentation quality in Australian aged care and disability services: a user-centred perspective.

BACKGROUND: Research has highlighted a need to improve the quality of clinical documentation and data within aged care and disability services in Australia to support improved regulatory reporting and ensure quality and safety of services. However, the specific causes of data quality issues within aged care and disability services and solutions for optimisation are not well understood. OBJECTIVES: This study explored aged care and disability workforce (referred to as 'data-users') experiences and perceived root causes of clinical data quality issues at a large aged care and disability services provider in Western Australia, to inform optimisation solutions. METHODS: A purposive sample of n = 135 aged care and disability staff (including community-based and residential-based) in clinical, care, administrative and/or management roles participated in semi-structured interviews and web-based surveys. Data were analysed using an inductive thematic analysis method, where themes and subthemes were derived. RESULTS: Eight overarching causes of data and documentation quality issues were identified: (1) staff-related challenges, (2) education and training, (3) external barriers, (4) operational guidelines and procedures, (5) organisational practices and culture, (6) technological infrastructure, (7) systems design limitations, and (8) systems configuration-related challenges. CONCLUSION: The quality of clinical data and documentation within aged care and disability services is influenced by a complex interplay of internal and external factors. Coordinated and collaborative effort is required between service providers and the wider sector to identify behavioural and technical optimisation solutions to support safe and high-quality care and improved regulatory reporting.

Who supports Bernie? Analyzing identity and ideological variation on Twitter during the 2020 democratic primaries.

Using a novel dataset of 590M messages by 21M users, we present the first large-scale examination of the behavior of likely Bernie supporters on Twitter during the 2020 U.S. Democratic primaries and presidential election. We use these data to dispel empirically the notion of a unified, stereotypical Bernie supporter (e.g., the "Bernie Bro"). Instead, our work uncovers significant variation in the identities and ideologies of Bernie supporters who were active on Twitter. Our work makes three contributions to the literature on social media and social movements. Methodologically, we present a novel mixed methods approach to surface identity and ideological variation within a movement via use of patterns in who retweets whom (i.e. who retweets which other users) and who retweets what (i.e. who retweets which specific tweets). Substantively, documentation of these variations challenges a trend in the social movement literature to assume actors within a particular movement are unified in their ideology, identity, and values.

Workload reduction through automated documentation in intensive and intermediate care - a monocentric observational study.

OBJECTIVE: The global coronavirus pandemic has placed an unprecedented and enormous burden on health systems worldwide. In addition to a shortage of resources, nurses were also confronted with high levels of sick leave and an increasing exodus from the profession. Automating documentation obligations is an effective way of reducing the burden on the workplace. PATIENTS AND METHODS: The study was conducted at a tertiary university hospital. The time required for the manual documentation of administered medication and dose changes of syringe and infusion pumps was recorded using the patient data management system (PDMS) representing all intensive and intermediate care wards (n = 6). Subsequently, all medication administration - grouped into five classes - was evaluated from January 1st, 2019, until December 31st, 2022. RESULTS: A total of 1,373,340 drug applications were studied, treating 32,499 patients. Data were obtained from ICUs (68%) and IMC wards (32%). This corresponds to an overall time of 2,901 ± 233 hours per year. Based on publicly known national rates for intensive care nurses, an annual financial expenditure of approximately 83,300 € (~ USD 89,300) per year was estimated. CONCLUSIONS: A non-negligible part of the daily working time in the medical sector is spent on documentation duties. This aggravates the high workload, which has increased in recent years. Automated documentation systems can lead to considerable relief and the possibility of focusing primarily on the patient and on other core competencies and activities. This is even more important, as available staff will be a key resource in patient care for the foreseeable future.

CareNet Tyrol - Information System Success Assessment for Case & Care Management Service.

BACKGROUND: CareNet is the IT-based tool for Case and Care Management (CCM) in Tyrol, which facilitates standardised documentation of CCM activities. OBJECTIVES: Analysing the pilot usage of CareNet Tyrol. METHODS: Evaluation of the success and user experience of CareNet, expert interviews and a questionnaire-based assessment. RESULTS: Feedback from users in both phases indicated that the CareNet platform provides general benefits, but falls short of fully supporting the daily work of CCM experts and avoiding the need for parallel use of different documentation tools. CONCLUSION: This paper provides an insight into the ongoing transition to digital documentation for CCM at LIV Tyrol. While user feedback highlights areas for improvement, digital documentation is proved to be beneficial for the CCM team.

Documentation of a novel FBP1 gene mutation in the Arabian ethnicity: a case report.

BACKGROUND: Fructose-1,6-bisphosphatase deficiency is a rare autosomal recessive disorder characterized by impaired gluconeogenesis. Fructose-1,6-bisphosphatase 1 (FBP1) mutations demonstrate ethnic patterns. For instance, Turkish populations commonly harbor exon 2 deletions. We present a case report of whole exon 2 deletion in a Syrian Arabian child as the first recording of this mutation among Arabian ethnicity and the first report of FBP1 gene mutation in Syria. CASE PRESENTATION: We present the case of a 2.5-year-old Syrian Arab child with recurrent hypoglycemic episodes, accompanied by nausea and lethargy. The patient's history, physical examination, and laboratory findings raised suspicion of fructose-1,6-bisphosphatase deficiency. Whole exome sequencing was performed, revealing a homozygous deletion of exon 2 in the FBP1 gene, confirming the diagnosis. CONCLUSION: This case highlights a potential novel mutation in the Arab population; this mutation is well described in the Turkish population, which suggests potential shared mutations due to ancestral relationships between the two ethnicities. Further studies are needed to confirm this finding.

Association Between Medicare's Sepsis Reporting Policy (SEP-1) and the Documentation of a Sepsis Diagnosis in the Clinical Record.

STUDY DESIGN: Interrupted time series analysis of a retrospective, electronic health record cohort. OBJECTIVE: To determine the association between the implementation of Medicare's sepsis reporting measure (SEP-1) and sepsis diagnosis rates as assessed in clinical documentation. BACKGROUND: The role of health policy in the effort to improve sepsis diagnosis remains unclear. PATIENTS AND METHODS: Adult patients hospitalized with suspected infection and organ dysfunction within 6 hours of presentation to the emergency department, admitted to one of 11 hospitals in a multi-hospital health system from January 2013 to December 2017. Clinician-diagnosed sepsis, as reflected by the inclusion of the terms "sepsis" or "septic" in the text of clinical notes in the first two calendar days following presentation. RESULTS: Among 44,074 adult patients with sepsis admitted to 11 hospitals over 5 years, the proportion with sepsis documentation was 32.2% just before the implementation of SEP-1 in the third quarter of 2015 and increased to 37.3% by the fourth quarter of 2017. Of the 9 post-SEP-1 quarters, 8 had odds ratios for a sepsis diagnosis >1 (overall range: 0.98-1.26; P value for a joint test of statistical significance = 0.005). The effects were clinically modest, with a maximum effect of an absolute increase of 4.2% (95% CI: 0.9-7.8) at the end of the study period. The effect was greater in patients who did not require vasopressors compared with patients who required vasopressors ( P value for test of interaction = 0.02). CONCLUSIONS: SEP-1 implementation was associated with modest increases in sepsis diagnosis rates, primarily among patients who did not require vasoactive medications.

The utility of synoptic operation reports in colorectal surgery: a systematic review.

PURPOSE: Accurate documentation is crucial in surgical patient care. Synoptic reports (SR) are structured checklist-based reports that offer a standardised alternative to traditional narrative reports (NR). This systematic review aims to assess the completeness of SR compared to NR in colorectal cancer (CRC) surgery. Secondary outcomes include the time to completion, surgeon satisfaction, educational value, research value, and barriers to implementation. METHODS: Prospective or retrospective studies that assessed SR compared to NR in colorectal cancer surgery procedures were identified through a systematic search of Ovid MEDLINE, Embase (Ovid), CIHNAL Plus with Full Text (EBSCOhost), and Cochrane. One thousand two articles were screened, and eight studies met the inclusion criteria after full-text review of 17 papers. RESULTS: Analysis included 1797 operative reports (NR, 729; SR, 1068). Across studies reporting this outcome, the completeness of documentation was significantly higher in SR (P < 0.001). Reporting of secondary outcomes was limited, with a predominant focus on research value. Several studies demonstrated significantly reduced data extraction times when utilising SR. Surgeon satisfaction with SR was high, and these reports were seen as valuable tools for research and education. Barriers to implementation included integrating SR into existing electronic medical records (EMR) and surgeon concerns regarding increased administrative burden. CONCLUSIONS: SR offer advantages in completeness, data extraction, and communication compared to NR. Surgeons perceive them as beneficial for research, quality improvement, and teaching. This review supports the necessity for development of user-friendly SR that seamlessly integrate into pre-existing EMRs, optimising patient care and enhancing the quality of CRC surgical documentation.

Identifying social determinants of health from clinical narratives: A study of performance, documentation ratio, and potential bias.

OBJECTIVE: To develop a natural language processing (NLP) package to extract social determinants of health (SDoH) from clinical narratives, examine the bias among race and gender groups, test the generalizability of extracting SDoH for different disease groups, and examine population-level extraction ratio. METHODS: We developed SDoH corpora using clinical notes identified at the University of Florida (UF) Health. We systematically compared 7 transformer-based large language models (LLMs) and developed an open-source package - SODA (i.e., SOcial DeterminAnts) to facilitate SDoH extraction from clinical narratives. We examined the performance and potential bias of SODA for different race and gender groups, tested the generalizability of SODA using two disease domains including cancer and opioid use, and explored strategies for improvement. We applied SODA to extract 19 categories of SDoH from the breast (n = 7,971), lung (n = 11,804), and colorectal cancer (n = 6,240) cohorts to assess patient-level extraction ratio and examine the differences among race and gender groups. RESULTS: We developed an SDoH corpus using 629 clinical notes of cancer patients with annotations of 13,193 SDoH concepts/attributes from 19 categories of SDoH, and another cross-disease validation corpus using 200 notes from opioid use patients with 4,342 SDoH concepts/attributes. We compared 7 transformer models and the GatorTron model achieved the best mean average strict/lenient F1 scores of 0.9122 and 0.9367 for SDoH concept extraction and 0.9584 and 0.9593 for linking attributes to SDoH concepts. There is a small performance gap (∼4%) between Males and Females, but a large performance gap (>16 %) among race groups. The performance dropped when we applied the cancer SDoH model to the opioid cohort; fine-tuning using a smaller opioid SDoH corpus improved the performance. The extraction ratio varied in the three cancer cohorts, in which 10 SDoH could be extracted from over 70 % of cancer patients, but 9 SDoH could be extracted from less than 70 % of cancer patients. Individuals from the White and Black groups have a higher extraction ratio than other minority race groups. CONCLUSIONS: Our SODA package achieved good performance in extracting 19 categories of SDoH from clinical narratives. The SODA package with pre-trained transformer models is available at https://github.com/uf-hobi-informatics-lab/SODA_Docker.