Developing a multimedia patient-reported outcomes measure for low literacy patients with a human-centered design approach.

INTRODUCTION: Almost all patient-reported outcomes measures (PROMs) are text-based, which impedes accurate completion by low and limited literacy patients. Few PROMs are designed or validated to be self-administered, either in clinical or research settings, by patients of all literacy levels. We aimed to adapt the Patient Reported Outcomes Measurement Information System Upper Extremity Short Form (PROMIS-UE) to a multimedia version (mPROMIS-UE) that can be self-administered by hand and upper extremity patients of all literacy levels. METHODS: Our study in which we applied the Multimedia Adaptation Protocol included seven phases completed in a serial, iterative fashion: planning with our community advisory board; direct observation; discovery interviews with patients, caregivers, and clinic staff; ideation; prototyping; member-checking interviews; and feedback. Direct observations were documented in memos that underwent rapid thematic analysis. Interviews were audio-recorded and documented using analytic memos; a rapid, framework-guided thematic analysis with both inductive and deductive themes was performed. Themes were distilled into design challenges to guide ideation and prototyping that involved our multidisciplinary research team. To assess completeness, credibility, and acceptability we completed additional interviews with member-checking of initial findings and consulted our community advisory board. RESULTS: We conducted 12 hours of observations. We interviewed 17 adult English-speaking participants (12 patients, 3 caregivers, 2 staff) of mixed literacy. Our interviews revealed two distinct user personas and three distinct literacy personas; we developed the mPROMIS-UE with these personas in mind. Themes from interviews were distilled into four broad design challenges surrounding literacy, customizability, convenience, and shame. We identified features (audio, animations, icons, avatars, progress indicator, illustrated response scale) that addressed the design challenges. The last 6 interviews included member-checking; participants felt that the themes, design challenges, and corresponding features resonated with them. These features were synthesized into an mPROMIS-UE prototype that underwent rounds of iterative refinement, the last of which was guided by recommendations from our community advisory board. DISCUSSION: We successfully adapted the PROMIS-UE to an mPROMIS-UE that addresses the challenges identified by a mixed literacy hand and upper extremity patient cohort. This demonstrates the feasibility of adapting PROMs to multimedia versions. Future research will include back adaptation, usability testing via qualitative evaluation, and psychometric validation of the mPROMIS-UE. A validated mPROMIS-UE will expand clinicians' and investigators' ability to capture patient-reported outcomes in mixed literacy populations.

Development and Evaluation of Patient Safety Interventions: Perspectives of Operational Safety Leaders and Patient Safety Organizations.

OBJECTIVES: The purpose of this study is to understand how patient safety professionals from healthcare facilities and patient safety organizations develop patient safety interventions and the resources used to support intervention development. METHODS: Semistructured interviews were conducted with patient safety professionals at nine healthcare facilities and nine patient safety organizations. Interview data were qualitatively analyzed, and findings were organized by the following: patient safety solutions and interventions, use of external databases, and evaluation of patient safety solutions. RESULTS: Development of patient safety interventions across healthcare facilities and patient safety organizations was similar and included literature searches, internal brainstorming, and interviews. Nearly all patient safety professionals at healthcare facilities reported contacting colleagues at other healthcare facilities to learn about similar safety issues and potential interventions. Additionally, less than half of patient safety professionals at healthcare facilities and patient safety organizations interviewed report data to publicly available patient safety databases. Finally, most patient safety professionals at healthcare facilities and patient safety organizations stated that they evaluate the effectiveness of patient safety interventions; however, they mentioned methods that may be less rigorous including audits, self-reporting, and subjective judgment. CONCLUSIONS: Patient safety professionals often utilize similar methods and resources to develop and evaluate patient safety interventions; however, many of these efforts are not coordinated across healthcare organizations and could benefit from working collectively in a systematic fashion. Additionally, healthcare facilities and patient safety organizations face similar challenges and there are several opportunities for optimization on a national level that may improve patient safety.

Reading Room Interruptions are Less Disruptive When Using Asynchronous Communication Methods.

Radiologist interruptions, though often necessary, can be disruptive. Prior literature has shown interruptions to be frequent, occurring during cases, and predominantly through synchronous communication methods such as phone or in person causing significant disengagement from the study being read. Asynchronous communication methods are now more widely available in hospital systems such as ours. Considering the increasing use of asynchronous communication methods, we conducted an observational study to understand the evolving nature of radiology interruptions. We hypothesize that compared to interruptions occurring through synchronous methods, interruptions via asynchronous methods reduce the disruptive nature of interruptions by occurring between cases, being shorter, and less severe. During standard weekday hours, 30 radiologists (14 attendings, 12 residents, and 4 fellows) were directly observed for approximately 90-min sessions across three different reading rooms (body, neuroradiology, general). The frequency of interruptions was documented including characteristics such as timing, severity, method, and length. Two hundred twenty-five interruptions (43 Teams, 47 phone, 89 in-person, 46 other) occurred, averaging 2 min and 5 s with 5.2 interruptions per hour. Microsoft Teams interruptions averaged 1 min 12 s with only 60.5% during cases. In-person interruptions averaged 2 min 12 s with 82% during cases. Phone interruptions averaged 2 min and 48 s with 97.9% during cases. A substantial portion of reading room interruptions occur via predominantly asynchronous communication tools, a new development compared to prior literature. Interruptions via predominantly asynchronous communications tools are shorter and less likely to occur during cases. In our practice, we are developing tools and mechanisms to promote asynchronous communication to harness these benefits.

Accuracy of Noninvasive Blood Pressure Monitoring in Critically Ill Adults.

Background: Blood pressure (BP) is routinely invasively monitored by an arterial catheter in the intensive care unit (ICU). However, the available data comparing the accuracy of noninvasive methods to arterial catheters for measuring BP in the ICU are limited by small numbers and diverse methodologies. Purpose: To determine agreement between invasive arterial blood pressure monitoring (IABP) and noninvasive blood pressure (NIBP) in critically ill patients. Methods: This was a single center, observational study of critical ill adults in a tertiary care facility evaluating agreement (≤10% difference) between simultaneously measured IABP and NIBP. We measured clinical features at time of BP measurement inclusive of patient demographics, laboratory data, severity of illness, specific interventions (mechanical ventilation and dialysis), and vasopressor dose to identify particular clinical scenarios in which measurement agreement is more or less likely. Results: Of the 1852 critically ill adults with simultaneous IABP and NIBP readings, there was a median difference of 6 mm Hg in mean arterial pressure (MAP), interquartile range (1-12), P < .01. A logistic regression analysis identified 5 independent predictors of measurement discrepancy: increasing doses of norepinephrine (adjusted odds ratio [aOR] 1.10 [95% confidence interval, CI 1.08-1.12] P = .03 for every change in 5 µg/min), lower MAP value (aOR 0.98 [0.98-0.99] P < .01 for every change in 1 mm Hg), higher body mass index (aOR 1.04 [1.01-1.09] P = .01 for an increase in 1), increased patient age (aOR 1.31 [1.30-1.37] P < .01 for every 10 years), and radial arterial line location (aOR 1.74 [1.16-2.47] P = .04). Conclusions: There was broad agreement between IABP and NIBP in critically ill patients over a range of BPs and severity of illness. Several variables are associated with measurement discrepancy; however, their predictive capacity is modest. This may guide future study into which patients may specifically benefit from an arterial catheter.

A text mining approach to categorize patient safety event reports by medication error type.

Patient safety reporting systems give healthcare provider staff the ability to report medication related safety events and errors; however, many of these reports go unanalyzed and safety hazards go undetected. The objective of this study is to examine whether natural language processing can be used to better categorize medication related patient safety event reports. 3,861 medication related patient safety event reports that were previously annotated using a consolidated medication error taxonomy were used to develop three models using the following algorithms: (1) logistic regression, (2) elastic net, and (3) XGBoost. After development, models were tested, and model performance was analyzed. We found the XGBoost model performed best across all medication error categories. 'Wrong Drug', 'Wrong Dosage Form or Technique or Route', and 'Improper Dose/Dose Omission' categories performed best across the three models. In addition, we identified five words most closely associated with each medication error category and which medication error categories were most likely to co-occur. Machine learning techniques offer a semi-automated method for identifying specific medication error types from the free text of patient safety event reports. These algorithms have the potential to improve the categorization of medication related patient safety event reports which may lead to better identification of important medication safety patterns and trends.

A natural language processing approach to categorise contributing factors from patient safety event reports.

OBJECTIVES: The objective of this study was to explore the use of natural language processing (NLP) algorithm to categorise contributing factors from patient safety event (PSE). Contributing factors are elements in the healthcare process (eg, communication failures) that instigate an event or allow an event to occur. Contributing factors can be used to further investigate why safety events occurred. METHODS: We used 10 years of self-reported PSE reports from a multihospital healthcare system in the USA. Reports were first selected by event date. We calculated χ2 values for each ngram in the bag-of-words then selected N ngrams with the highest χ2 values. Then, PSE reports were filtered to only include the sentences containing the selected ngrams. Such sentences were called information-rich sentences. We compared two feature extraction techniques from free-text data: (1) baseline bag-of-words features and (2) features from information-rich sentences. Three machine learning algorithms were used to categorise five contributing factors representing sociotechnical errors: communication/hand-off failure, technology issue, policy/procedure issue, distractions/interruptions and lapse/slip. We trained 15 binary classifiers (five contributing factors * three machine learning models). The models' performances were evaluated according to the area under the precision-recall curve (AUPRC), precision, recall, and F1-score. RESULTS: Applying the information-rich sentence selection algorithm boosted the contributing factor categorisation performance. Comparing the AUPRCs, the proposed NLP approach improved the categorisation performance of two and achieved comparable results with baseline in categorising three contributing factors. CONCLUSIONS: Information-rich sentence selection can be incorporated to extract the sentences in free-text event narratives in which the contributing factor information is embedded.

HCV universal EHR prompt successfully increases screening, highlights potential disparities.

BACKGROUND & OBJECTIVES: Screening for hepatitis C virus is the first critical decision point for preventing morbidity and mortality from HCV cirrhosis and hepatocellular carcinoma and will ultimately contribute to global elimination of a curable disease. This study aims to portray the changes over time in HCV screening rates and the screened population characteristics following the 2020 implementation of an electronic health record (EHR) alert for universal screening in the outpatient setting in a large healthcare system in the US mid-Atlantic region. METHODS: Data was abstracted from the EHR on all outpatients from 1/1/2017 through 10/31/2021, including individual demographics and their HCV antibody (Ab) screening dates. For a limited period centered on the implementation of the HCV alert, mixed effects multivariable regression analyses were performed to compare the timeline and characteristics of those screened and un-screened. The final models included socio-demographic covariates of interest, time period (pre/post) and an interaction term between time period and sex. We also examined a model with time as a monthly variable to look at the potential impact of COVID-19 on screening for HCV. RESULTS: Absolute number of screens and screening rate increased by 103% and 62%, respectively, after adopting the universal EHR alert. Patients with Medicaid were more likely to be screened than private insurance (ORadj 1.10, 95% CI: 1.05, 1.15), while those with Medicare were less likely (ORadj 0.62, 95% CI: 0.62, 0.65); and Black (ORadj 1.59, 95% CI: 1.53, 1.64) race more than White. CONCLUSIONS: Implementation of universal EHR alerts could prove to be a critical next step in HCV elimination. Those with Medicare and Medicaid insurance were not screened proportionately to the national prevalence of HCV in these populations. Our findings support increased screening and re-testing efforts for those at high risk of HCV.

Symptom and Anatomical Phenotypes Provide Insights Into Interactions of Prolapse Symptoms and Anatomy.

IMPORTANCE: Women pursue treatment to relieve symptoms, while surgeons repair anatomy, underlining the importance of the relationship between symptoms and anatomy. OBJECTIVE: We hypothesized different anatomical and symptom phenotypes associated with pelvic organ prolapse (POP). Our objective was to investigate prevalence of phenotypes to explore associations of symptoms with anatomical defects. METHODS: We defined 420 anatomical phenotypes from combinations of POP Quantification parameters and 128 symptom phenotypes from symptoms described by condition-specific questionnaires (Pelvic Floor Disorders Inventory, Short Form of the Personal Experience Questionnaire). We applied these to an anonymized database of 719 subjects with symptomatic pelvic floor disorders. Bar graphs were used to illustrate the distribution of anatomical and symptom phenotypes, as well as anatomical phenotypes of patients with specific symptoms. We then used biclustering analysis with the multiple latent block model, to identify patterns of clustered groups of subjects and features. RESULTS: The most common symptom phenotypes have multiple (3-5) symptoms. A third of the theoretical anatomical phenotypes existed in our cohort. Bar graphs for specific symptom composites demonstrated unique distributions of anatomical phenotypes suggesting associations between anatomy and symptoms. Biclustering converged on 2 subject clusters (C1, C2) and 8 feature clusters. Cluster 1 (68%) represented a younger subpopulation with lower stage POP, more stress urinary incontinence and sexual dysfunction (P < 0.001 all). Cluster 2 had more protrusion (P < 0.001) and obstructed voiding (P = 0.001). Features that clustered together, such as stress urinary incontinence and sexual dysfunction, may represent underlying relationships. CONCLUSIONS: We demonstrated a relationship between locations of anatomical POP and certain symptoms, which may generate new hypotheses and guide clinical decision making.

Using Community Detection Techniques to Identify Themes in COVID-19-Related Patient Safety Event Reports.

OBJECTIVES: The COVID-19 pandemic has transformed how healthcare is delivered to patients. As the pandemic progresses and healthcare systems continue to adapt, it is important to understand how these changes in care have changed patient care. This study aims to use community detection techniques to identify and facilitate analysis of themes in patient safety event (PSE) reports to better understand COVID-19 pandemic's impact on patient safety. With this approach, we also seek to understand how community detection techniques can be used to better identify themes and extract information from PSE reports. METHODS: We used community detection techniques to group 2082 PSE reports from January 1, 2020, to January 31, 2021, that mentioned COVID-19 into 65 communities. We then grouped these communities into 8 clinically relevant themes for analysis. RESULTS: We found the COVID-19 pandemic is associated with the following clinically relevant themes: (1) errors due to new and unknown COVID-19 protocols/workflows; (2) COVID-19 patients developing pressure ulcers; (3) unsuccessful/incomplete COVID-19 testing; (4) inadequate isolation of COVID-19 patients; (5) inappropriate/inadequate care for COVID-19 patients; (6) COVID-19 patient falls; (7) delays or errors communicating COVID-19 test results; and (8) COVID-19 patients developing venous thromboembolism. CONCLUSIONS: Our study begins the long process of understanding new challenges created by the pandemic and highlights how machine learning methods can be used to understand these and similar challenges. Using community detection techniques to analyze PSE reports and identify themes within them can help give healthcare systems the necessary information to improve patient safety and the quality of care they deliver.

Evaluation of a Text Message-Based COVID-19 Vaccine Outreach Program Among Older Patients: Cross-sectional Study.

BACKGROUND: COVID-19 vaccines are vital tools in the defense against infection and serious disease due to SARS-CoV-2. There are many challenges to implementing mass vaccination campaigns for large, diverse populations from crafting vaccine promotion messages to reaching individuals in a timely and effective manner. During this unprecedented period, with COVID-19 mass vaccination campaigns essential for protecting vulnerable patient populations and attaining herd immunity, health care systems were faced with the dual challenges of vaccine outreach and distribution. OBJECTIVE: The aim of this cross-sectional study was to assess the effectiveness of a COVID-19 vaccine text outreach approach for patients aged 65 years and older. Our goal was to determine whether this approach was successful in scheduling patients for COVID-19 vaccine appointments. METHODS: We developed SMS text messages using the Tavoca platform. These messages informed patients of their vaccine eligibility and allowed them to indicate their interest in scheduling an appointment via a specific method (email or phone) or indicate their lack of interest in the vaccine. We tracked the status of these messages and how patients responded. Messages were sent to patients aged 65 years and older (N=30,826) at a nonprofit health care system in Washington, DC. Data were collected and examined from January 14 to May 10, 2021. Data were analyzed using multivariate multinomial and binary logistic regression models in SAS (version 9.4; SAS Institute Inc). RESULTS: Approximately 57% of text messages were delivered to patients, but many messages received no response from patients (40%). Additionally, 42.1% (12,978/30,826) of messages were not delivered. Of the patients who expressed interest in the vaccine (2938/30,826, 9.5%), Black or African American patients preferred a phone call rather than an email for scheduling their appointment (odds ratio [OR] 1.69, 95% CI 1.29-2.21) compared to White patients. Patients aged 70-74 years were more likely to schedule an appointment (OR 1.38, 95% CI 1.01-1.89) than those aged 65-69 years, and Black or African American patients were more likely to schedule an appointment (OR 2.90, 95% CI 1.72-4.91) than White patients. CONCLUSIONS: This study provides insights into some advantages and challenges of using a text messaging vaccine outreach for patients aged 65 years and older. Lessons learned from this vaccine campaign underscore the importance of using multiple outreach methods and sharing of patient vaccination status between health systems, along with a patient-centered approach to address vaccine hesitancy and access issues.

Usability and Accessibility of Publicly Available Patient Safety Databases.

OBJECTIVES: The aims of the study were to identify publicly available patient safety report databases and to determine whether these databases support safety analyst and data scientist use to identify patterns and trends. METHODS: An Internet search was conducted to identify publicly available patient safety databases that contained patient safety reports. Each database was analyzed to identify features that enable patient safety analyst and data scientist use of these databases. RESULTS: Seven databases (6 hosted by federal agencies, 1 hosted by a nonprofit organization) containing more than 28.3 million safety reports were identified. Some, but not all, databases contained features to support patient safety analyst use: 57.1% provided the ability to sort/compare/filter data, 42.9% provided data visualization, and 85.7% enabled free-text search. None of the databases provided regular updates or monitoring and only one database suggested solutions to patient safety reports. Analysis of features to support data scientist use showed that only 42.9% provided an application programing interface, most (85.7%) provided batch downloading, all provided documentation about the database, and 71.4% provided a data dictionary. All databases provided open access. Only 28.6% provided a data diagram. CONCLUSIONS: Patient safety databases should be improved to support patient safety analyst use by, at a minimum, allowing for data to be sorted/compared/filtered, providing data visualization, and enabling free-text search. Databases should also enable data scientist use by, at a minimum, providing an application programing interface, batch downloading, and a data dictionary.

Cluster Analysis of Primary Care Physician Phenotypes for Electronic Health Record Use: Retrospective Cohort Study.

BACKGROUND: Electronic health records (EHRs) have become ubiquitous in US office-based physician practices. However, the different ways in which users engage with EHRs remain poorly characterized. OBJECTIVE: The aim of this study is to explore EHR use phenotypes among ambulatory care physicians. METHODS: In this retrospective cohort analysis, we applied affinity propagation, an unsupervised clustering machine learning technique, to identify EHR user types among primary care physicians. RESULTS: We identified 4 distinct phenotype clusters generalized across internal medicine, family medicine, and pediatrics specialties. Total EHR use varied for physicians in 2 clusters with above-average ratios of work outside of scheduled hours. This finding suggested that one cluster of physicians may have worked outside of scheduled hours out of necessity, whereas the other preferred ad hoc work hours. The two remaining clusters represented physicians with below-average EHR time and physicians who spend the largest proportion of their EHR time on documentation. CONCLUSIONS: These findings demonstrate the utility of cluster analysis for exploring EHR use phenotypes and may offer opportunities for interventions to improve interface design to better support users' needs.

Application of artificial intelligence in cataract management: current and future directions.

The rise of artificial intelligence (AI) has brought breakthroughs in many areas of medicine. In ophthalmology, AI has delivered robust results in the screening and detection of diabetic retinopathy, age-related macular degeneration, glaucoma, and retinopathy of prematurity. Cataract management is another field that can benefit from greater AI application. Cataract  is the leading cause of reversible visual impairment with a rising global clinical burden. Improved diagnosis, monitoring, and surgical management are necessary to address this challenge. In addition, patients in large developing countries often suffer from limited access to tertiary care, a problem further exacerbated by the ongoing COVID-19 pandemic. AI on the other hand, can help transform cataract management by improving automation, efficacy and overcoming geographical barriers. First, AI can be applied as a telediagnostic platform to screen and diagnose patients with cataract using slit-lamp and fundus photographs. This utilizes a deep-learning, convolutional neural network (CNN) to detect and classify referable cataracts appropriately. Second, some of the latest intraocular lens formulas have used AI to enhance prediction accuracy, achieving superior postoperative refractive results compared to traditional formulas. Third, AI can be used to augment cataract surgical skill training by identifying different phases of cataract surgery on video and to optimize operating theater workflows by accurately predicting the duration of surgical procedures. Fourth, some AI CNN models are able to effectively predict the progression of posterior capsule opacification and eventual need for YAG laser capsulotomy. These advances in AI could transform cataract management and enable delivery of efficient ophthalmic services. The key challenges include ethical management of data, ensuring data security and privacy, demonstrating clinically acceptable performance, improving the generalizability of AI models across heterogeneous populations, and improving the trust of end-users.

A descriptive analysis of an on-demand telehealth approach for remote COVID-19 patient screening.

INTRODUCTION: COVID-19 requires methods for screening patients that adhere to physical distancing and other Centers for Disease Control and Prevention guidelines. There is little data on the use of on-demand telehealth to meet this need. METHODS: The functional performance of on-demand telehealth as a COVID-19 remote patient screening approach was conducted by analysing 9270 patient requests. RESULTS: Most on-demand telehealth requests (5712 of 9270 total requests; 61.6%) had a visit reason that was likely COVID-19 related. Of these, 79.1% (4518 of 5712) resulted in a completed encounter and 20.9% (1194 of 5712) resulted in left without being seen. Of the 4518 completed encounters, 19.1% were referred to an urgent care centre, emergency department or COVID-19 testing centre. The average completed encounter wait time was 26.5 min and the mean visit length was 8.8 min. For patients that completed an encounter 42.8% (1935 of 4518) stated they would have sought in-person care and 9.1% stated they would have done nothing if on-demand telehealth was unavailable. DISCUSSION: On-demand telehealth can serve as a low-barrier approach to screen patients for COVID-19. This approach can prevent patients from visiting healthcare facilities, which reduces physical contact and reduces healthcare worker use of personal protective equipment.

Social Determinants of Health and Patient Safety: An Analysis of Patient Safety Event Reports Related to Limited English-Proficient Patients.

BACKGROUND: Racial and ethnic disparities in healthcare safety have persisted for decades, particularly for patients with language barriers. Previous studies have investigated the frequency and nature of safety events impacting patients with language barriers; others have proposed solutions to fix them. A gap analysis, however, of how we are currently addressing safety issues and why these efforts have not been effective is lacking. METHOD: This analysis uses reports from a patient safety event reporting system. Reports contain information regarding no-harm (near miss) events and events where harm may have reached the patient. Reports occurring with patients with a preferred language other than English were extracted and analyzed to determine whether the language barrier contributed to the safety event, the language barrier was mentioned in the resolution, and themes were mentioned for addressing language barriers. RESULTS: A subset of 1553 events pertaining to non-English-speaking patients were first categorized as "likely" (3%), "plausibly" (10%), or "unlikely" (87%) related to the patient's language barrier. Second, events related to the patient's language barrier were categorized as directly addressing (19%), indirectly addressing (3%), not mentioning (69%) the language barrier, or containing insufficient information to determine whether the language barrier was addressed (7%). Third, thematic analysis revealed that the most common methods for addressing language barriers included presenting issues to interpreter services and subsequent use of interpreter services. CONCLUSIONS: This study found that it is challenging to determine the direct role of certain social determinants of health (e.g., language barriers) in safety events. In many cases, the language barrier was not addressed in the event report. Furthermore, when the language barrier was addressed, solution themes typically involved weaker, less sustainable suggested actions.

Realizing the Power of Text Mining and Natural Language Processing for Analyzing Patient Safety Event Narratives: The Challenges and Path Forward.

ABSTRACT: Patient safety event (PSE) reports are a useful lens to understand hazards and patient safety risks in healthcare systems. However, patient safety officers and analysts in healthcare systems and safety organizations are challenged to make sense of the ever-increasing volume of PSE reports, including the free-text narratives. As a result, there is a growing emphasis on applying text mining and natural language processing (NLP) approaches to assist in the processing and understanding of these narratives. Although text mining and NLP in healthcare have advanced significantly over the past decades, the utility of the resulting models, ontologies, and algorithms to analyze PSE narratives are limited given the unique difference and challenges in content and language between PSE narratives and clinical documentation. To promote the application of text mining and NLP for PSE narratives, these unique challenges must be addressed. Improving data access, developing NLP resources to practically use contributing factor taxonomies, and developing and adopting shared specifications for interoperability will help create an infrastructure and environment that unlocks the collaborative potential between patient safety, research, and machine learning communities, in the development of reproducible and generalizable methods and models to better understand and improve patient safety and patient care.

Analysis of Electronic Health Record Use and Clinical Productivity and Their Association With Physician Turnover.

Importance: Physician turnover takes a heavy toll on patients, physicians, and health care organizations. Survey research has established associations of electronic health record (EHR) use with professional burnout and reduction in professional effort, but these findings are subject to response fatigue and bias. Objective: To evaluate the association of physician productivity and EHR use patterns, as determined by vendor-derived EHR use data platforms, with physician turnover. Design, Setting, and Participants: This retrospective cohort study was conducted among nonteaching ambulatory physicians at a large ambulatory practice network based in New England. Data were collected from March 2018 to February 2020. Main Outcomes and Measures: Physician departure from the practice network; 4 time-based core measures of EHR use, normalized to 8 hours of scheduled clinical time; teamwork, percentage of a physician's orders that are placed by other members of the care team; and productivity measures of patient volume, intensity, and demand. Results: Among 335 physicians assessed for eligibility, 314 unique physicians (89.2%) were included in the analysis (123 [39%] women; 100 [32%] aged 45-54 years), with 5663 physician-months of data. The turnover rate was 5.1%/year (32 of 314 physicians). Physicians completed a mean 2.6 appointments/hour (95% CI, 2.5-2.6 appointments/hour) and 206 appointments/month (95% CI, 197-215 appointments/month) with 5.5 hours (95% CI, 5.3-5.8 hours) of EHR time for every 8 hours of scheduled patient time. After controlling for gender, medical specialty, and time, the following variables were associated with turnover: inbox time (odds ratio [OR], 0.70; 95% CI, 0.61-0.82; P < .001), teamwork (OR, 0.68; 95% CI, 0.52-0.87; P = .003), demand (ie, proportion of available appointments filled: OR, 0.49; 95% CI, 0.35-0.70; P < .001), and age 45 to 54 years vs 25 to 34 years (OR, 0.19; 95% CI, 0.04-0.93; P = .04). Conclusions and Relevance: In this study, physician productivity and EHR use metrics were associated with physician departure. Prospectively tracking these metrics could identify physicians at high risk of departure who would benefit from early, team-based, targeted interventions. The counterintuitive finding that less time spent on the EHR (in particular inbox management) was associated with physician departure warrants further investigation.

Using Machine Learning to Capture Quality Metrics from Natural Language: A Case Study of Diabetic Eye Exams.

BACKGROUND AND OBJECTIVE: The prevalence of value-based payment models has led to an increased use of the electronic health record to capture quality measures, necessitating additional documentation requirements for providers. METHODS: This case study uses text mining and natural language processing techniques to identify the timely completion of diabetic eye exams (DEEs) from 26,203 unique clinician notes for reporting as an electronic clinical quality measure (eCQM). Logistic regression and support vector machine (SVM) using unbalanced and balanced datasets, using the synthetic minority over-sampling technique (SMOTE) algorithm, were evaluated on precision, recall, sensitivity, and f1-score for classifying records positive for DEE. We then integrate a high precision DEE model to evaluate free-text clinical narratives from our clinical EHR system. RESULTS: Logistic regression and SVM models had comparable f1-score and specificity metrics with models trained and validated with no oversampling favoring precision over recall. SVM with and without oversampling resulted in the best precision, 0.96, and recall, 0.85, respectively. These two SVM models were applied to the unannotated 31,585 text segments representing 24,823 unique records and 13,714 unique patients. The number of records classified as positive for DEE using the SVM models ranged from 667 to 8,935 (2.7-36% out of 24,823, respectively). Unique patients classified as positive for DEE ranged from 3.5 to 41.8% highlighting the potential utility of these models. DISCUSSION: We believe the impact of oversampling on SVM model performance to be caused by the potential of overfitting of the SVM SMOTE model on the synthesized data and the data synthesis process. However, the specificities of SVM with and without SMOTE were comparable, suggesting both models were confident in their negative predictions. By prioritizing to implement the SVM model with higher precision over sensitivity or recall in the categorization of DEEs, we can provide a highly reliable pool of results that can be documented through automation, reducing the burden of secondary review. Although the focus of this work was on completed DEEs, this method could be applied to completing other necessary documentation by extracting information from natural language in clinician notes. CONCLUSION: By enabling the capture of data for eCQMs from documentation generated by usual clinical practice, this work represents a case study in how such techniques can be leveraged to drive quality without increasing clinician work.