Effect of mental health staffing inputs on initiation of care among recently separated Veterans.

OBJECTIVE: To estimate a causal relationship between mental health staffing and time to initiation of mental health care for new patients. DATA SOURCES AND STUDY SETTING: As the largest integrated health care delivery system in the United States, the Veterans Health Administration (VHA) provides a unique setting for isolating the effects of staffing on initiation of mental health care where demand is high and out-of-pocket costs are not a relevant confounder. We use data from the Department of Defense and VHA to obtain patient and facility characteristics and health care use. STUDY DESIGN: To isolate exogenous variation in mental health staffing, we used an instrumental variables approach-two-stage residual inclusion with a discrete time hazard model. Our outcome is time to initiation of mental health care after separation from active duty (first appointment) and our exposure is mental health staffing (standardized clinic time per 1000 VHA enrollees per pay period). DATA COLLECTION/EXTRACTION METHODS: Our cohort consists of all Veterans separating from active duty between July 2014 and September 2017, who were enrolled in the VHA, and had at least one diagnosis of post-traumatic stress disorder, major depressive disorder, and/or substance use disorder in the year prior to separation from active duty (N = 54,209). PRINCIPAL FINDINGS: An increase of 1 standard deviation in mental health staffing results in a higher likelihood of initiating mental health care (adjusted hazard ratio: 3.17, 95% confidence interval: 2.62, 3.84, p < 0.001). Models stratified by tertile of mental health staffing exhibit decreasing returns to scale. CONCLUSIONS: Increases in mental health staffing led to faster initiation of care and are especially beneficial in facilities where staffing is lower, although initiation of care appears capacity-limited everywhere.

Defining the Subtypes of Long COVID and Risk Factors for Prolonged Disease: Population-Based Case-Crossover Study.

BACKGROUND: There have been over 772 million confirmed cases of COVID-19 worldwide. A significant portion of these infections will lead to long COVID (post-COVID-19 condition) and its attendant morbidities and costs. Numerous life-altering complications have already been associated with the development of long COVID, including chronic fatigue, brain fog, and dangerous heart rhythms. OBJECTIVE: We aim to derive an actionable long COVID case definition consisting of significantly increased signs, symptoms, and diagnoses to support pandemic-related clinical, public health, research, and policy initiatives. METHODS: This research employs a case-crossover population-based study using International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) data generated at Veterans Affairs medical centers nationwide between January 1, 2020, and August 18, 2022. In total, 367,148 individuals with ICD-10-CM data both before and after a positive COVID-19 test were selected for analysis. We compared ICD-10-CM codes assigned 1 to 7 months following each patient's positive test with those assigned up to 6 months prior. Further, 350,315 patients had novel codes assigned during this window of time. We defined signs, symptoms, and diagnoses as being associated with long COVID if they had a novel case frequency of ≥1:1000, and they significantly increased in our entire cohort after a positive test. We present odds ratios with CIs for long COVID signs, symptoms, and diagnoses, organized by ICD-10-CM functional groups and medical specialty. We used our definition to assess long COVID risk based on a patient's demographics, Elixhauser score, vaccination status, and COVID-19 disease severity. RESULTS: We developed a long COVID definition consisting of 323 ICD-10-CM diagnosis codes grouped into 143 ICD-10-CM functional groups that were significantly increased in our 367,148 patient post-COVID-19 population. We defined 17 medical-specialty long COVID subtypes such as cardiology long COVID. Patients who were COVID-19-positive developed signs, symptoms, or diagnoses included in our long COVID definition at a proportion of at least 59.7% (268,320/449,450, based on a denominator of all patients who were COVID-19-positive). The long COVID cohort was 8 years older with more comorbidities (2-year Elixhauser score 7.97 in the patients with long COVID vs 4.21 in the patients with non-long COVID). Patients who had a more severe bout of COVID-19, as judged by their minimum oxygen saturation level, were also more likely to develop long COVID. CONCLUSIONS: An actionable, data-driven definition of long COVID can help clinicians screen for and diagnose long COVID, allowing identified patients to be admitted into appropriate monitoring and treatment programs. This long COVID definition can also support public health, research, and policy initiatives. Patients with COVID-19 who are older or have low oxygen saturation levels during their bout of COVID-19, or those who have multiple comorbidities should be preferentially watched for the development of long COVID.

A framework for inferring and analyzing pharmacotherapy treatment patterns.

BACKGROUND: To discover pharmacotherapy prescription patterns and their statistical associations with outcomes through a clinical pathway inference framework applied to real-world data. METHODS: We apply machine learning steps in our framework using a 2006 to 2020 cohort of veterans with major depressive disorder (MDD). Outpatient antidepressant pharmacy fills, dispensed inpatient antidepressant medications, emergency department visits, self-harm, and all-cause mortality data were extracted from the Department of Veterans Affairs Corporate Data Warehouse. RESULTS: Our MDD cohort consisted of 252,179 individuals. During the study period there were 98,417 emergency department visits, 1,016 cases of self-harm, and 1,507 deaths from all causes. The top ten prescription patterns accounted for 69.3% of the data for individuals starting antidepressants at the fluoxetine equivalent of 20-39 mg. Additionally, we found associations between outcomes and dosage change. CONCLUSIONS: For 252,179 Veterans who served in Iraq and Afghanistan with subsequent MDD noted in their electronic medical records, we documented and described the major pharmacotherapy prescription patterns implemented by Veterans Health Administration providers. Ten patterns accounted for almost 70% of the data. Associations between antidepressant usage and outcomes in observational data may be confounded. The low numbers of adverse events, especially those associated with all-cause mortality, make our calculations imprecise. Furthermore, our outcomes are also indications for both disease and treatment. Despite these limitations, we demonstrate the usefulness of our framework in providing operational insight into clinical practice, and our results underscore the need for increased monitoring during critical points of treatment.

EHR-BERT: A BERT-based model for effective anomaly detection in electronic health records.

OBJECTIVE: Physicians and clinicians rely on data contained in electronic health records (EHRs), as recorded by health information technology (HIT), to make informed decisions about their patients. The reliability of HIT systems in this regard is critical to patient safety. Consequently, better tools are needed to monitor the performance of HIT systems for potential hazards that could compromise the collected EHRs, which in turn could affect patient safety. In this paper, we propose a new framework for detecting anomalies in EHRs using sequence of clinical events. This new framework, EHR-Bidirectional Encoder Representations from Transformers (BERT), is motivated by the gaps in the existing deep-learning related methods, including high false negatives, sub-optimal accuracy, higher computational cost, and the risk of information loss. EHR-BERT is an innovative framework rooted in the BERT architecture, meticulously tailored to navigate the hurdles in the contemporary BERT method; thus, enhancing anomaly detection in EHRs for healthcare applications. METHODS: The EHR-BERT framework was designed using the Sequential Masked Token Prediction (SMTP) method. This approach treats EHRs as natural language sentences and iteratively masks input tokens during both training and prediction stages. This method facilitates the learning of EHR sequence patterns in both directions for each event and identifies anomalies based on deviations from the normal execution models trained on EHR sequences. RESULTS: Extensive experiments on large EHR datasets across various medical domains demonstrate that EHR-BERT markedly improves upon existing models. It significantly reduces the number of false positives and enhances the detection rate, thus bolstering the reliability of anomaly detection in electronic health records. This improvement is attributed to the model's ability to minimize information loss and maximize data utilization effectively. CONCLUSION: EHR-BERT showcases immense potential in decreasing medical errors related to anomalous clinical events, positioning itself as an indispensable asset for enhancing patient safety and the overall standard of healthcare services. The framework effectively overcomes the drawbacks of earlier models, making it a promising solution for healthcare professionals to ensure the reliability and quality of health data.

Effect of mental health staffing inputs on suicide-related events.

OBJECTIVE: To estimate the effects of changes in Veterans Health Administration (VHA) mental health services staffing levels on suicide-related events among a cohort of Veterans. DATA SOURCES: Data were obtained from the VHA Corporate Data Warehouse, the Department of Defense and Veterans Administration Infrastructure for Clinical Intelligence, the VHA survey of enrollees, and customized VHA databases tracking suicide-related events. Geographic variables were obtained from the Area Health Resources Files and the Centers for Medicare and Medicaid Services. STUDY DESIGN: We used an instrumental variables (IV) design with a Heckman correction for non-random partial observability of the use of mental health services. The principal predictor was a measure of provider staffing per 10,000 enrollees. The outcome was the probability of a suicide-related event. DATA COLLECTION/EXTRACTION METHODS: Data were obtained for a cohort of Veterans who recently separated from active service. PRINCIPAL FINDINGS: From 2014 to 2018, the per-pay period probability of a suicide-related event among our cohort was 0.05%. We found that a 1% increase in mental health staffing led to a 1.6 percentage point reduction in suicide-related events. This was driven by the first tertile of staffing, suggesting diminishing returns to scale for mental health staffing. CONCLUSIONS: VHA facilities appear to be staffing-constrained when providing mental health care. Targeted increases in mental health staffing would be likely to reduce suicidality.

Detecting anomalous sequences in electronic health records using higher-order tensor networks.

Detecting anomalous sequences is an integral part of building and protecting modern large-scale health information technology (HIT) systems. These HIT systems generate a large volume of records of patients' state and significant events, which provide a valuable resource to help improve clinical decisions, patient care processes, and other issues. However, detecting anomalous sequences in electronic health records (EHR) remains a challenge in healthcare applications for several reasons, including imbalances in the data, complexity of relationships between events in the sequence, and the curse of dimensionality. Conventional anomaly detection methods use the finite sequence of events to discriminate sequences. They fail to incorporate salient event details under variable higher-order dependencies (e.g., duration between events) that can provide better discrimination of sequences in their models. To address this problem, we propose event sequence and subsequence anomaly detection algorithms that (1) use network-based representations of interactions in the data, (2) account for variable higher-order dependencies in the data, and (3) incorporate events duration for adequate discrimination of the data. The proposed approach identifies anomalies by monitoring the change in the graph after the test sequence is removed from the network. The change is quantified using graph distance metrics so that dramatic changes in the network can be attributed to the removed sequence. Furthermore, the proposed subsequence algorithm recommends plausible paths and salient information for the detected anomalous subsequences. Our results show that the proposed event sequence anomaly detection algorithm outperforms the baseline methods for both synthetic data and real-world EHR data.

CDS-Compare: A Web Application for Machine Learning Assisted Curation of Clinical Order Sets.

Order sets that adhere to disease-specific guidelines have been shown to increase clinician efficiency and patient safety but curating these order sets, particularly for consistency across multiple sites, is difficult and time consuming. We created software called CDS-Compare to alleviate the burden on expert reviewers in rapidly and effectively curating large databases of order sets. We applied our clustering-based software to a database of NLP-processed order sets extracted from VA's Electronic Health Record, then had subject-matter experts review the web application version of our software for clustering validity.

Integrated Health Record Viewers and Reduction in Duplicate Medical Imaging: Retrospective Observational Analysis.

BACKGROUND: Health information exchange and multiplatform health record viewers support more informed medical decisions, improve quality of care, and reduce the risk of adverse outcomes due to fragmentation and discontinuity in care during transition of care. An example of a multiplatform health record viewer is the VA/DoD Joint Longitudinal Viewer (JLV), which supports the Department of Veterans Affairs (VA) and Department of Defense (DoD) health care providers with read-only access to patient medical records integrated from multiple sources. JLV is intended to support more informed medical decisions such as reducing duplicate medical imaging when previous image study results may meet current clinical needs. OBJECTIVE: We estimated the impact of provider usage of JLV on duplicate imaging for service members transitioning from the DoD to the VA health care system. METHODS: We conducted a retrospective cross-sectional study in fiscal year 2018 to examine the relationship between providers' use of JLV and the likelihood of ordering duplicate images. Our sample included recently separated service members who had a VA primary care visit in fiscal year 2018 within 90 days of a DoD imaging study. Patients who received at least one imaging study at VA within 90 days of a DoD imaging study of the same imaging mode and on the same body part are considered to have received potentially duplicate imaging studies. We use a logistic regression model with "JLV provider" (providers with 1 or more JLV audits in the prior 6 months) as the independent variable to estimate the relationship between JLV use and ordering of duplicate images. Control variables included provider image ordering rates in the prior 6 months, provider type, patient demographics (age, race, gender), and clinical characteristics (Elixhauser comorbidity score). RESULTS: Providers known to utilize JLV in the prior 6 months order fewer duplicate images relative to providers not utilizing JLV for similar visits over time (odds ratio 0.44, 95% CI 0.24-0.78; P=.005). This effect is robust across multiple specifications of linear and logistic regression models. The provider's practice pattern of ordering image studies and the patient's health status are powerful confounders. CONCLUSIONS: This study provides evidence that adoption of a longitudinal viewer of health records from multiple electronic health record systems is associated with a reduced likelihood of ordering duplicate images. Investments in health information exchange systems may be effective ways to improve the quality of care and reduce adverse outcomes for patients experiencing fragmentation and discontinuity of care.

A new methodological framework for hazard detection models in health information technology systems.

The adoption of health information technology (HIT) has facilitated efforts to increase the quality and efficiency of health care services and decrease health care overhead while simultaneously generating massive amounts of digital information stored in electronic health records (EHRs). However, due to patient safety issues resulting from the use of HIT systems, there is an emerging need to develop and implement hazard detection tools to identify and mitigate risks to patients. This paper presents a new methodological framework to develop hazard detection models and to demonstrate its capability by using the US Department of Veterans Affairs' (VA) Corporate Data Warehouse, the data repository for the VA's EHR. The overall purpose of the framework is to provide structure for research and communication about research results. One objective is to decrease the communication barriers between interdisciplinary research stakeholders and to provide structure for detecting hazards and risks to patient safety introduced by HIT systems through errors in the collection, transmission, use, and processing of data in the EHR, as well as potential programming or configuration errors in these HIT systems. A nine-stage framework was created, which comprises programs about feature extraction, detector development, and detector optimization, as well as a support environment for evaluating detector models. The framework forms the foundation for developing hazard detection tools and the foundation for adapting methods to particular HIT systems.

Analysis of the cognitive demands of electronic health record use.

The goal of this study was to elicit the cognitive demands facing clinicians when using an electronic health record (EHR) system and learn the cues and strategies expert clinicians rely on to manage those demands. This study differs from prior research by applying a joint cognitive systems perspective to examining the cognitive aspects of clinical work. We used a cognitive task analysis (CTA) method specifically tailored to elicit the cognitive demands of an EHR system from expert clinicians from different sites in a variety of inpatient and outpatient roles. The analysis of the interviews revealed 145 unique cognitive demands of using an EHR, which were organized into 22 distinct themes across seven broad categories. In addition to confirming previously published themes of cognitive demands, the main emergent themes of this study are: 1) The EHR does not help clinicians develop and maintain awareness of the big picture; 2) The EHR does not support clinicians' need to reason about patients' current and future states, including effects of potential treatments; and 3) The EHR limits agency of clinicians to work individually and collaboratively. Implications for theory and EHR design and evaluation are discussed.

Patient-centered care and the electronic health record: exploring functionality and gaps.

OBJECTIVE: Healthcare systems have adopted electronic health records (EHRs) to support clinical care. Providing patient-centered care (PCC) is a goal of many healthcare systems. In this study, we sought to explore how existing EHR systems support PCC; defined as understanding the patient as a whole person, building relational connections between the clinician and patient, and supporting patients in health self-management. MATERIALS AND METHODS: We assessed availability of EHR functions consistent with providing PCC including patient goals and preferences, integrated care plans, and contextual and patient-generated data. We surveyed and then interviewed technical representatives and expert clinical users of 6 leading EHR systems. Questions focused on the availability of specific data and functions related to PCC (for technical representatives) and the clinical usefulness of PCC functions (for clinicians) in their EHR. RESULTS: Technical representatives (n = 6) reported that patient communication preferences, personalized indications for medications, and end of life preferences were functions implemented across 6 systems. Clinician users (n = 10) reported moderate usefulness of PCC functions (medians of 2-4 on a 5-pointy -35t scale), suggesting the potential for improvement across systems. Interviews revealed that clinicians do not have a shared conception of PCC. In many cases, data needed to deliver PCC was available in the EHR only in unstructured form. Data systems and functionality to support PCC are under development in these EHRs. DISCUSSION AND CONCLUSION: There are current gaps in PCC functionality in EHRs and opportunities to support the practice of PCC through EHR redesign.

Feature Engineering and Process Mining to Enable Hazard Detection in Health Information Technology.

In this work, we aim to enhance the reliability of health information technology (HIT) systems by detection of plausible HIT hazards in clinical order transactions. In the absence of well-defined event logs in corporate data warehouses, our proposed approach identifies relevant timestamped data fields that could indicate transactions in the clinical order life cycle generating raw event sequences. Subsequently, we adopt state transitions of the OASIS Human Task standard to map the raw event sequences and simplify the complex process that clinical radiology orders go through. We describe how the current approach provides the potential to investigate areas of improvement and potential hazards in HIT systems using process mining. The discussion concludes with a use case and opportunities for future applications.

Real-Time Automated Hazard Detection Framework for Health Information Technology Systems.

An increase in the reliability of Health Information Technology (HIT) will facilitate institutional trust and credibility of the systems. In this paper, we present an end-to-end framework for improving the reliability and performance of HIT systems. Specifically, we describe the system model, present some of the methods that drive the model, and discuss an initial implementation of two of the proposed methods using data from the Veterans Affairs HIT and Corporate Data Warehouse systems. The contributions of this paper, thus, include (1) the design of a system model for monitoring and detecting hazards in HIT systems, (2) a data-driven approach for analysing the health care data warehouse, (3) analytical methods for characterising and analysing failures in HIT systems, and (4) a tool architecture for generating and reporting hazards in HIT systems. Our goal is to work towards an automated system that will help identify opportunities for improvements in HIT systems.

A Tale of Two Databases: The DoD and VA Infrastructure for Clinical Intelligence (DaVINCI).

The Department of Defense (DoD) and Department of Veterans Affairs (VA) Infrastructure for Clinical Intelligence (DaVINCI) creates an electronic network between the two United States federal agencies that provides a consolidated view of electronic medical record data for both service members and Veterans. This inter-agency collaboration has created new opportunities for supporting transitions in clinical care, reporting to Congress, and longitudinal research.

One-Way and Round-Trip Analysis Demonstrates Surprising Limitations of Standards-Based Terminology Maps.

The informatics community has a long-standing vision of freely flowing and highly re-usable patient-specific clinical data that improves care quality and safety. We sought to evaluate the extent to which a standards-based mapping approach is sufficient to support semantic interoperability. We simulated large-scale clinical data transmission and measured semantic success between VA and DoD systems via one-way testing (OWT) and round-trip testing (RTT). Simulations were accomplished via SQL queries and production standards-based maps for medications, allergens, document titles, vitals and payers. Success rates for mapping local codes to national standards varied from 62.5% for DoD document titles and medications, to 100% for VA and DoD vital signs. Successful, one-way testing was considerably lower, ranging from 8.52% to 62.7%. Round-trip success rates were lower still, ranging from 1.7% to 76.3%. We present an error framework, lessons learned, and proposed mitigating steps to enhance standards-based semantic interoperability.

Effect on VA Patient Satisfaction of Provider's Use of an Integrated Viewer of Multiple Electronic Health Records.

PURPOSE: To examine associations between patient perceptions that their provider was knowledgeable of their medical history and clinicians' early adoption of an application that presents providers with an integrated longitudinal view of a patient's electronic health records (EHR) from multiple healthcare systems. METHOD: This retrospective analysis utilizes provider audit logs from the Veterans Health Administration Joint Legacy Viewer (JLV) and patient responses to the Survey of Patient Healthcare Experiences Patient-Centered Medical Home (SHEP/PCMH) patient satisfaction survey (FY2016) to assess the relationship between the primary care provider being an early adopter of JLV and patient perception of the provider's knowledge of their medical history. Multivariate logistic regression models were used to control for patient age, race, sex education, health status, duration of patient-provider relationship, and provider characteristics. RESULTS: The study used responses from 203,903 patients to the SHEP-PCMH survey in FY2016 who received outpatient primary care services from 11,421 unique providers. Most (91%) clinicians had no JLV utilization in the 6 months prior to the studied patient visit. Controlling for patient demographics, length of the patient-provider relationship, and provider and facility characteristics, being an early adopter of the JLV system was associated with a 14% (adj OR 1.14, p < 0.000) increased odds that patients felt their provider was knowledgeable about their medical history. When evaluating the interaction between duration of patient-provider relationship and being an early adopter of JLV, a greater effect was seen with patient-provider relationships that were greater than 3 years (adj OR 1.23, p < 0.000), compared to those less than 3 years. CONCLUSIONS: Increasing the interoperability of medical information systems has the potential to improve both patient care and patient experience of care. This study demonstrates that early adopters of an integrated view of electronic health records from multiple delivery systems are more likely to have their patients report that their clinician was knowledgeable of their medical history. With provider payments often linked to patient satisfaction performance metrics, investments in interoperability may be worthwhile.

Interoperability Progress and Remaining Data Quality Barriers of Certified Health Information Technologies.

The Consolidated Clinical Document Architecture (C-CDA) is the primary standard for clinical document exchange in the United States. While document exchange is prevalent today, prior research has documented challenges to high quality, effective interoperability using this standard. Many electronic health records (EHRs) have recently been certified to a new version of the C-CDA standard as part of federal programs for EHR adoption. This renewed certification generated example documents from 52 health information technologies that have been made publicly available. This research applies automated tooling and manual inspection to evaluate conformance and data quality of these testing artifacts. It catalogs interoperability progress as well as remaining barriers to effective data exchange. Its findings underscore the importance of programs that evaluate data quality beyond schematron conformance to enable the high quality and safe exchange of clinical data.

Veterans Health Information Exchange: Successes and Challenges of Nationwide Interoperability.

Health Information Exchange (HIE) between organizations is necessary to have more complete information and coordinate care. Given its nationwide footprint, the Department of Veterans Affairs (VA) connects and exchanges health information with a large number of organizations and consequently has a unique opportunity to experience most issues affecting clinical interoperability. The Veterans HIE Program manages these exchanges by focusing on patient engagement, provider adoption, partner relationship, technology platform, and performance. This paper analyzes the success and challenges of this program over the last five years, and offers valuable metrics and insights. Although significant progress has been made, nationwide interoperability remains fragmented and laborious, requiring multiple agreements and exchange methods. A future nationwide interoperability would embrace a single "on-ramp" to connect to everyone.