Effect of mental health staffing inputs on initiation of care among recently separated Veterans.

OBJECTIVE: To estimate a causal relationship between mental health staffing and time to initiation of mental health care for new patients. DATA SOURCES AND STUDY SETTING: As the largest integrated health care delivery system in the United States, the Veterans Health Administration (VHA) provides a unique setting for isolating the effects of staffing on initiation of mental health care where demand is high and out-of-pocket costs are not a relevant confounder. We use data from the Department of Defense and VHA to obtain patient and facility characteristics and health care use. STUDY DESIGN: To isolate exogenous variation in mental health staffing, we used an instrumental variables approach-two-stage residual inclusion with a discrete time hazard model. Our outcome is time to initiation of mental health care after separation from active duty (first appointment) and our exposure is mental health staffing (standardized clinic time per 1000 VHA enrollees per pay period). DATA COLLECTION/EXTRACTION METHODS: Our cohort consists of all Veterans separating from active duty between July 2014 and September 2017, who were enrolled in the VHA, and had at least one diagnosis of post-traumatic stress disorder, major depressive disorder, and/or substance use disorder in the year prior to separation from active duty (N = 54,209). PRINCIPAL FINDINGS: An increase of 1 standard deviation in mental health staffing results in a higher likelihood of initiating mental health care (adjusted hazard ratio: 3.17, 95% confidence interval: 2.62, 3.84, p < 0.001). Models stratified by tertile of mental health staffing exhibit decreasing returns to scale. CONCLUSIONS: Increases in mental health staffing led to faster initiation of care and are especially beneficial in facilities where staffing is lower, although initiation of care appears capacity-limited everywhere.

A framework for inferring and analyzing pharmacotherapy treatment patterns.

BACKGROUND: To discover pharmacotherapy prescription patterns and their statistical associations with outcomes through a clinical pathway inference framework applied to real-world data. METHODS: We apply machine learning steps in our framework using a 2006 to 2020 cohort of veterans with major depressive disorder (MDD). Outpatient antidepressant pharmacy fills, dispensed inpatient antidepressant medications, emergency department visits, self-harm, and all-cause mortality data were extracted from the Department of Veterans Affairs Corporate Data Warehouse. RESULTS: Our MDD cohort consisted of 252,179 individuals. During the study period there were 98,417 emergency department visits, 1,016 cases of self-harm, and 1,507 deaths from all causes. The top ten prescription patterns accounted for 69.3% of the data for individuals starting antidepressants at the fluoxetine equivalent of 20-39 mg. Additionally, we found associations between outcomes and dosage change. CONCLUSIONS: For 252,179 Veterans who served in Iraq and Afghanistan with subsequent MDD noted in their electronic medical records, we documented and described the major pharmacotherapy prescription patterns implemented by Veterans Health Administration providers. Ten patterns accounted for almost 70% of the data. Associations between antidepressant usage and outcomes in observational data may be confounded. The low numbers of adverse events, especially those associated with all-cause mortality, make our calculations imprecise. Furthermore, our outcomes are also indications for both disease and treatment. Despite these limitations, we demonstrate the usefulness of our framework in providing operational insight into clinical practice, and our results underscore the need for increased monitoring during critical points of treatment.

Effect of mental health staffing inputs on suicide-related events.

OBJECTIVE: To estimate the effects of changes in Veterans Health Administration (VHA) mental health services staffing levels on suicide-related events among a cohort of Veterans. DATA SOURCES: Data were obtained from the VHA Corporate Data Warehouse, the Department of Defense and Veterans Administration Infrastructure for Clinical Intelligence, the VHA survey of enrollees, and customized VHA databases tracking suicide-related events. Geographic variables were obtained from the Area Health Resources Files and the Centers for Medicare and Medicaid Services. STUDY DESIGN: We used an instrumental variables (IV) design with a Heckman correction for non-random partial observability of the use of mental health services. The principal predictor was a measure of provider staffing per 10,000 enrollees. The outcome was the probability of a suicide-related event. DATA COLLECTION/EXTRACTION METHODS: Data were obtained for a cohort of Veterans who recently separated from active service. PRINCIPAL FINDINGS: From 2014 to 2018, the per-pay period probability of a suicide-related event among our cohort was 0.05%. We found that a 1% increase in mental health staffing led to a 1.6 percentage point reduction in suicide-related events. This was driven by the first tertile of staffing, suggesting diminishing returns to scale for mental health staffing. CONCLUSIONS: VHA facilities appear to be staffing-constrained when providing mental health care. Targeted increases in mental health staffing would be likely to reduce suicidality.

CDS-Compare: A Web Application for Machine Learning Assisted Curation of Clinical Order Sets.

Order sets that adhere to disease-specific guidelines have been shown to increase clinician efficiency and patient safety but curating these order sets, particularly for consistency across multiple sites, is difficult and time consuming. We created software called CDS-Compare to alleviate the burden on expert reviewers in rapidly and effectively curating large databases of order sets. We applied our clustering-based software to a database of NLP-processed order sets extracted from VA's Electronic Health Record, then had subject-matter experts review the web application version of our software for clustering validity.

A new methodological framework for hazard detection models in health information technology systems.

The adoption of health information technology (HIT) has facilitated efforts to increase the quality and efficiency of health care services and decrease health care overhead while simultaneously generating massive amounts of digital information stored in electronic health records (EHRs). However, due to patient safety issues resulting from the use of HIT systems, there is an emerging need to develop and implement hazard detection tools to identify and mitigate risks to patients. This paper presents a new methodological framework to develop hazard detection models and to demonstrate its capability by using the US Department of Veterans Affairs' (VA) Corporate Data Warehouse, the data repository for the VA's EHR. The overall purpose of the framework is to provide structure for research and communication about research results. One objective is to decrease the communication barriers between interdisciplinary research stakeholders and to provide structure for detecting hazards and risks to patient safety introduced by HIT systems through errors in the collection, transmission, use, and processing of data in the EHR, as well as potential programming or configuration errors in these HIT systems. A nine-stage framework was created, which comprises programs about feature extraction, detector development, and detector optimization, as well as a support environment for evaluating detector models. The framework forms the foundation for developing hazard detection tools and the foundation for adapting methods to particular HIT systems.

Analysis of the cognitive demands of electronic health record use.

The goal of this study was to elicit the cognitive demands facing clinicians when using an electronic health record (EHR) system and learn the cues and strategies expert clinicians rely on to manage those demands. This study differs from prior research by applying a joint cognitive systems perspective to examining the cognitive aspects of clinical work. We used a cognitive task analysis (CTA) method specifically tailored to elicit the cognitive demands of an EHR system from expert clinicians from different sites in a variety of inpatient and outpatient roles. The analysis of the interviews revealed 145 unique cognitive demands of using an EHR, which were organized into 22 distinct themes across seven broad categories. In addition to confirming previously published themes of cognitive demands, the main emergent themes of this study are: 1) The EHR does not help clinicians develop and maintain awareness of the big picture; 2) The EHR does not support clinicians' need to reason about patients' current and future states, including effects of potential treatments; and 3) The EHR limits agency of clinicians to work individually and collaboratively. Implications for theory and EHR design and evaluation are discussed.

Feature Engineering and Process Mining to Enable Hazard Detection in Health Information Technology.

In this work, we aim to enhance the reliability of health information technology (HIT) systems by detection of plausible HIT hazards in clinical order transactions. In the absence of well-defined event logs in corporate data warehouses, our proposed approach identifies relevant timestamped data fields that could indicate transactions in the clinical order life cycle generating raw event sequences. Subsequently, we adopt state transitions of the OASIS Human Task standard to map the raw event sequences and simplify the complex process that clinical radiology orders go through. We describe how the current approach provides the potential to investigate areas of improvement and potential hazards in HIT systems using process mining. The discussion concludes with a use case and opportunities for future applications.

Real-Time Automated Hazard Detection Framework for Health Information Technology Systems.

An increase in the reliability of Health Information Technology (HIT) will facilitate institutional trust and credibility of the systems. In this paper, we present an end-to-end framework for improving the reliability and performance of HIT systems. Specifically, we describe the system model, present some of the methods that drive the model, and discuss an initial implementation of two of the proposed methods using data from the Veterans Affairs HIT and Corporate Data Warehouse systems. The contributions of this paper, thus, include (1) the design of a system model for monitoring and detecting hazards in HIT systems, (2) a data-driven approach for analysing the health care data warehouse, (3) analytical methods for characterising and analysing failures in HIT systems, and (4) a tool architecture for generating and reporting hazards in HIT systems. Our goal is to work towards an automated system that will help identify opportunities for improvements in HIT systems.

A Tale of Two Databases: The DoD and VA Infrastructure for Clinical Intelligence (DaVINCI).

The Department of Defense (DoD) and Department of Veterans Affairs (VA) Infrastructure for Clinical Intelligence (DaVINCI) creates an electronic network between the two United States federal agencies that provides a consolidated view of electronic medical record data for both service members and Veterans. This inter-agency collaboration has created new opportunities for supporting transitions in clinical care, reporting to Congress, and longitudinal research.

Effect on VA Patient Satisfaction of Provider's Use of an Integrated Viewer of Multiple Electronic Health Records.

PURPOSE: To examine associations between patient perceptions that their provider was knowledgeable of their medical history and clinicians' early adoption of an application that presents providers with an integrated longitudinal view of a patient's electronic health records (EHR) from multiple healthcare systems. METHOD: This retrospective analysis utilizes provider audit logs from the Veterans Health Administration Joint Legacy Viewer (JLV) and patient responses to the Survey of Patient Healthcare Experiences Patient-Centered Medical Home (SHEP/PCMH) patient satisfaction survey (FY2016) to assess the relationship between the primary care provider being an early adopter of JLV and patient perception of the provider's knowledge of their medical history. Multivariate logistic regression models were used to control for patient age, race, sex education, health status, duration of patient-provider relationship, and provider characteristics. RESULTS: The study used responses from 203,903 patients to the SHEP-PCMH survey in FY2016 who received outpatient primary care services from 11,421 unique providers. Most (91%) clinicians had no JLV utilization in the 6 months prior to the studied patient visit. Controlling for patient demographics, length of the patient-provider relationship, and provider and facility characteristics, being an early adopter of the JLV system was associated with a 14% (adj OR 1.14, p < 0.000) increased odds that patients felt their provider was knowledgeable about their medical history. When evaluating the interaction between duration of patient-provider relationship and being an early adopter of JLV, a greater effect was seen with patient-provider relationships that were greater than 3 years (adj OR 1.23, p < 0.000), compared to those less than 3 years. CONCLUSIONS: Increasing the interoperability of medical information systems has the potential to improve both patient care and patient experience of care. This study demonstrates that early adopters of an integrated view of electronic health records from multiple delivery systems are more likely to have their patients report that their clinician was knowledgeable of their medical history. With provider payments often linked to patient satisfaction performance metrics, investments in interoperability may be worthwhile.

One-Way and Round-Trip Analysis Demonstrates Surprising Limitations of Standards-Based Terminology Maps.

The informatics community has a long-standing vision of freely flowing and highly re-usable patient-specific clinical data that improves care quality and safety. We sought to evaluate the extent to which a standards-based mapping approach is sufficient to support semantic interoperability. We simulated large-scale clinical data transmission and measured semantic success between VA and DoD systems via one-way testing (OWT) and round-trip testing (RTT). Simulations were accomplished via SQL queries and production standards-based maps for medications, allergens, document titles, vitals and payers. Success rates for mapping local codes to national standards varied from 62.5% for DoD document titles and medications, to 100% for VA and DoD vital signs. Successful, one-way testing was considerably lower, ranging from 8.52% to 62.7%. Round-trip success rates were lower still, ranging from 1.7% to 76.3%. We present an error framework, lessons learned, and proposed mitigating steps to enhance standards-based semantic interoperability.

Effect of health information exchange on recognition of medication discrepancies is interrupted when data charges are introduced: results of a cluster-randomized controlled trial.

OBJECTIVES: To determine the effect of health information exchange (HIE) on medication prescribing for hospital inpatients in a cluster-randomized controlled trial, and to examine the prescribing effect of availability of information from a large pharmacy insurance plan in a natural experiment. METHODS: Patients admitted to an urban hospital received structured medication reconciliation by an intervention pharmacist with (intervention) or without (control) access to a regional HIE. The HIE contained prescribing information from the largest hospitals and pharmacy insurance plan in the region for the first 10 months of the study, but only from the hospitals for the last 21 months, when data charges were imposed by the insurance plan. The primary endpoint was discrepancies between preadmission and inpatient medication regimens, and secondary endpoints included adverse drug events (ADEs) and proportions of rectified discrepancies. RESULTS: Overall, 186 and 195 patients were assigned to intervention and control, respectively. Patients were 60 years old on average and took a mean of 7 medications before admission. There was no difference between intervention and control in number of risk-weighted discrepancies (6.4 vs 5.8, P = .452), discrepancy-associated ADEs (0.102 vs 0.092 per admission, P = .964), or rectification of discrepancies (0.026 vs 0.036 per opportunity, P = .539). However, patients who received medication reconciliation with pharmacy insurance data available had more risk-weighted medication discrepancies identified than those who received usual care (8.0 vs 5.9, P = .038). DISCUSSION AND CONCLUSION: HIE may improve outcomes of medication reconciliation. Charging for access to medication information interrupts this effect. Efforts are needed to understand and increase prescribers' rectification of medication discrepancies.

Translation of Contextual Control Model to chronic disease management: A paradigm to guide design of cognitive support systems.

BACKGROUND: Electronic health records (EHRs) continue to be criticized for providing poor cognitive support. Defining cognitive support has lacked theoretical foundation. We developed a measurement model of cognitive support based on the Contextual Control Model (COCOM), which describes control characteristics of an "orderly" joint system and proposes 4 levels of control: scrambled, opportunistic, tactical, and strategic. METHODS: 35 clinicians (5 centers) were interviewed pre and post outpatient clinical visits and audiotaped during the visit. Behaviors pertaining to hypertension management were systematically mapped to the COCOM control characteristics of: (1) time horizon, (2) uncertainty assessment, (3) consideration of multiple goals, (4) causal model described, and (5) explicitness of plan. Each encounter was classified for overall mode of control. Visits with deviation versus no deviation from hypertension goals were compared. RESULTS: Reviewer agreement was high. Control characteristics differed significantly between deviation groups (Wilcox rank sum p<.01). K-means cluster analysis of control characteristics, stratified by deviation were distinct, with higher goal deviations associated with more control characteristics. CONCLUSION: The COCOM control characteristics appear to be areas of potential yield for improved user-experience design.

Assessing the potential adoption and usefulness of concurrent, action-oriented, electronic adverse drug event triggers designed for the outpatient setting.

BACKGROUND: Adverse drug event (ADE) detection is an important priority for patient safety research. Trigger tools have been developed to help identify ADEs. In previous work we developed seven concurrent, action-oriented, electronic trigger algorithms designed to prompt clinicians to address ADEs in outpatient care. OBJECTIVES: We assessed the potential adoption and usefulness of the seven triggers by testing the positive predictive validity and obtaining stakeholder input. METHODS: We adapted ADE triggers, "bone marrow toxin-white blood cell count (BMT-WBC)," "bone marrow toxin - platelet (BMT-platelet)," "potassium raisers," "potassium reducers," "creatinine," "warfarin," and "sedative hypnotics," with logic to suppress flagging events with evidence of clinical intervention and applied the triggers to 50,145 patients from three large health care systems. Four pharmacists assessed trigger positive predictive value (PPV) with respect to ADE detection (conservatively excluding ADEs occurring during clinically appropriate care) and clinical usefulness (i.e., whether the trigger alert could change care to prevent harm). We measured agreement between raters using the free kappa and assessed positive PPV for the trigger's detection of harm, clinical usefulness, and both. Stakeholders from the participating health care systems rated the likelihood of trigger adoption and the perceived ease of implementation. FINDINGS: Agreement between pharmacist raters was moderately high for each ADE trigger (kappa free > 0.60). Trigger PPVs for harm ranged from 0 (Creatinine, BMT-WBC) to 17 percent (potassium raisers), while PPV for care change ranged from 0 (WBC) to 60 percent (Creatinine). Fifteen stakeholders rated the triggers. Our assessment identified five of the seven triggers as good candidates for implementation: Creatinine, BMT-Platelet, Potassium Raisers, Potassium Reducers, and Warfarin. CONCLUSIONS: At least five outpatient ADE triggers performed well and merit further evaluation in outpatient clinical care. When used in real time, these triggers may promote care changes to ameliorate patient harm.

Comparing fracture absolute risk assessment (FARA) tools: an osteoporosis clinical informatics tool to improve identification and care of men at high risk of first fracture.

BACKGROUND: Fracture absolute risk assessment (FARA) is recommended for guiding osteoporosis treatment decisions in males. The best strategy for applying FARA in the clinic setting is not known. OBJECTIVES: We compared 2 FARA tools for use with electronic health records (EHRs) to determine which would more accurately identify patients known to be high risk for fracture. Tools evaluated were an adaptation of the World Health Organization's Fracture Risk Assessment Tool used with electronic data (eFRAX) and the Veterans Affairs (VA)-based tool, VA-FARA. METHODS: We compared accuracies of VA-FARA and eFRAX for correctly classifying male veterans who fractured and who were seen in the VA's Sierra Pacific Network in 2002-2013. We then matched those cases to nonfracture controls to compare odds of fracture in patients classified as high risk by either tool. RESULTS: Among 8740 patients, the mean (SD) age was 67.0 (11.1) years. Based on risk factors present in the EHR, VA-FARA correctly classified 40.1% of fracture patients as high risk (33.0% and 34.6% for hip and any major fracture, respectively); eFRAX classified 17.4% correctly (17.4% for hip and 0.2% for any major fracture). Compared with non-high-risk patients, those classified as high risk by VA-FARA were 35% more likely to fracture (95% CI = 23%-47%; P < 0.01) compared with 17% for eFRAX (95% CI = 5%-32%; P < 0.01). CONCLUSIONS: VA-FARA is more predictive of first fracture than eFRAX using EHR data. Decision support tools based on VA-FARA may improve early identification and care of men at risk.

pSCANNER: patient-centered Scalable National Network for Effectiveness Research.

This article describes the patient-centered Scalable National Network for Effectiveness Research (pSCANNER), which is part of the recently formed PCORnet, a national network composed of learning healthcare systems and patient-powered research networks funded by the Patient Centered Outcomes Research Institute (PCORI). It is designed to be a stakeholder-governed federated network that uses a distributed architecture to integrate data from three existing networks covering over 21 million patients in all 50 states: (1) VA Informatics and Computing Infrastructure (VINCI), with data from Veteran Health Administration's 151 inpatient and 909 ambulatory care and community-based outpatient clinics; (2) the University of California Research exchange (UC-ReX) network, with data from UC Davis, Irvine, Los Angeles, San Francisco, and San Diego; and (3) SCANNER, a consortium of UCSD, Tennessee VA, and three federally qualified health systems in the Los Angeles area supplemented with claims and health information exchange data, led by the University of Southern California. Initial use cases will focus on three conditions: (1) congestive heart failure; (2) Kawasaki disease; (3) obesity. Stakeholders, such as patients, clinicians, and health service researchers, will be engaged to prioritize research questions to be answered through the network. We will use a privacy-preserving distributed computation model with synchronous and asynchronous modes. The distributed system will be based on a common data model that allows the construction and evaluation of distributed multivariate models for a variety of statistical analyses.

Support for contextual control in primary care: a qualitative analysis.

Providing support for high-level cognitive performance is largely missing in many decision support designs. Most development in this area is structured to minimize attention, decrease the need for deeper processing and limit intense goal-directed cognitive processing. However, from a dual process perspective, both automatic and deliberative processes need to be supported. The purpose of this qualitative analysis is to explore complex cognitive processing. We used the Contextual Control Model to guide the analysis. Transcripts from 33 taped primary care visits across 4 locations in the VA were analyzed using iterative process of construct and thematic development. Five themes related to high-level cognitive processes were identified: 1) Joint Exchange and Patient Activation; 2) Planning and Proactive Problem Solving; 3) Script and heuristic processing; 4) Time perspectives and 5) Uncertainty management. Results are discussed in terms of the need to support integrated views for complex situation mental models.

Comparing adherence to two different HIV antiretroviral regimens: an instrumental variable analysis.

The objective of this observational cohort study was to compare adherence to protease inhibitor (PI)-based regimens or non-nucleoside reverse transcriptase inhibitor (NNRTI)-based regimens. HIV-seropositive, antiretroviral-naïve patients initiating therapy between 1998 and 2006 were identified using Veterans Health Administration databases. First-year adherence ratios were calculated as proportion of days covered (PDC). Multivariable regressions were run with an indicator for PDC >95, 90, 85, and 80 % as the dependent variable and an indicator for a PI-based regimen as the key independent variable. We controlled for residual unmeasured confounding by indication using an instrumental variable technique, using the physician's prescribing preference as the instrument. Out of 929 veterans on PI-based and 747 on NNRTI-based regimens, only 19.7 % of PI patients had PDC >80 %, compared to 35.1 % of NNRTI patients. In multivariable analysis, starting a PI regimen was significantly associated with poor adherence for all 4 adherence thresholds using conventional regressions and instrumental variable methods.

Pharmacist- versus physician-initiated admission medication reconciliation: impact on adverse drug events.

BACKGROUND: Medication reconciliation (MR) has proven to be a problematic task for many hospitals to accomplish. It is important to know the clinical impact of physician- versus pharmacist-initiated MR in the resource-limited hospital environment. METHODS: This quasi-experimental study took place from December 2005 to February 2006 at an urban US Veterans Affairs hospital. MR was implemented on 2 similar general medical units: one received physician-initiated MR and the other received pharmacist-initiated MR. Adverse drug events (ADEs) and a 72-hour medication-prescribing risk score were ascertained by research pharmacists for all admitted patients by structured record review. Multivariable models were tested for intervention effect, accounting for quasi-experimental design and clustered observations, and were adjusted for patient and encounter covariates. RESULTS: Pharmacists completed the MR process in 102 admissions and physicians completed the process in 116 admissions. In completing the MR process, pharmacists documented statistically more admission medication changes than physicians (3.6 vs 0.8; P < 0.001). The adjusted odds of an ADE caused by an admission prescribing change with pharmacist-initiated MR compared with a physician-initiated MR were 1.04 with a 95% CI of 0.53 to 2.0. The adjusted odds of an ADE caused by an admission prescribing change that was a prescribing error with pharmacist-initiated MR compared with a physician-initiated MR were 0.38 with a confidence interval of 0.14 to 1.05. No difference was observed in 72-hour prescribing risk score (coefficient = 0.10; 95% CI, -0.54 to 0.75). CONCLUSION: MR performed by pharmacists versus physicians was more comprehensive and was followed by lower odds of ADEs from admission prescribing errors but with similar odds of all types of ADEs. Further research is warranted to examine how MR tasks may be optimally divided among clinicians and the mechanisms by which MR affects the likelihood of subsequent ADEs.