Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation.

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Outcome after surgical and conservative treatments of canine peritoneopericardial diaphragmatic hernia: A multi-institutional study of 128 dogs.

OBJECTIVE: To compare demographics and disease characteristics in dogs in which peritoneopericardial diaphragmatic hernia (PPDH) had been diagnosed and report outcomes after surgical treatment (ST) or conservative treatment (CT). STUDY DESIGN: Retrospective study. SAMPLE POPULATION: One hundred twenty-eight dogs (91 ST, 37 CT) in which PPDH had been diagnosed. METHODS: Medical records were reviewed for demographics, perioperative findings, and outcomes. Follow-up was obtained via telephone interview and email correspondence with owners and referring veterinarians. Baseline variables were compared between treatment groups. RESULTS: Dogs treated surgically were younger (P < .001), more likely to be sexually intact (P = .002), more likely to have clinical signs from PPDH vs an incidental diagnosis (P < .001), and more likely to have other congenital abnormalities (P = .003) compared with dogs treated conservatively. Ninety-seven percent of ST dogs were discharged from hospitals. Intraoperative and postoperative complications were reported in 22% and 41% of dogs, respectively, although most complications were classified as low grade (75% and 83%, respectively). Follow-up was available in 87 dogs, at a median of 1062 days. Hernia recurrence was not reported in any surgically treated dog. The deaths of nine dogs (five ST, four CT) could be attributed to PPDH, and long median survival times were observed in both the ST and CT groups (8.2 and 5 years, respectively). CONCLUSION: Preoperative characteristics differed between dogs treated conservatively vs surgically. Surgical treatment was associated with low operative mortality, and both ST and CT dogs had good long-term survival. CLINICAL SIGNIFICANCE: A diagnosis of PPDH can confer a good long-term prognosis for both ST and CT dogs.

Presumptive seeding metastasis of pericardial mesothelioma following repeated pericardiocentesis in a dog.

A dog with a history of recurrent pericardial effusion that required repeated pericardiocentesis was presented to the surgical service at the Ontario Veterinary College Health Sciences Centre for thoracoscopic pericardiectomy. Physical examination revealed a subcutaneous mass in the right lateral thorax. Cytology of the subcutaneous mass and histopathology of the pericardium were consistent with mesothelioma. This article details the first reported case of pericardial mesothelioma with suspected extra-thoracic metastasis following pericardiocentesis in a dog.

Implantation métastasique présumée d'un mésothéliome péricardique à la suite de péricardiocentèses répétées chez un chien. Un chien avec une historique d'effusions péricardiques récurrentes qui nécessitaient des péricardiocentèses répétées fut présenté au service de chirurgie du Ontario Veterinary College Health Sciences Centre pour une péricardiectomie thoracoscopique. L'examen physique a révélé une masse souscutanée dans le thorax latéral droit. L'examen cytologique de la masse sous-cutanée et l'histopathologie du péricarde étaient cohérents avec un mésothéliome. Le présent article donne les détails du premier cas rapporté chez un chien de mésothéliome péricardique avec métastase extra-thoracique suspectée consécutive à la suite de péricardiocentèses.(Traduit par Dr Serge Messier).

Using CDS Hooks to increase SMART on FHIR app utilization: a cluster-randomized trial.

OBJECTIVE: HL7 SMART on FHIR apps have the potential to improve healthcare delivery and EHR usability, but providers must be aware of the apps and use them for these potential benefits to be realized. The HL7 CDS Hooks standard was developed in part for this purpose. The objective of this study was to determine if contextually relevant CDS Hooks prompts can increase utilization of a SMART on FHIR medical reference app (MDCalc for EHR). MATERIALS AND METHODS: We conducted a 7-month, provider-randomized trial with 70 providers in a single emergency department. The intervention was a collection of CDS Hooks prompts suggesting the use of 6 medical calculators in a SMART on FHIR medical reference app. The primary outcome was the percentage of provider-patient interactions in which the app was used to view a recommended calculator. Secondary outcomes were app usage stratified by individual calculators. RESULTS: Intervention group providers viewed a study calculator in the app in 6.0% of interactions compared to 2.6% in the control group (odds ratio = 2.45, 95% CI, 1.2-5.2, P value .02), an increase of 130%. App use was significantly greater for 2 of 6 calculators. DISCUSSION AND CONCLUSION: Contextually relevant CDS Hooks prompts led to a significant increase in SMART on FHIR app utilization. This demonstrates the potential of using CDS Hooks to guide appropriate use of SMART on FHIR apps and was a primary motivation for the development of the standard. Future research may evaluate potential impacts on clinical care decisions and outcomes.

The potential for leveraging machine learning to filter medication alerts.

OBJECTIVE: To evaluate the potential for machine learning to predict medication alerts that might be ignored by a user, and intelligently filter out those alerts from the user's view. MATERIALS AND METHODS: We identified features (eg, patient and provider characteristics) proposed to modulate user responses to medication alerts through the literature; these features were then refined through expert review. Models were developed using rule-based and machine learning techniques (logistic regression, random forest, support vector machine, neural network, and LightGBM). We collected log data on alerts shown to users throughout 2019 at University of Utah Health. We sought to maximize precision while maintaining a false-negative rate <0.01, a threshold predefined through discussion with physicians and pharmacists. We developed models while maintaining a sensitivity of 0.99. Two null hypotheses were developed: H1-there is no difference in precision among prediction models; and H2-the removal of any feature category does not change precision. RESULTS: A total of 3,481,634 medication alerts with 751 features were evaluated. With sensitivity fixed at 0.99, LightGBM achieved the highest precision of 0.192 and less than 0.01 for the pre-defined maximal false-negative rate by subject-matter experts (H1) (P < 0.001). This model could reduce alert volume by 54.1%. We removed different combinations of features (H2) and found that not all features significantly contributed to precision. Removing medication order features (eg, dosage) most significantly decreased precision (-0.147, P = 0.001). CONCLUSIONS: Machine learning potentially enables the intelligent filtering of medication alerts.

Identifying Patients Who Meet Criteria for Genetic Testing of Hereditary Cancers Based on Structured and Unstructured Family Health History Data in the Electronic Health Record: Natural Language Processing Approach.

BACKGROUND: Family health history has been recognized as an essential factor for cancer risk assessment and is an integral part of many cancer screening guidelines, including genetic testing for personalized clinical management strategies. However, manually identifying eligible candidates for genetic testing is labor intensive. OBJECTIVE: The aim of this study was to develop a natural language processing (NLP) pipeline and assess its contribution to identifying patients who meet genetic testing criteria for hereditary cancers based on family health history data in the electronic health record (EHR). We compared an algorithm that uses structured data alone with structured data augmented using NLP. METHODS: Algorithms were developed based on the National Comprehensive Cancer Network (NCCN) guidelines for genetic testing for hereditary breast, ovarian, pancreatic, and colorectal cancers. The NLP-augmented algorithm uses both structured family health history data and the associated unstructured free-text comments. The algorithms were compared with a reference standard of 100 patients with a family health history in the EHR. RESULTS: Regarding identifying the reference standard patients meeting the NCCN criteria, the NLP-augmented algorithm compared with the structured data algorithm yielded a significantly higher recall of 0.95 (95% CI 0.9-0.99) versus 0.29 (95% CI 0.19-0.40) and a precision of 0.99 (95% CI 0.96-1.00) versus 0.81 (95% CI 0.65-0.95). On the whole data set, the NLP-augmented algorithm extracted 33.6% more entities, resulting in 53.8% more patients meeting the NCCN criteria. CONCLUSIONS: Compared with the structured data algorithm, the NLP-augmented algorithm based on both structured and unstructured family health history data in the EHR increased the number of patients identified as meeting the NCCN criteria for genetic testing for hereditary breast or ovarian and colorectal cancers.

Correction: Identifying Patients Who Meet Criteria for Genetic Testing of Hereditary Cancers Based on Structured and Unstructured Family Health History Data in the Electronic Health Record: Natural Language Processing Approach.

[This corrects the article DOI: 10.2196/37842.].

GARDE: a standards-based clinical decision support platform for identifying population health management cohorts.

Population health management (PHM) is an important approach to promote wellness and deliver health care to targeted individuals who meet criteria for preventive measures or treatment. A critical component for any PHM program is a data analytics platform that can target those eligible individuals. OBJECTIVE: The aim of this study was to design and implement a scalable standards-based clinical decision support (CDS) approach to identify patient cohorts for PHM and maximize opportunities for multi-site dissemination. MATERIALS AND METHODS: An architecture was established to support bidirectional data exchanges between heterogeneous electronic health record (EHR) data sources, PHM systems, and CDS components. HL7 Fast Healthcare Interoperability Resources and CDS Hooks were used to facilitate interoperability and dissemination. The approach was validated by deploying the platform at multiple sites to identify patients who meet the criteria for genetic evaluation of familial cancer. RESULTS: The Genetic Cancer Risk Detector (GARDE) platform was created and is comprised of four components: (1) an open-source CDS Hooks server for computing patient eligibility for PHM cohorts, (2) an open-source Population Coordinator that processes GARDE requests and communicates results to a PHM system, (3) an EHR Patient Data Repository, and (4) EHR PHM Tools to manage patients and perform outreach functions. Site-specific deployments were performed on onsite virtual machines and cloud-based Amazon Web Services. DISCUSSION: GARDE's component architecture establishes generalizable standards-based methods for computing PHM cohorts. Replicating deployments using one of the established deployment methods requires minimal local customization. Most of the deployment effort was related to obtaining site-specific information technology governance approvals.

A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study.

BACKGROUND: Exposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric. OBJECTIVE: This study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs. METHODS: We used an SDM framework and user-centered design methods to guide the design and usability of DDInteract-an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale. RESULTS: The designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5. CONCLUSIONS: This study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center.

OBJECTIVE: To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations. MATERIALS AND METHODS: We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations. RESULTS: University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape. DISCUSSION: Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation. CONCLUSION: EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Troponin Testing and Coronary Syndrome in Geriatric Patients With Nonspecific Complaints: Are We Overtesting?

BACKGROUND: Elderly patients presenting to the emergency department (ED) with nonspecific complaints (NSCs) often undergo troponin testing to assess for atypical acute coronary syndrome (ACS). However, the rate of ACS and utility of troponin testing in this population is unknown. We sought to determine the rate of ACS and diagnostic yield of troponin testing in elderly patients with NSCs. METHODS: We retrospectively identified all patients aged ≥ 65 years triaged in the ED with NSCs from January 1, 2017, to June 30, 2017. NSCs were defined a priori and included complaints such as weakness, dizziness, or fatigue. NSCs were verified in ED provider notes by trained abstractors blind to testing results. Exclusions were focal chief complaint in provider notes, fever, and no troponin ordered. ACS was strictly defined and independently adjudicated by two trained physician researchers blind to the study hypothesis. We calculated the proportion of patients with ACS within 30 days and the test characteristics of troponin to diagnose ACS. RESULTS: Screening identified 1,146 encounters, and 552 were excluded for fever or focal chief complaints in the provider notes. Of the remaining 594 patients, troponin was ordered in 412 (69%), comprising the study cohort. The mean (±SD) age was 78.7 (±8.3) years, with 58% female and 75% admitted. Troponin elevation occurred in 81 patients (20%). ACS occurred in 5 of 412 (1.2%). Troponin was 100% sensitive (95% confidence interval [CI] = 48% to 100%) and 81% specific (95% CI = 77% to 85%) for ACS. Of patients with elevated troponin, 93.8% were false positives (no ACS). All patients with troponin elevation were admitted, but only one underwent angiography and no patients received reperfusion therapy. CONCLUSIONS: While consideration for ACS is prudent in selected elderly patients with NSCs, ACS was rare and no patients received reperfusion therapy. Given the false-positive rate in our study, our results may not support routine troponin testing for ACS in this population.

Sonographic features of peritoneal lymphomatosis in 4 cats.

BACKGROUND: The metastatic spread of feline lymphoma to the peritoneum ("lymphomatosis") has been rarely reported in the literature. The sonographic features specific to this rare disease manifestation have not been described and have important treatment and prognostic considerations prompting definitive diagnosis. OBJECTIVES: To describe the ultrasonic features of feline peritoneal lymphomatosis. ANIMALS: Four cats with alimentary lymphoma and peritoneal metastasis confirmed using cytology, histology, or both. RESULTS: The sonographic features described include either a nonobstructive, focally diffuse, and circumferential intestinal mass, or an eccentric, focally diffuse, gastric mass. The intestinal and gastric lesions exhibited hypo-to-anechoic transmural wall thickening with loss of wall layering in association with discrete-to-coalescing plaques or sheets of thickened, hypoechoic tissue throughout the mesentery or omenta. All cases exhibited only small volumes of anechoic free peritoneal fluid. Three of the 4 cats also had multiple small hypoechoic nodular foci on the parietal and/or visceral peritoneal surfaces. Two cats had bilateral renomegaly because of lymphoma invasion (2/4) and 1 cat had local lymphadenopathy secondary to lymphoma invasion (1/4). CONCLUSIONS AND CLINICAL IMPORTANCE: Peritoneal lymphomatosis is a rare manifestation of lymphoma metastasis and to date appears to be associated specifically with B-cell alimentary lymphoma.

Evidence of polyclonality in neurofibromatosis type 2-associated multilobulated vestibular schwannomas.

BACKGROUND: Neurofibromatosis type 2 (NF2) is a tumor syndrome that results from mutation of the NF2 tumor suppressor gene. The hallmark of NF2 is the presence of bilateral vestibular schwannoma (VS). Though NF2-associated and sporadic VS share identical histopathologic findings and cytogenetic alterations, NF2-associated VS often appears multilobulated, is less responsive to radiosurgery, and has worse surgical outcomes. Temporal bone autopsy specimens and MRI of the inner ear performed on NF2 patients suggest that multiple discrete tumors may be present within the labyrinth and cerebellopontine angle. METHODS: Treatment-naïve ears in patients enrolled in a prospective NF2 natural history study (NIH#08-N-0044) were included for MRI analysis. T2-weighted and postcontrast T1-weighted MRIs were evaluated for the presence of multiple discrete tumors or a multilobulated mass. Peripheral blood (germline) and regional samples of tumor tissue were procured from consecutive patients enrolled in this study undergoing resection of a multilobulated VS (MVS). Histopathologic evaluation and genetic analysis (single nucleotide polymorphism array analysis, NF2 sequencing) were performed on each specimen. RESULTS: Over half of NF2 ears harbored either an MVS (60/139 ears) or multiple discrete masses (19/139 ears). For 4 successive MVSs, genetic analysis revealed an admixture of cell populations, each with its own somatic NF2 mutation or deletion. CONCLUSIONS: These findings suggest that the majority of NF2-associated VSs are polyclonal, such that the tumor mass represents a collision of multiple, distinct tumor clones. This explains the characteristic lobulated gross appearance of NF2-associated VS, and may also explain the substantially different treatment outcomes compared with sporadic VS.

Long-term natural history of neurofibromatosis Type 2-associated intracranial tumors.

OBJECT: Neurofibromatosis Type 2 (NF2) is a heritable tumor predisposition syndrome that leads to the development of multiple intracranial tumors, including meningiomas and schwannomas. Because the natural history of these tumors has not been determined, their optimal management has not been established. To define the natural history of NF2-associated intracranial tumors and to optimize management strategies, the authors evaluated long-term clinical and radiographic data in patients with NF2. METHODS: Consecutive NF2 patients with a minimum of 4 years of serial clinical and MRI follow-up were analyzed. RESULTS: Seventeen patients, 9 males and 8 females, were included in this analysis (mean follow-up 9.5±4.8 years, range 4.0-20.7 years). The mean age at initial evaluation was 33.2±15.5 years (range 12.3-57.6 years). Patients harbored 182 intracranial neoplasms, 164 of which were assessable for growth rate analysis (18 vestibular schwannomas [VSs], 11 nonvestibular cranial nerve [CN] schwannomas, and 135 meningiomas) and 152 of which were assessable for growth pattern analysis (15 VSs, 9 nonvestibular CN schwannomas, and 128 meningiomas). New tumors developed in patients over the course of the imaging follow-up: 66 meningiomas, 2 VSs, and 2 nonvestibular CN schwannomas. Overall, 45 tumors (29.6%) exhibited linear growth, 17 tumors (11.2%) exhibited exponential growth, and 90 tumors (59.2%) displayed a saltatory growth pattern characterized by alternating periods of growth and quiescence (mean quiescent period 2.3±2.1 years, range 0.4-11.7 years). Further, the saltatory pattern was the most frequently identified growth pattern for each tumor type: meningiomas 60.9%, VSs 46.7%, and nonvestibular schwannoma 55.6%. A younger age at the onset of NF2-related symptoms (p=0.01) and female sex (p=0.05) were associated with an increased growth rate in meningiomas. The identification of saltatory growth in meningiomas increased with the duration of follow-up (p=0.01). CONCLUSIONS: Neurofibromatosis Type 2-associated intracranial tumors most frequently demonstrated a saltatory growth pattern. Because new tumors can develop in NF2 patients over their lifetime and because radiographic progression and symptom formation are unpredictable, resection may be best reserved for symptom-producing tumors. Moreover, establishing the efficacy of nonsurgical therapeutic interventions must be based on long-term follow-up (several years).

Mechanisms of hearing loss in neurofibromatosis type 2.

INTRODUCTION: Patients with neurofibromatosis type 2 (NF2) develop bilateral cochleovestibular schwannomas (CVSs) that cause binaural deafness in most individuals. Hearing loss occurs in an unpredictable manner and the underlying mechanisms are not known. To gain insight into the pathophysiologic basis for hearing loss in NF2, we performed a prospective cross-sectional study of untreated ears in NF2 patients. METHODS: One hundred consecutive NF2 patients in a prospective natural history study were included. Clinical and audiometric data were analyzed for treatment naïve ears. In addition to standard MR-imaging sequences, alterations in intralabyrinthine protein content were determined utilizing high resolution FLAIR, the presence of cochlear aperture obstruction was determined by examining 3D T2 sequences, and endolymphatic hydrops was identified on delayed post-contrast FLAIR sequences. RESULTS: Eighty-nine ears harboring 84 untreated CVSs in 56 consecutive NF2 patients (age 30 ± 16 years) were analyzed. Thirty-four (38%) ears had varying degrees of hearing loss. Elevated intralabyrinthine protein was identified in 70 (75%) ears by FLAIR MR-imaging and was strongly associated with the presence of hearing loss (32/34 hearing loss ears; 94%)(Fisher's exact test; P= .005). Elevated intralabyrinthine protein was associated with the presence of CVS-associated cochlear aperture obstruction (64 of 67 ears with elevated protein; 96%)(Fisher's exact test; P<0.0001) in both normal and hearing loss ears. Elevated intralabyrinthine protein was not identified in ears without CVS (5 ears). While larger tumor size was associated with hearing loss (P=0.006), 16 hearing loss ears (47%) harbored CVSs less than 0.5 cm(3), including 14 ears (88%) with block of the cochlear aperture and elevated protein. DISCUSSION: These findings are consistent with a model in which hearing loss develops as a result of cochlear aperture obstruction and accumulation of intralabyrinthine protein. MRI based identification of elevated intralabyrinthine protein may help identify the ear at-risk for developing hearing loss.