Economic evidence of clinical decision support systems in mental health: A systematic literature review.

Mental health conditions are among the highest disease burden on society, affecting approximately 20% of children and adolescents at any point in time, with depression and anxiety being the leading causes of disability globally. To improve treatment outcomes, healthcare organizations turned to clinical decision support systems (CDSSs) that offer patient-specific diagnoses and recommendations. However, the economic impact of CDSS is limited, especially in child and adolescent mental health. This systematic literature review examined the economic impacts of CDSS implemented in mental health services. We planned to follow PRISMA reporting guidelines and found only one paper to describe health and economic outcomes. A randomized, controlled trial of 336 participants found that 60% of the intervention group and 32% of the control group achieved symptom reduction, i.e. a 50% decrease as per the Symptom Checklist-90-Revised (SCL-90-R), a method to evaluate psychological problems and identify symptoms. Analysis of the incremental cost-effectiveness ratio found that for every 1% of patients with a successful treatment result, it added €57 per year. There are not enough studies to draw conclusions about the cost-effectiveness in a mental health context. More studies on economic evaluations of the viability of CDSS within mental healthcare have the potential to contribute to patients and the larger society.

A review of information sources and analysis methods for data driven decision aids in child and adolescent mental health services.

OBJECTIVE: Clinical data analysis relies on effective methods and appropriate data. Recognizing distinctive clinical services and service functions may lead to improved decision-making. Our first objective is to categorize analytical methods, data sources, and algorithms used in current research on information analysis and decision support in child and adolescent mental health services (CAMHS). Our secondary objective is to identify the potential for data analysis in different clinical services and functions in which data-driven decision aids can be useful. MATERIALS AND METHODS: We searched related studies in Science Direct and PubMed from 2018 to 2023(Jun), and also in ACM (Association for Computing Machinery) Digital Library, DBLP (Database systems and Logic Programming), and Google Scholar from 2018 to 2021. We have reviewed 39 studies and extracted types of analytical methods, information content, and information sources for decision-making. RESULTS: In order to compare studies, we developed a framework for characterizing health services, functions, and data features. Most data sets in reviewed studies were small, with a median of 1,176 patients and 46,503 record entries. Structured data was used for all studies except two that used textual clinical notes. Most studies used supervised classification and regression. Service and situation-specific data analysis dominated among the studies, only two studies used temporal, or process features from the patient data. This paper presents and summarizes the utility, but not quality, of the studies according to the care situations and care providers to identify service functions where data-driven decision aids may be relevant. CONCLUSIONS: Frameworks identifying services, functions, and care processes are necessary for characterizing and comparing electronic health record (EHR) data analysis studies. The majority of studies use features related to diagnosis and assessment and correspondingly have utility for intervention planning and follow-up. Profiling the disease severity of referred patients is also an important application area.

Predicting In-Hospital Death from Derived EHR Trajectory Features.

Medical histories of patients can predict a patient's immediate future. While most studies propose to predict survival from vital signs and hospital tests within one episode of care, we carried out selective feature engineering from longitudinal medical records in this study to develop a dataset with derived features. We thereafter trained multiple machine learning models for the binary prediction of whether an episode of care will culminate in death among patients suspected of bloodstream infections. The machine learning classifier performance is evaluated and compared and the feature importance impacting the model output is explored. The extreme gradient boosting model achieved the best performance for predicting death in the next hospital episode with an accuracy of 92%. Age at the time of the first visit, length of history, and information related to recent episodes were the most critical features.

Challenges in Interpreting Norwegian Child and Adolescent Mental Health Records.

The Electronic Health Record system BUPdata served Norwegian Child and Adolescent Mental Health Services (CAMHS) for over 35 years and is still an important source of information for understanding clinical practice. Secondary usage of clinical data enables learning and service quality improvement. We present some insights from explorative data analysis for interpreting the records of patients referred for hyperkinetic disorders. The major challenges were data preparation, pre-analysis, imputation, and validation. We summarize the main characteristics, spot anomalies, and detect errors. The results include observations about the patient referral diversity based on 12 different variables. We modeled the activities in an individual episode of care, described our clinical observations among data, and discussed the challenges of data analysis.

Improving the Quality and Utility of Electronic Health Record Data through Ontologies.

The translational research community, in general, and the Clinical and Translational Science Awards (CTSA) community, in particular, share the vision of repurposing EHRs for research that will improve the quality of clinical practice. Many members of these communities are also aware that electronic health records (EHRs) suffer limitations of data becoming poorly structured, biased, and unusable out of original context. This creates obstacles to the continuity of care, utility, quality improvement, and translational research. Analogous limitations to sharing objective data in other areas of the natural sciences have been successfully overcome by developing and using common ontologies. This White Paper presents the authors' rationale for the use of ontologies with computable semantics for the improvement of clinical data quality and EHR usability formulated for researchers with a stake in clinical and translational science and who are advocates for the use of information technology in medicine but at the same time are concerned by current major shortfalls. This White Paper outlines pitfalls, opportunities, and solutions and recommends increased investment in research and development of ontologies with computable semantics for a new generation of EHRs.

Usability of the IDDEAS prototype in child and adolescent mental health services: A qualitative study for clinical decision support system development.

Introduction: Child and adolescent mental health services (CAMHS) clinical decision support system (CDSS) provides clinicians with real-time support as they assess and treat patients. CDSS can integrate diverse clinical data for identifying child and adolescent mental health needs earlier and more comprehensively. Individualized Digital Decision Assist System (IDDEAS) has the potential to improve quality of care with enhanced efficiency and effectiveness. Methods: We examined IDDEAS usability and functionality in a prototype for attention deficit hyperactivity disorder (ADHD), using a user-centered design process and qualitative methods with child and adolescent psychiatrists and clinical psychologists. Participants were recruited from Norwegian CAMHS and were randomly assigned patient case vignettes for clinical evaluation, with and without IDDEAS. Semi-structured interviews were conducted as one part of testing the usability of the prototype following a five-question interview guide. All interviews were recorded, transcribed, and analyzed following qualitative content analysis. Results: Participants were the first 20 individuals from the larger IDDEAS prototype usability study. Seven participants explicitly stated a need for integration with the patient electronic health record system. Three participants commended the step-by-step guidance as potentially helpful for novice clinicians. One participant did not like the aesthetics of the IDDEAS at this stage. All participants were pleased about the display of the patient information along with guidelines and suggested that wider guideline coverage will make IDDEAS much more useful. Overall, participants emphasized the importance of maintaining the clinician as the decision-maker in the clinical process, and the overall potential utility of IDDEAS within Norwegian CAMHS. Conclusion: Child and adolescent mental health services psychiatrists and psychologists expressed strong support for the IDDEAS clinical decision support system if better integrated in daily workflow. Further usability assessments and identification of additional IDDEAS requirements are necessary. A fully functioning, integrated version of IDDEAS has the potential to be an important support for clinicians in the early identification of risks for youth mental disorders and contribute to improved assessment and treatment of children and adolescents.

Success Factors of an Early EHR System for Child and Adolescent Mental Health: Lessons Learned for Future Practice Data-Driven Decision Aids.

This paper recounts the successful BUPdata, a discontinued electronic health record (EHR) system for Child and Adolescent Mental Health Services (CAMHS) in Norway. It was developed and owned by the national association for CAMHS and fulfilled needs for collaborative care, practice insight, and service management. It aimed to unify the requirements of government, administration, clinicians, patients, and researchers alike, with the goal of providing uniform quality of care nationally. When CAMHS became integrated with specialist healthcare, BUPdata was replaced with more a general EHR system offering far less functionality and insight into CAMHS practice. We have studied BUPdata, and interviewed stakeholders in order to develop decision aids based on practice data analysis and give clinicians and patients insight into successful local practice, collaboration patterns, and overview of local resources.

Attitudes of Mental Health Service Users Toward Storage and Use of Electronic Health Records.

OBJECTIVE: Electronic health records (EHRs) are used for both clinical practice and research. Because mental health service users' views are underrepresented in perspectives on EHR use, the authors examined service users' awareness, attitudes, and opinions about EHR data storage and sharing. METHODS: A mixed-methods, cross-sectional design was used to examine attitudes of 253 Norwegian mental health service users who were recruited online to complete a quantitative and qualitative (free-text) survey about EHR utilization. RESULTS: Most participants were aware that EHRs were stored (95%) and shared (58%). Most thought that patients benefited from EHR storage (84%), trusted authorities with EHR sharing (71%), were willing to share their EHRs to help others (75%), felt they benefited from EHR sharing (75%), and thought EHR sharing was ethical for health care and research (71%). Fewer were aware of EHR sharing for research (36%), and 62% were aware that shared data were anonymized. Of the participants, 69% recognized privacy risks associated with sharing. Lack of transparency and skepticism about anonymization and misuse of EHR data were concerns and perceived risks. Mental health service users thought that EHRs should be shared for policy development (81%), education and training (85%), improving care quality (89%), research (91%), and clinical decision support (81%). CONCLUSIONS: Participants were aware of and supported EHR sharing for research and clinical care. They supported sharing to help others and were willing to fully participate in clinical care and research, as well as to share EHR information for their own care, research, and the care of others.

Sepsis prediction, early detection, and identification using clinical text for machine learning: a systematic review.

OBJECTIVE: To determine the effects of using unstructured clinical text in machine learning (ML) for prediction, early detection, and identification of sepsis. MATERIALS AND METHODS: PubMed, Scopus, ACM DL, dblp, and IEEE Xplore databases were searched. Articles utilizing clinical text for ML or natural language processing (NLP) to detect, identify, recognize, diagnose, or predict the onset, development, progress, or prognosis of systemic inflammatory response syndrome, sepsis, severe sepsis, or septic shock were included. Sepsis definition, dataset, types of data, ML models, NLP techniques, and evaluation metrics were extracted. RESULTS: The clinical text used in models include narrative notes written by nurses, physicians, and specialists in varying situations. This is often combined with common structured data such as demographics, vital signs, laboratory data, and medications. Area under the receiver operating characteristic curve (AUC) comparison of ML methods showed that utilizing both text and structured data predicts sepsis earlier and more accurately than structured data alone. No meta-analysis was performed because of incomparable measurements among the 9 included studies. DISCUSSION: Studies focused on sepsis identification or early detection before onset; no studies used patient histories beyond the current episode of care to predict sepsis. Sepsis definition affects reporting methods, outcomes, and results. Many methods rely on continuous vital sign measurements in intensive care, making them not easily transferable to general ward units. CONCLUSIONS: Approaches were heterogeneous, but studies showed that utilizing both unstructured text and structured data in ML can improve identification and early detection of sepsis.

Using neural networks to support high-quality evidence mapping.

BACKGROUND: The Living Evidence Map Project at the Norwegian Institute of Public Health (NIPH) gives an updated overview of research results and publications. As part of NIPH's mandate to inform evidence-based infection prevention, control and treatment, a large group of experts are continously monitoring, assessing, coding and summarising new COVID-19 publications. Screening tools, coding practice and workflow are incrementally improved, but remain largely manual. RESULTS: This paper describes how deep learning methods have been employed to learn classification and coding from the steadily growing NIPH COVID-19 dashboard data, so as to aid manual classification, screening and preprocessing of the rapidly growing influx of new papers on the subject. Our main objective is to make manual screening scalable through semi-automation, while ensuring high-quality Evidence Map content. CONCLUSIONS: We report early results on classifying publication topic and type from titles and abstracts, showing that even simple neural network architectures and text representations can yield acceptable performance.

Synthetic data for annotation and extraction of family history information from clinical text.

BACKGROUND: The limited availability of clinical texts for Natural Language Processing purposes is hindering the progress of the field. This article investigates the use of synthetic data for the annotation and automated extraction of family history information from Norwegian clinical text. We make use of incrementally developed synthetic clinical text describing patients' family history relating to cases of cardiac disease and present a general methodology which integrates the synthetically produced clinical statements and annotation guideline development. The resulting synthetic corpus contains 477 sentences and 6030 tokens. In this work we experimentally assess the validity and applicability of the annotated synthetic corpus using machine learning techniques and furthermore evaluate the system trained on synthetic text on a corpus of real clinical text, consisting of de-identified records for patients with genetic heart disease. RESULTS: For entity recognition, an SVM trained on synthetic data had class weighted precision, recall and F1-scores of 0.83, 0.81 and 0.82, respectively. For relation extraction precision, recall and F1-scores were 0.74, 0.75 and 0.74. CONCLUSIONS: A system for extraction of family history information developed on synthetic data generalizes well to real, clinical notes with a small loss of accuracy. The methodology outlined in this paper may be useful in other situations where limited availability of clinical text hinders NLP tasks. Both the annotation guidelines and the annotated synthetic corpus are made freely available and as such constitutes the first publicly available resource of Norwegian clinical text.

Neural classification of Norwegian radiology reports: using NLP to detect findings in CT-scans of children.

BACKGROUND: With a motivation of quality assurance, machine learning techniques were trained to classify Norwegian radiology reports of paediatric CT examinations according to their description of abnormal findings. METHODS: 13.506 reports from CT-scans of children, 1000 reports from CT scan of adults and 1000 reports from X-ray examination of adults were classified as positive or negative by a radiologist, according to the presence of abnormal findings. Inter-rater reliability was evaluated by comparison with a clinician's classifications of 500 reports. Test-retest reliability of the radiologist was performed on the same 500 reports. A convolutional neural network model (CNN), a bidirectional recurrent neural network model (bi-LSTM) and a support vector machine model (SVM) were trained on a random selection of the children's data set. Models were evaluated on the remaining CT-children reports and the adult data sets. RESULTS: Test-retest reliability: Cohen's Kappa = 0.86 and F1 = 0.919. Inter-rater reliability: Kappa = 0.80 and F1 = 0.885. Model performances on the Children-CT data were as follows. CNN: (AUC = 0.981, F1 = 0.930), bi-LSTM: (AUC = 0.978, F1 = 0.927), SVM: (AUC = 0.975, F1 = 0.912). On the adult data sets, the models had AUC around 0.95 and F1 around 0.91. CONCLUSIONS: The models performed close to perfectly on its defined domain, and also performed convincingly on reports pertaining to a different patient group and a different modality. The models were deemed suitable for classifying radiology reports for future quality assurance purposes, where the fraction of the examinations with abnormal findings for different sub-groups of patients is a parameter of interest.

Testing an individualized digital decision assist system for the diagnosis and management of mental and behavior disorders in children and adolescents.

BACKGROUND: Nearly half of all mental health disorders develop prior to the age of 15. Early assessments, diagnosis, and treatment are critical to shortening single episodes of care, reducing possible comorbidity and long-term disability. In Norway, approximately 20% of all children and adolescents are experiencing mental health problems. To address this, health officials in Norway have called for the integration of innovative approaches. A clinical decision support system (CDSS) is an innovative, computer-based program that provides health professionals with clinical decision support as they care for patients. CDSS use standardized clinical guidelines and big data to provide guidance and recommendations to clinicians in real-time. IDDEAS (Individualised Digital DEcision Assist System) is a CDSS for diagnosis and treatment of child and adolescent mental health disorders. The aim of IDDEAS is to enhance quality, competency, and efficiency in child and adolescent mental health services (CAMHS). METHODS/DESIGN: IDDEAS is a mixed-methods innovation and research project, which consists of four stages: 1) Assessment of Needs and Preparation of IDDEAS; 2) The Development of IDDEAS CDSS Model; 3) The Evaluation of the IDDEAS CDSS; and, 4) Implementation & Dissemination. Both qualitative and quantitative methods will be used for the evaluation of IDDEAS CDSS model. Child and adolescent psychologists and psychiatrists (n = 30) will evaluate the IDDEAS` usability, acceptability and relevance for diagnosis and treatment of attention-deficit/hyperactivity disorder. DISCUSSION: The IDDEAS CDSS model is the first guidelines and data-driven CDSS to improve efficiency of diagnosis and treatment of child and adolescent mental health disorders in Norway. Ultimately, IDDEAS will help to improve patient health outcomes and prevent long-term adverse outcomes by providing each patient with evidence-based, customized clinical care. TRIAL REGISTRATION: ISRCTN, ISRCTN12094788. Ongoing study, registered prospectively 8 April 2020 https://doi.org/10.1186/ISRCTN12094788.

A Minimal Information Model for Potential Drug-Drug Interactions.

Despite the significant health impacts of adverse events associated with drug-drug interactions, no standard models exist for managing and sharing evidence describing potential interactions between medications. Minimal information models have been used in other communities to establish community consensus around simple models capable of communicating useful information. This paper reports on a new minimal information model for describing potential drug-drug interactions. A task force of the Semantic Web in Health Care and Life Sciences Community Group of the World-Wide Web consortium engaged informaticians and drug-drug interaction experts in in-depth examination of recent literature and specific potential interactions. A consensus set of information items was identified, along with example descriptions of selected potential drug-drug interactions (PDDIs). User profiles and use cases were developed to demonstrate the applicability of the model. Ten core information items were identified: drugs involved, clinical consequences, seriousness, operational classification statement, recommended action, mechanism of interaction, contextual information/modifying factors, evidence about a suspected drug-drug interaction, frequency of exposure, and frequency of harm to exposed persons. Eight best practice recommendations suggest how PDDI knowledge artifact creators can best use the 10 information items when synthesizing drug interaction evidence into artifacts intended to aid clinicians. This model has been included in a proposed implementation guide developed by the HL7 Clinical Decision Support Workgroup and in PDDIs published in the CDS Connect repository. The complete description of the model can be found at https://w3id.org/hclscg/pddi.

Local, Early, and Precise: Designing a Clinical Decision Support System for Child and Adolescent Mental Health Services.

Mental health disorders often develop during childhood and adolescence, causing long term and debilitating impacts at individual and societal levels. Local, early, and precise assessment and evidence-based treatment are key to achieve positive mental health outcomes and to avoid long-term care. Technological advancements, such as computerized Clinical Decision Support Systems (CDSSs), can support practitioners in providing evidence-based care. While previous studies have found CDSS implementation helps to improve aspects of medical care, evidence is limited on its use for child and adolescent mental health care. This paper presents challenges and opportunities for adapting CDSS design and implementation to child and adolescent mental health services (CAMHS). To highlight the complexity of incorporating CDSSs within local CAMHS, we have structured the paper around four components to consider before designing and implementing the CDSS: supporting collaboration among multiple stakeholders involved in care; optimally using health data; accounting for comorbidities; and addressing the temporality of patient care. The proposed perspective is presented within the context of the child and adolescent mental health services in Norway and an ongoing Norwegian innovative research project, the Individualized Digital DEcision Assist System (IDDEAS), for child and adolescent mental health disorders. Attention deficit hyperactivity disorder (ADHD) among children and adolescents serves as the case example. The integration of IDDEAS in Norway intends to yield significantly improved outcomes for children and adolescents with enduring mental health disorders, and ultimately serve as an educational opportunity for future international approaches to such CDSS design and implementation.

Reviewing clinical guideline development tools: features and characteristics.

BACKGROUND: To improve consistency and streamline development and publication of clinical guidelines (GL), there is a need for appropriate software support. We have found few specific tools for the actual authoring and maintaining of GLs, and correspondingly few analyses or reviews of GL development tool functionality. In order to assist GL developers in selecting and evaluating tools, this study tries to address the perceived gap by pursuing four goals: 1) identifying available tools, 2) reviewing a representative group of tools and their supported functionalities, 3) uncovering themes of features that the studied tools support, and 4) compare the selected tools with respect to the themes. METHODS: We conducted a literature search using PubMed and Google Scholar in order to find GL development tools (GDT). We also explored tools and Content Management Systems (CMS) used in representative organisations and international communities that develop and maintain GLs. By reading a selected representative group of five GL tool manuals, exploring tools hands-on, we uncovered 8 themes of features. All found tools were compared according to these themes in order to identify the level of functionality they offer to support the GL development and publishing process. In order to limit the scope, tools for designing computer-interpretable/executable GL are excluded. RESULTS: After finding 1552 published papers, contacting 7 organizations and international communities, we identified a total of 19 unique tools, of which 5 tools were selected as representative in this paper. We uncovered a total of 8 themes of features according to the identified functionalities that each tool provides. Four features were common among tools: Collaborative authoring process support, user access control, GL repository management, electronic publishing. We found that the GRADE methodology was supported by three of the reviewed tools, while only two tools support annotating GL with MeSH terms. We also identified that monitoring progress, reference management, Managing versions (version control), and Change control (tracking) were often the missing features. CONCLUSION: The results can promote sector discussion and eventual agreement on important tool functionality. It may aid tool and GL developers towards more efficient, and effective, GL authoring.

A Systematic Literature Review on Evaluation of Digital Tools for Authoring Evidence-Based Clinical Guidelines.

To facilitate the clinical guideline (GL) development process, different groups of researchers have proposed tools that enable computer-supported tools for authoring and publishing GLs. In a previous study we interviewed GL authors in different Norwegian institutions and identified tool shortcomings. In this follow-up study our goal is to explore to what extent GL authoring tools have been evaluated by researchers, guideline organisations, or GL authors. This article presents results from a systematic literature review of evaluation (including usability) of GL authoring tools. A controlled database search and backward snow-balling were used to identify relevant articles. From the 12692 abstracts found, 188 papers were fully reviewed and 26 papers were identified as relevant. The GRADEPro tool has attracted some evaluation, however popular tools and platforms such as DECIDE, Doctor Evidence, JBI-SUMARI, G-I-N library have not been subject to specific evaluation from an authoring perspective. Therefore, we found that little attention was paid to the evaluation of the tools in general. We could not find any evaluation relevant to how tools integrate and support the complex GL development workflow. The results of this paper are highly relevant to GL authors, tool developers and GL publishing organisations in order to improve and control the GL development and maintenance process.

Clinical decision support systems in child and adolescent psychiatry: a systematic review.

Psychiatric disorders are amongst the most prevalent and impairing conditions in childhood and adolescence. Unfortunately, it is well known that general practitioners (GPs) and other frontline health providers (i.e., child protection workers, public health nurses, and pediatricians) are not adequately trained to address these ubiquitous problems (Braddick et al. Child and Adolescent mental health in Europe: infrastructures, policy and programmes, European Communities, 2009; Levav et al. Eur Child Adolesc Psychiatry 13:395-401, 2004). Advances in technology may offer a solution to this problem with clinical decision support systems (CDSS) that are designed to help professionals make sound clinical decisions in real time. This paper offers a systematic review of currently available CDSS for child and adolescent mental health disorders prepared according to the PRISMA-Protocols (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols). Applying strict eligibility criteria, the identified studies (n = 5048) were screened. Ten studies, describing eight original clinical decision support systems for child and adolescent psychiatric disorders, fulfilled inclusion criteria. Based on this systematic review, there appears to be a need for a new, readily available CDSS for child neuropsychiatric disorder which promotes evidence-based, best practices, while enabling consideration of national variation in practices by leveraging data-reuse to generate predictions regarding treatment outcome, addressing a broader cluster of clinical disorders, and targeting frontline practice environments.

EHR Improvement Using Incident Reports.

This paper discusses reactive improvement of clinical software using methods for incident analysis. We used the "Five Whys" method because we had only descriptive data and depended on a domain expert for the analysis. The analysis showed that there are two major root causes for EHR software failure, and that they are related to human and organizational errors. A main identified improvement is allocating more resources to system maintenance and user training.