Diagnostic test accuracy in longitudinal study settings: theoretical approaches with use cases from clinical practice.

OBJECTIVES: In this study, we evaluate how to estimate diagnostic test accuracy (DTA) correctly in the presence of longitudinal patient data (ie, repeated test applications per patient). STUDY DESIGN AND SETTING: We used a nonparametric approach to estimate the sensitivity and specificity of three tests for different target conditions with varying characteristics (ie, episode length and disease-free intervals between episodes): 1) systemic inflammatory response syndrome (n = 36), 2) depression (n = 33), and 3) epilepsy (n = 30). DTA was estimated on the levels 'time', 'block', and 'patient-time' for each diagnosis, representing different research questions. The estimation was conducted for the time units per minute, per hour, and per day. RESULTS: A comparison of DTA per and across use cases showed variations in the estimates, which resulted from the used level, the time unit, the resulting number of observations per patient, and the diagnosis-specific characteristics. Intra- and inter-use-case comparisons showed that the time-level had the highest DTA, particularly the larger the time unit, and that the patient-time-level approximated 50% sensitivity and specificity. CONCLUSION: Researchers need to predefine their choices (ie, estimation levels and time units) based on their individual research aims, estimands, and diagnosis-specific characteristics of the target outcomes to make sure that unbiased and clinically relevant measures are communicated. In cases of uncertainty, researchers could report the DTA of the test using more than one estimation level and/or time unit.

Effect of SARS-CoV-2 Incidence and Immunisation Rates on Sporadic Creutzfeldt-Jakob Disease Incidence.

BACKGROUND: Recent case studies and media outlets have hypothesised an effect of SARS-CoV-2 infection and immunisation on the development or progression of neurodegenerative diseases such as Alzheimer's disease or sporadic Creutzfeldt-Jakob disease (sCJD). OBJECTIVES: This study aims to identify potential associations of SARS-CoV-2 infections and SARS-CoV-2 immunisation with sCJD incidence, disease duration, and age of onset. METHOD: We used data from a prospective sCJD surveillance study in Germany (2016-2022) and publicly available datasets of SARS-CoV-2 cases and vaccination numbers in Germany for the years 2020-2022. Associations of SARS-CoV-2 incidence and immunisation rates with sCJD incidence were assessed by comparing quarterly and annual cumulative sCJD incidences in the periods before (2016-2019) and during the pandemic (2020-2022). RESULTS: We could not identify any time-related effect of SARS-CoV-2 incidence or immunisation rate on the sCJD incidence. Moreover, we did not find any sCJD incidence alterations before and during the SARS-CoV-2 pandemic on a federal or state level. The overall sCJD incidence was within expected ranges in the years 2020-2022. There were no changes in age of onset and clinical disease duration in these years. CONCLUSIONS: We found no evidence supporting a short-term effect of the pandemic on sCJD incidence. However, considering the extended pre-clinical phase of sCJD, continued surveillance is needed to identify potential future incidence alterations.

Tracing the Progression of Sepsis in Critically Ill Children: Clinical Decision Support for Detection of Hematologic Dysfunction.

BACKGROUND: One of the major challenges in pediatric intensive care is the detection of life-threatening health conditions under acute time constraints and performance pressure. This includes the assessment of pediatric organ dysfunction (OD) that demands extraordinary clinical expertise and the clinician's ability to derive a decision based on multiple information and data sources. Clinical decision support systems (CDSS) offer a solution to support medical staff in stressful routine work. Simultaneously, detection of OD by using computerized decision support approaches has been scarcely investigated, especially not in pediatrics. OBJECTIVES: The aim of the study is to enhance an existing, interoperable, and rule-based CDSS prototype for tracing the progression of sepsis in critically ill children by augmenting it with the capability to detect SIRS/sepsis-associated hematologic OD, and to determine its diagnostic accuracy. METHODS: We reproduced an interoperable CDSS approach previously introduced by our working group: (1) a knowledge model was designed by following the commonKADS methodology, (2) routine care data was semantically standardized and harmonized using openEHR as clinical information standard, (3) rules were formulated and implemented in a business rule management system. Data from a prospective diagnostic study, including 168 patients, was used to estimate the diagnostic accuracy of the rule-based CDSS using the clinicians' diagnoses as reference. RESULTS: We successfully enhanced an existing interoperable CDSS concept with the new task of detecting SIRS/sepsis-associated hematologic OD. We modeled openEHR templates, integrated and standardized routine data, developed a rule-based, interoperable model, and demonstrated its accuracy. The CDSS detected hematologic OD with a sensitivity of 0.821 (95% CI: 0.708-0.904) and a specificity of 0.970 (95% CI: 0.942-0.987). CONCLUSION: We could confirm our approach for designing an interoperable CDSS as reproducible and transferable to other critical diseases. Our findings are of direct practical relevance, as they present one of the first interoperable CDSS modules that detect pediatric SIRS/sepsis-associated hematologic OD.

Systematic review identifies deficiencies in reporting of diagnostic test accuracy among clinical decision support systems.

OBJECTIVES: This systematic review assesses the reporting quality and risk of bias in studies evaluating the diagnostic test accuracy (DTA) of clinical decision support systems (CDSS). STUDY DESIGN AND SETTING: The Cochrane Library, PubMed/MEDLINE, Scopus, and Web of Science were searched for studies, published between January 1, 2016 and May 31, 2021, evaluating the DTA of CDSS for human patients. Articles using a patient's self-diagnosis, assessing disease severity, focusing on treatment/follow-up, or comparing pre-post CDSS implementation periods were excluded. All eligible studies were assessed for reporting quality using STARD 2015 and for risk of bias using QUADAS-2. Item ratings were presented using heat maps. This study was reported as per PRISMA-DTA. RESULTS: In total, 158 of 2,820 screened articles were included in the analysis. The studies were heterogeneous in terms of study characteristics, reporting quality, risk of biases, and applicability concerns with few highly rated studies. Mostly the overall quality was deficient for items addressing the domains 'methodology,' 'results,' and 'other information'. CONCLUSION: Our analysis revealed shortcomings in critical domains of reporting quality and risk of bias, indicating the need for additional guidance and training in an interdisciplinary scientific field with mixed biostatistical expertise.

Prediction models for SIRS, sepsis and associated organ dysfunctions in paediatric intensive care: study protocol for a diagnostic test accuracy study.

INTRODUCTION: Systemic inflammatory response syndrome (SIRS), sepsis and associated organ dysfunctions are life-threating conditions occurring at paediatric intensive care units (PICUs). Early recognition and treatment within the first hours of onset are critical. However, time pressure, lack of personnel resources, and the need for complex age-dependent diagnoses impede an accurate and timely diagnosis by PICU physicians. Data-driven prediction models integrated in clinical decision support systems (CDSS) could facilitate early recognition of disease onset. OBJECTIVES: To estimate the sensitivity and specificity of previously developed prediction models (index tests) for the detection of SIRS, sepsis and associated organ dysfunctions in critically ill children up to 12 hours before reference standard diagnosis is possible. METHODS AND ANALYSIS: We conduct a monocentre, prospective diagnostic test accuracy study. Clinicians in the PICU of the tertiary care centre Hannover Medical School, Germany, continuously screen and recruit patients until the adaptive sample size (originally intended sample size of 500 patients) is enrolled. Eligible are children (0-17 years, all sexes) who stay in the PICU for ≥12 hours and for whom an informed consent is given. All eligible patients are independently assessed for SIRS, sepsis and organ dysfunctions using corresponding predictive and knowledge-based CDSS models. The knowledge-based CDSS models serve as imperfect reference standards. The assessments are used to estimate the sensitivities and specificities of each predictive model using a clustered nonparametric approach (main analysis). Subgroup analyses ('age groups', 'sex' and 'age groups by sex') are predefined. ETHICS AND DISSEMINATION: This study obtained ethics approval from the Hannover Medical School Ethics Committee (No. 10188_BO_SK_2022). Results will be disseminated as peer-reviewed publications, at scientific conferences, and to patients in an appropriate dissemination approach. TRIAL REGISTRATION NUMBER: This study was registered with the German Clinical Trial Register (DRKS00029071) on 2022-05-23. PROTOCOL VERSION: 10188_BO_SK_2022_V.2.0-20220330_4_Studienprotokoll.