Predicting mucosal inflammation in IBD patients using patient-reported symptom scores and a faecal calprotectin home test: protocol for a multicentre prospective validation study.

INTRODUCTION: Crohn's disease and ulcerative colitis are chronic inflammatory bowel diseases (IBD) with a relapsing-remitting nature. With adequate non-invasive prediction of mucosal inflammation, endoscopies can be prevented and treatment optimised earlier for better disease control. We aim to validate and recalibrate commonly used patient-reported symptom scores combined with a faecal calprotectin (FC) home test as non-invasive diagnostic tool for remote monitoring of IBD, both in daily practice and in a strict trial setting. Endoscopy will be used as the gold standard. METHODS AND ANALYSIS: In this multicentre prospective validation study, adult IBD patients are asked to fill out questionnaires regarding disease activity (Monitor IBD At Home, mobile Health Index, Manitoba IBD Index, IBD control and patient-HBI/patient-Simple Clinical Colitis Activity Index), perform a FC home test and collect a stool sample for routine laboratory FC measurement, before the start of the bowel preparation for the ileocolonoscopy. Endoscopic disease activity will be scored according to the simplified endoscopic score for Crohn's disease (CD) for CD patients or Ulcerative Colitis Endoscopic Index for Severity and Mayo Endoscopic Subscore for ulcerative colitis patients. The main study outcome is the diagnostic test accuracy of the various patient-reported scores to assess mucosal inflammation in combination with a FC home test. ETHICS AND DISSEMINATION: This study is approved by the Medical Research Ethics Committee of azM/UM in Maastricht dated 03 March 2021 (METC 20-085) and is monitored by the Clinical Trial Centre Maastricht according to Good Clinical Practice guidelines. Written informed consent will be obtained from all patients. Study results will be published in international peer-reviewed medical journals. TRIAL REGISTRATION NUMBER: NCT05886322.

Artificial intelligence in mammography: a systematic review of the external validation.

Objective: To conduct a systematic review of external validation studies on the use of different Artificial Intelligence algorithms in breast cancer screening with mammography. Data source: Our systematic review was conducted and reported following the PRISMA statement, using the PubMed, EMBASE, and Cochrane databases with the search terms "Artificial Intelligence," "Mammography," and their respective MeSH terms. We filtered publications from the past ten years (2014 - 2024) and in English. Study selection: A total of 1,878 articles were found in the databases used in the research. After removing duplicates (373) and excluding those that did not address our PICO question (1,475), 30 studies were included in this work. Data collection: The data from the studies were collected independently by five authors, and it was subsequently synthesized based on sample data, location, year, and their main results in terms of AUC, sensitivity, and specificity. Data synthesis: It was demonstrated that the Area Under the ROC Curve (AUC) and sensitivity were similar to those of radiologists when using independent Artificial Intelligence. When used in conjunction with radiologists, statistically higher accuracy in mammogram evaluation was reported compared to the assessment by radiologists alone. Conclusion: AI algorithms have emerged as a means to complement and enhance the performance and accuracy of radiologists. They also assist less experienced professionals in detecting possible lesions. Furthermore, this tool can be used to complement and improve the analyses conducted by medical professionals.

Systematic review of subjective validation methods for computerized colonoscopy simulators.

Introduction: In recent years, different approaches have been used to conduct a subjective assessment of colonoscopy simulators. The purpose of this paper is to review these different approaches, specifically the ones used for computerized simulators, as the first step for the design of a standard validation procedure for this type of simulators. Methods: A systematic review was conducted by searching papers after 2010 in PubMed, Google Scholar, ScienceDirect, and IEEE Xplore databases. Papers were screened and reviewed for procedures regarding the subjective validation of computerized simulators for traditional colonoscopy with an endoscope. Results: An initial search in the databases identified 2094 papers, of which 7 remained after exhaustive review and application of exclusion criteria. All studies used questionnaires for subjective validation, with "face" being the most common validity type tested, while "content" validity and "usability" were less prominent. Conclusions: A classification of subscales for testing face validity was derived from the studies. The Colonoscopy Simulator Realism Questionnaire (CSRQ) was selected as the guide to follow for the development of future questionnaires related to subjective validation. Mislabeling of the validity tested in the studies due to ambiguous interpretations of the validity types was a common occurrence observed in the reviewed studies.

ICH M10 Bioanalytical Method Validation Guideline-1 year Later.

The International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) adopted Guideline M10 entitled "Bioanalytical Method Validation and Study Sample Analysis" in May 2022. In October 2023, approximately one year after the adoption of the ICH M10 guideline, a "Hot Topic" session was held during the AAPS PharmSci 360 meeting to discuss the implementation of the guideline. The session focused on items the bioanalytical community felt were challenging to implement or ambiguous within the guideline. These topics included cross-validation, parallelism, comparative bioavailability studies, combination drug stability, endogenous analyte bioanalysis, and dilution QCs. In addition, the regulatory perspective on the guideline was presented. This report provides a summary of the Hot Topic session.

Research inefficiencies in external validation studies of the Framingham Wilson coronary heart disease risk rule: A systematic review.

BACKGROUND: External validation studies create evidence about a clinical prediction rule's (CPR's) generalizability by evaluating and updating the CPR in populations different from those used in the derivation, and also by contributing to estimating its overall performance when meta-analysed in a systematic review. While most cardiovascular CPRs do not have any external validation, some CPRs have been externally validated repeatedly. Hence, we examined whether external validation studies of the Framingham Wilson coronary heart disease (CHD) risk rule contributed to generating evidence to their full potential. METHODS: A forward citation search of the Framingham Wilson CHD risk rule's derivation study was conducted to identify studies that evaluated the Framingham Wilson CHD risk rule in different populations. For external validation studies of the Framingham Wilson CHD risk rule, we examined whether authors updated the Framingham Wilson CHD risk rule when it performed poorly. We also assessed the contribution of external validation studies to understanding the Predicted/Observed (P/O) event ratio and c statistic of the Framingham Wilson CHD risk rule. RESULTS: We identified 98 studies that evaluated the Framingham Wilson CHD risk rule; 40 of which were external validation studies. Of these 40 studies, 27 (67.5%) concluded the Framingham Wilson CHD risk rule performed poorly but did not update it. Of 23 external validation studies conducted with data that could be included in meta-analyses, 13 (56.5%) could not fully contribute to the meta-analyses of P/O ratio and/or c statistic because these performance measures were neither reported nor could be calculated from provided data. DISCUSSION: Most external validation studies failed to generate evidence about the Framingham Wilson CHD risk rule's generalizability to their full potential. Researchers might increase the value of external validation studies by presenting all relevant performance measures and by updating the CPR when it performs poorly.

Development and validation of meta-measurement instruments: A methodological approach.

A valid and reliable nursing assessment is essential for identifying required care and ensuring patient safety. The convenience of conducting a comprehensive assessment of the patient has led to a significant increase in assessment tools that may slow down the process. Nevertheless, the possibility of consolidating various instruments that measure common or similar constructs into a meta-instrument is considered an alternative that could enhance assessment efficiency. A meta-instrument can be defined as a measurement tool that consolidates other instruments based on measuring related constructs and sharing dimensions or items, aiming to achieve a more parsimonious measurement. Literature on such assessment tools is scarce, and there are numerous options for their construction and initial validation. Additionally, it is advisable to confirm their psychometric properties and ensure that they maintain, at the very least, the same diagnostic capacity as the original instruments. This article presents a proposal for the phases to follow in constructing meta-instruments, along with various methodological alternatives that can be employed based on the characteristics of the original instruments and the purpose of creating the meta-instrument. Furthermore, special attention is given to the checklists that should be used to study the psychometric properties and diagnostic capacity of the meta-instruments. Finally, future lines of research and challenges in the development of nursing assessment meta-instruments are discussed.

How to verify and validate a clinical microbiology test before it can be used in routine diagnostics: a practical guide.

BACKGROUND: Before a new test can be routinely used in your laboratory, its reliability must be established in the laboratory where it will be used. International standards demand validation and verification procedures for new tests. The International Organization for Standardization (ISO) 15189 was recently updated, and the European Commission's In Vitro Diagnostic Regulation (IVDR) came into effect. These events will likely increase the need for validation and verification procedures. OBJECTIVES: This paper aims to provide practical guidance in validating or verifying microbiology tests, including antimicrobial susceptibility tests in a clinical microbiology laboratory. SOURCES: It summarizes and interprets certain parts of standards such as ISO 15189:2022, and regulations, such as IVDR 2017/746 regarding validation or verification of a new test in a routine clinical microbiology laboratory. CONTENT: The reasons for choosing a new test and the outline of the validation and verification plan are discussed. Furthermore, the following topics are touched upon: the choice of reference standard, number of samples, testing procedures, how to solve the discrepancies between results from new test and reference standard, and acceptance criteria. Arguments for selecting certain parameters (such as reference standard and sample size) and examples are given. IMPLICATIONS: With the expected increase in validation and verification procedures because of the implementation of IVDR, this paper may aid in planning and executing these procedures.

Understanding the Complexity of Microbiology Method Validation to Meet State-Specific Requirements for Cannabis.

BACKGROUND: Federal legislation still prohibits the cultivation, sale, and consumption of cultivars of delta 9-tetrahydrocannabinol cannabis (>0.3%); however, as of November 2022, 39 states have legalized cannabis and cannabis derived products for medicinal consumption and 21 states have legalized for adult-use consumption. This state-by-state approach has produced a patch work of regulations that multi-state operators (MSOs) must learn to navigate. Furthermore, cannabis laboratories often lack the space and skill needed to perform method validations, adding another layer of complexity. While these barriers exist, it is paramount for MSOs to demonstrate the fitness of purpose of their methods. OBJECTIVE: This review presents the complexity that an MSO navigated in developing microbiology method validation study designs for two proprietary real-time PCR (RT-PCR) assays to meet four state (California, Florida, Michigan, and Oklahoma) testing requirements. The testing in each state was conducted in addition to certification of the assays through the AOAC Performance Tested MethodSM (PTM) program. METHODS: Matrix studies were conducted targeting three analytes (Aspergillus spp., Salmonella spp., and Shiga toxin-producing Escherichia coli) as directed by the state regulatory authorities. For California, inclusivity and exclusivity studies were performed. The number of contamination levels, test portion replicates, and total target organisms evaluated varied by state. Culture confirmation was not performed outside of the AOAC PTM studies. RESULTS: Data from each study was collected, summarized, and provided to the state regulatory agencies. Review of the data consisted of identifying discrepant results and verification of controls. After review, each assay was certified for use in the respective state. CONCLUSION: Requirements from each of the states demonstrate the complexities MSOs face and emphasize the need for a more standardized approach to streamline acceptance of alternative methods. HIGHLIGHTS: While varying state regulations can be complex, validation studies performed for the candidate methods have led to adoption across 31 states.

Progress in the validation of nociception monitoring in guiding intraoperative analgesic therapy.

PURPOSE OF REVIEW: This article summarizes the current level of validation for several nociception monitors using a categorized validation process to facilitate the comparison of performance. RECENT FINDINGS: Nociception monitors improve the detection of a shift in the nociception and antinociception balance during anesthesia, guiding perioperative analgesic therapy. A clear overview and comparison of the validation process for these monitors is missing. RESULTS: Within a 2-year time-frame, we identified validation studies for four monitors [analgesia nociception index (ANI), nociception level monitor (NOL), surgical pleth index (SPI), and pupillometry]. We categorized these studies in one out of six mandatory validation steps: developmental studies, clinical validation studies, pharmacological validation studies, clinical utility studies, outcome improvement studies and economical evaluation studies. The current level of validation for most monitors is mainly focused on the first three categories, whereas ANI, NOL, and SPI advanced most in the availability of clinical utility studies and provide confirmation of a clinical outcome improvement. Analysis of economical value for public health effects is not yet publicly available for the studied monitors. SUMMARY: This review proposes a stepwise structure for validation of new monitoring technology, which facilitates comparison between the level of validation of different devices and identifies the need for future research questions.

A systematic review of sham acupuncture validation studies.

BACKGROUND: Acupuncture is widely used worldwide; however, studies on its effectiveness have been impeded by limitations regarding the design of appropriate control groups. In clinical research, noninvasive sham acupuncture techniques can only be applied through validation studies. Therefore, this systematic review aimed to evaluate the scope of existing literature on this topic to identify trends. METHODS: We queried Pubmed, EMBASE, and the Cochrane Central Register of Controlled Trials databases from inception to July 2022 for relevant articles. Author names were used to identify additional relevant articles. Two independent reviewers assessed the identified articles based on the inclusion and exclusion criteria. The following data were extracted: study design, information regarding acupuncturists and participants, general and treatment-related characteristics of the intervention and control groups, participants' experience of acupuncture, and research findings. RESULTS: The database query yielded 673 articles, of which 29 articles were included in the final review. Among these, 18 involved the use of one of three devices: Streitberger (n = 5), Park (n = 7), and Takakura (n = 6) devices. The remaining 11 studies used other devices, including self-developed needles. All the included studies were randomized controlled trials. The methodological details of the included studies were heterogeneous with respect to outcomes assessed, blinding, and results. CONCLUSIONS: Sham acupuncture validation studies have been conducted using healthy volunteers, with a focus on blind review and technological developments in sham acupuncture devices. However, theren may be language bias in our findings since we could not query Chinese and Japanese databases due to language barriers. There is a need for more efforts toward establishing control groups suitable for various acupuncture therapy interventions. Moreover, there is a need for more rigorous sham acupuncture validation studies, which could lead to higher-quality clinical studies.

Don't be misled: 3 misconceptions about external validation of clinical prediction models.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

Content Validity and Psychometric Properties of the German Version of the Holm and Cordoba Urinary Tract Infection Score for Uncomplicated Urinary Tract Infections in Women: Protocol for a Validation Study.

BACKGROUND: Uncomplicated urinary tract infections (UTIs) in women are among the most common bacterial infections in primary care. Given the health threats related to the overuse of antibiotics, alternative options are of increasing importance. Patient-reported outcome measures are valuable tools for including the patients' perspective when evaluating the efficacy of these strategies. Aiming to identify a suitable instrument to measure the severity and bothersomeness of UTI symptoms in women, we performed a systematic review of the literature and identified the Holm and Cordoba Urinary Tract Infection Score (HCUTI), which measures the severity, bothersomeness, and impact of uncomplicated UTIs on daily activities. This instrument showed sufficient content validity but needs translation and further validation before it can be used in German research. OBJECTIVE: For use in the German setting, we aim (1) to perform translation and linguistic validation of the HCUTI and (2) to evaluate content validity and psychometric properties of the German version of the HCUTI in a population of women with uncomplicated UTIs. METHODS: The HCUTI will be translated and linguistically validated using the dual-panel method. This process involves a bilingual translation panel and a lay panel to check the comprehensibility of the translation. Content validity of the translated questionnaire will be assessed using cognitive interviews according to the criteria for good content validity as recommended by the COSMIN (Consensus-based Standards for the selection of health Measurement Instruments) group involving women with uncomplicated UTIs and health care professionals. Subsequent psychometric validation of the German version of the HCUTI in a population of women with uncomplicated UTIs will include the assessment of structural validity, internal consistency, test-retest reliability, construct validity, responsiveness, and interpretability. RESULTS: Results of the translation and linguistic validation process and the results of the content validity study were obtained in September 2023 and will be published separately. Data on the psychometric properties of the German version of the HCUTI are anticipated in mid-2024. CONCLUSIONS: We expect that data from the content validity study will provide important suggestions for potential modifications of the HCUTI for use in the German setting. The final version of the questionnaire will be used for the assessment of its psychometric properties in a large population of women with uncomplicated UTIs. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/49903.

Validation of the Scandinavian guidelines for minor and moderate head trauma in children: protocol for a pragmatic, prospective, observational, multicentre cohort study.

INTRODUCTION: Mild traumatic brain injury is common in children and it can be challenging to accurately identify those in need of urgent medical intervention. The Scandinavian guidelines for management of minor and moderate head trauma in children, the Scandinavian Neurotrauma Committee guideline 2016 (SNC16), were developed to aid in risk stratification and decision-making in Scandinavian emergency departments (EDs). This guideline has been validated externally with encouraging results, but internal validation in the intended healthcare system is warranted prior to broad clinical implementation. OBJECTIVE: We aim to validate the diagnostic accuracy of the SNC16 to predict clinically important intracranial injuries (CIII) in paediatric patients suffering from blunt head trauma, assessed in EDs in Sweden and Norway. METHODS AND ANALYSIS: This is a prospective, pragmatic, observational cohort study. Children (aged 0-17 years) with blunt head trauma, presenting with a Glasgow Coma Scale of 9-15 within 24 hours postinjury at an ED in 1 of the 16 participating hospitals, are eligible for inclusion. Included patients are assessed and managed according to the clinical management routines of each hospital. Data elements for risk stratification are collected in an electronic case report form by the examining doctor. The primary outcome is defined as CIII within 1 week of injury. Secondary outcomes of importance include traumatic CT findings, neurosurgery and 3-month outcome. Diagnostic accuracy of the SNC16 to predict endpoints will be assessed by point estimate and 95% CIs for sensitivity, specificity, likelihood ratio, negative predictive value and positive predictive value. ETHICS AND DISSEMINATION: The study is approved by the ethical board in both Sweden and Norway. Results from this validation will be published in scientific journals, and a tailored development and implementation process will follow if the SNC16 is found safe and effective. TRIAL REGISTRATION NUMBER: NCT05964764.

A Predictive Noninvasive Single-Nucleotide Variation-Based Biomarker Signature for Resectable Pancreatic Cancer: Protocol for a Prospective Validation Study.

BACKGROUND: Single-nucleotide variations (SNVs; formerly SNPs) are inherited genetic variants that can be easily determined in routine clinical practice using a simple blood or saliva test. SNVs have potential to serve as noninvasive biomarkers for predicting cancer-specific patient outcomes after resection of pancreatic ductal adenocarcinoma (PDAC). Two recent analyses led to the identification and validation of three SNVs in the CD44 and CHI3L2 genes (rs187115, rs353630, and rs684559), which can be used as predictive biomarkers to help select patients most likely to benefit from pancreatic resection. These variants were associated with an over 2-fold increased risk for tumor-related death in three independent PDAC study cohorts from Europe and the United States, including The Cancer Genome Atlas cohorts (reaching a P value of 1×10-8). However, these analyses were limited by the inherent biases of a retrospective study design, such as selection and publication biases, thereby limiting the clinical use of these promising biomarkers in guiding PDAC therapy. OBJECTIVE: To overcome the limitations of previous retrospectively designed studies and translate the findings into clinical practice, we aim to validate the association of the identified SNVs with survival in a controlled setting using a prospective cohort of patients with PDAC following pancreatic resection. METHODS: All patients with PDAC who will undergo pancreatic resection at three participating hospitals in Switzerland and fulfill the inclusion criteria will be included in the study consecutively. The SNV genotypes will be determined using standard genotyping techniques from patient blood samples. For each genotyped locus, log-rank and Cox multivariate regression tests will be performed, accounting for the relevant covariates American Joint Committee on Cancer stage and resection status. Clinical follow-up data will be collected for at least 3 years. Sample size calculation resulted in a required sample of 150 patients to sufficiently power the analysis. RESULTS: The follow-up data collection started in August 2019 and the estimated end of data collection will be in May 2027. The study is still recruiting participants and 142 patients have been recruited as of November 2023. The DNA extraction and genotyping of the SNVs will be performed after inclusion of the last patient. Since no SNV genotypes have been determined, no data analysis has been performed to date. The results are expected to be published in 2027. CONCLUSIONS: This is the first prospective study of the CD44 and CHI3L2 SNV-based biomarker signature in PDAC. A prospective validation of this signature would enable its clinical use as a noninvasive predictive biomarker of survival after pancreatic resection that is readily available at the time of diagnosis and can assist in guiding PDAC therapy. The results of this study may help to individualize treatment decisions and potentially improve patient outcomes. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/54042.

GRADE concept paper 8: judging the certainty of discrimination performance estimates of prognostic models in a body of validation studies.

BACKGROUND: Prognostic models incorporate multiple prognostic factors to estimate the likelihood of future events for individual patients based on their prognostic factor values. Evaluating these models crucially involves conducting studies to assess their predictive performance, like discrimination. Systematic reviews and meta-analyses of these validation studies play an essential role in selecting models for clinical practice. METHODS: In this paper, we outline 3 thresholds to determine the target for certainty rating in the discrimination of prognostic models, as observed across a body of validation studies. RESULTS AND CONCLUSION: We propose 3 thresholds when rating the certainty of evidence about a prognostic model's discrimination. The first threshold amounts to rating certainty in the model's ability to classify better than random chance. The other 2 approaches involve setting thresholds informed by other mechanisms for classification: clinician intuition or an alternative prognostic model developed for the same disease area and outcome. The choice of threshold will vary based on the context. Instead of relying on arbitrary discrimination cut-offs, our approach positions the observed discrimination within an informed spectrum, potentially aiding decisions about a prognostic model's practical utility.