Guidance of development, validation, and evaluation of algorithms for populating health status in observational studies of routinely collected data (DEVELOP-RCD).

BACKGROUND: In recent years, there has been a growing trend in the utilization of observational studies that make use of routinely collected healthcare data (RCD). These studies rely on algorithms to identify specific health conditions (e.g. diabetes or sepsis) for statistical analyses. However, there has been substantial variation in the algorithm development and validation, leading to frequently suboptimal performance and posing a significant threat to the validity of study findings. Unfortunately, these issues are often overlooked. METHODS: We systematically developed guidance for the development, validation, and evaluation of algorithms designed to identify health status (DEVELOP-RCD). Our initial efforts involved conducting both a narrative review and a systematic review of published studies on the concepts and methodological issues related to algorithm development, validation, and evaluation. Subsequently, we conducted an empirical study on an algorithm for identifying sepsis. Based on these findings, we formulated specific workflow and recommendations for algorithm development, validation, and evaluation within the guidance. Finally, the guidance underwent independent review by a panel of 20 external experts who then convened a consensus meeting to finalize it. RESULTS: A standardized workflow for algorithm development, validation, and evaluation was established. Guided by specific health status considerations, the workflow comprises four integrated steps: assessing an existing algorithm's suitability for the target health status; developing a new algorithm using recommended methods; validating the algorithm using prescribed performance measures; and evaluating the impact of the algorithm on study results. Additionally, 13 good practice recommendations were formulated with detailed explanations. Furthermore, a practical study on sepsis identification was included to demonstrate the application of this guidance. CONCLUSIONS: The establishment of guidance is intended to aid researchers and clinicians in the appropriate and accurate development and application of algorithms for identifying health status from RCD. This guidance has the potential to enhance the credibility of findings from observational studies involving RCD.

The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: manuscript checklist.

Adequate and transparent reporting is necessary for critically appraising published research. Yet, ample evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-identified the minimum information needed to report and evaluate observational studies and clinical trials in oral health: the OHStat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The guidelines were subsequently revised by the Task Force's writing group. The guidelines draw heavily from the Consolidated Standards for Reporting Trials (CONSORT), Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), and CONSORT harms guidelines and incorporate the SAMPL guidelines for reporting statistics, the CLIP principles for documenting images, and the GRADE indicating the quality of evidence. The guidelines also recommend reporting estimates in clinically meaningful units using confidence intervals, rather than relying on P values. In addition, OHStat introduces 7 new guidelines that concern the text itself, such as checking the congruence between abstract and text, structuring the discussion, and listing conclusions to make them more specific. OHStat does not replace other reporting guidelines; it incorporates those most relevant to dental research into a single document. Manuscripts using the OHStat guidelines will provide more information specific to oral health research.

The STROCSS 2024 guideline: strengthening the reporting of cohort, cross-sectional, and case-control studies in surgery.

INTRODUCTION: First released in 2017, the STROCSS guidelines have become integral for promoting high-quality reporting of observational research in surgery. However, regular updates are essential to ensure they remain relevant and of value to surgeons. Building on the 2021 updates, the authors have developed the STROCSS 2024 guidelines. This timely revision aims to address residual reporting gaps, assimilate recent advances, and further strengthen observational study quality across all surgical disciplines. METHODS: A core steering committee compiled proposed changes to update the STROCSS 2021 guidelines based on identified gaps in prior iterations. An expert panel of surgical research leaders then evaluated the proposed changes for inclusion. A Delphi consensus exercise was used. Proposals that scored between 7-9 on a nine-point Likert agreement scale, by ≥70% of Delphi participants, were integrated into the STROCSS 2024 checklist. RESULTS: In total, 46 of 56 invited participants (82%) completed the Delphi survey and hence participated in the development of STROCSS 2024. All suggested amendments met the criteria for inclusion, indicating a high level of agreement among the Delphi group. All proposed items were therefore integrated into the final revised checklist. CONCLUSION: The authors present the updated STROCSS 2024 guidelines, which have been developed through expert consensus to further enhance the transparency and reporting quality of observational research in surgery.

Some common, fatal flaws in systematic reviews of observational studies.

When evidence from randomized controlled trials about the effectiveness and safety of an intervention is unclear, researchers may choose to review the nonrandomized evidence. All systematic reviews pose considerable challenges, and the level of methodological expertise required to undertake a useful review of nonrandomized intervention studies is both high and often severely underestimated. Using the example of the endometrial receptivity array, we review some common, critical flaws in systematic reviews of this nature, including errors in critical appraisal and meta-analysis.

Emulating an RCT in observational research in chest trauma: The target trial approach.

BACKGROUND: While the concept of a "target trial"-optimizing the quality of observational studies by attempting to emulate the ideal world conditions of a randomized controlled trial-was first expounded over a decade ago, the take up of this concept in the design and analysis of trials in trauma is lacking. The target trial approach avoids common errors in observational research to increase its scientific validity as well as potentially enable causal questions to be answered without the expense and intricacies of a randomized controlled trial. This review article briefly introduces the reader to the concepts and utility of a "target trial" approach before providing demonstrations of its application in the subject area of chest trauma. METHODS: Four articles published in the last 5 years-two case control and two cohort studies-are chosen and considered in terms of their causal question; study population; inclusion and exclusion criteria; designation of time 0; clarity of the follow-up period; study outcomes; methods to minimize confounding; results; overall issues regarding study time; and the presence of avoidable errors such as introduction of immortal time bias or information bias. RESULTS: Two of the studies had an unclear causal question; none of the studies designated a time 0; the follow-up period was unclear for all but one of the studies; and one study had a serious issue with information bias resulting from differential misclassification. CONCLUSION: Failure to emulate a "target trial" framework may lead to serious methodologic issues in observational research. Expansion of the awareness of this approach in trauma literature will improve the quality of our observational research and potentially translate into significant benefits for our patients.

Global epidemiology of acromegaly: a systematic review and meta-analysis.

OBJECTIVE: To date, no systematic reviews and meta-analysis on the global epidemiology of acromegaly are available in the literature. The aims of this study are to provide a systematic review and a meta-analysis of the global epidemiology of acromegaly and to evaluate the quality of study reporting for the identified studies. METHODS: MEDLINE, EMBASE and The Cochrane Library databases were searched for studies assessing the epidemiology of acromegaly from inception until 31 January 2020. We included original observational studies written in English, reporting acromegaly prevalence and/or incidence for a well-defined geographic area. Two reviewers independently extracted data and performed quality assessments. Prevalence and incidence pooled estimates were derived by performing a random-effects meta-analysis. RESULTS: A total of 32 studies were included in the systematic review, and 22 of them were included in the meta-analysis. The pooled prevalence of acromegaly was 5.9 (95% CI: 4.4-7.9) per 100 000 persons, while the incidence rate (IR) was 0.38 (95% CI: 0.32-0.44) cases per 100 000 person-years. For both prevalence and IR, considerable between-study heterogeneity was found (I2 = 99.3 and 86.0%, respectively). The quality of study reporting was rated as the medium for 20 studies and low for 12 studies. CONCLUSIONS: Although the largest amount of heterogeneity was due to the high precision of the studies' estimates, data source and geographic area could represent relevant study-level factors which could explain about 50% of the total between-study variability. Large-scale high-quality studies on the epidemiology of acromegaly are warranted to help the public health system in making decisions.

A core outcome set development for a French national prospective study about the effect of mediolateral episiotomy on obstetric anal sphincter injury during operative vaginal delivery (INSTRUMODA).

BACKGROUND: We aimed at developing a core outcome and variables of interest set to investigate the effects of mediolateral episiotomy on Obstetric Anal Sphincter Injury (OASI) during and after operative delivery in nulliparous women in a large-scale one-year observational French study including 15,000 women (INSTRUMODA). METHODS: A list of outcomes and variables of interest was suggested to obstetricians participating in the INSTRUMODA study using online questionnaires divided into 7 categories: the woman's history and course of pregnancy, course of labor, modalities of operative delivery, episiotomy characteristics, immediate maternal morbidity, one-year maternal morbidity, immediate neonatal morbidity. We used a three-round DELPHI method to reach a consensus. In the first round, outcomes and variables considered as essential by 70% or more of obstetricians were included in the corpus whereas they were excluded when 70% rated them as "not important". In the second round, non-consensual outcomes and variables were reassessed and excluded or definitively included if considered as "not important" or essential by 50% or more of the obstetricians. During the first round, obstetricians were invited to suggest new outcomes and/or variables that were then assessed in the second and third round. We used the same method to develop a core outcome and variables of interest set in a population of women in the community recruited via an association of patients. At the end of the procedure the core outcome and variables of interest sets were merged to provide the final core outcome set for the INSTRUMODA study. RESULTS: Fifty-three obstetricians and 16 women filled out questionnaires. After the 3 rounds of Delphi procedure in each population, 74 outcomes and variables were consensually reported by obstetricians and 92 by women in the community. By mixing these two consensual corpora we reported a final consensual list of 114 variables of interest and outcomes for both obstetricians and women. CONCLUSION: We established a core outcome and variables of interest set among obstetricians and women in the community to investigate the association between mediolateral episiotomy and OASI during operative delivery. TRIAL REGISTRATION: The INSTRUMODA study was registered on https://clinicaltrials.gov on June 25, 2020 ( NCT04446780 ).

Intraoperative Magnetic Resonance Imaging for Low-Grade and High-Grade Gliomas: What Is the Evidence? A Meta-Analysis.

BACKGROUND: The benefit of intraoperative magnetic resonance imaging (iMRI) in gliomas remains unclear. We performed a meta-analysis of outcomes with iMRI-guided surgery in high-grade gliomas (HGGs) and low-grade gliomas (LGGs). METHODS: Databases were searched until November 29, 2018 for randomized controlled trials (RCTs) and observational studies (OBS) comparing iMRI use with conventional neurosurgery. Pooled risk ratios (RRs) or hazard ratios were evaluated with the random-effects model. Outcomes included extent of resection (EOR), gross total resection (GTR), progression-free survival (PFS), overall survival (OS), and length of surgery (LOS), stratified by study design and glioma grade. RESULTS: Fifteen articles (3 RCTs and 12 OBS) were included. In RCTs, GTR was higher in iMRI compared with conventional neurosurgery (RR, 1.42; 95% confidence interval [CI], 1.17-1.73; I2, 7%) overall, for LGGs (1.91; 95% CI, 1.19-3.06), but not HGGs (1.24; 95% CI, 0.89-1.73), with no difference in EOR, PFS, OS, and LOS. For OBS, GTR was higher (RR, 1.65; 95% CI, 1.43-1.90; I2, 4%) overall, and for LGGs (1.63; 95% CI, 1.17-2.28; I2, 0%) and HGGs (1.62; 95% CI, 1.36-1.92; I2, 19%). EOR was greater with iMRI (6%; 95% CI, 4%-8%; I2, 44%) overall, in LGGs (5%; 95% CI, 2%-8%; I2, 37%) and HGGs (7%; 95% CI, 4%-10%; I2, 13%). There was no difference in PFS, OS, and LOS with iMRI. CONCLUSIONS: IMRI use improved GTR in gliomas, including LGGs. However, no PFS and OS benefit was shown in the meta-analysis.

Defining treatment regimens and lines of therapy using real-world data in oncology.

Retrospective observational research relies on databases that do not routinely record lines of therapy or reasons for treatment change. Standardized approaches to estimate lines of therapy were developed and evaluated in this study. A number of rules were developed, assumptions varied and macros developed to apply to large datasets. Results were investigated in an iterative process to refine line of therapy algorithms in three different cancers (lung, colorectal and gastric). Three primary factors were evaluated and included in the estimation of lines of therapy in oncology: defining a treatment regimen, addition/removal of drugs and gap periods. Algorithms and associated Statistical Analysis Software (SAS®) macros for line of therapy identification are provided to facilitate and standardize the use of real-world databases for oncology research.

Lay abstract Most, if not all, real-world healthcare databases do not contain data explaining treatment changes, requiring that rules be applied to estimate when treatment changes may reflect advancement of underlying disease. This study investigated three tumor types (lung, colorectal and gastric cancer) to develop and provide rules that researchers can apply to real-world databases. The resulting algorithms and associated SAS^® macros from this work are provided for use in the Supplementary data.

Informing Healthcare Decisions with Observational Research Assessing Causal Effect. An Official American Thoracic Society Research Statement.

Rationale: Decisions in medicine are made on the basis of knowledge and reasoning, often in shared conversations with patients and families in consideration of clinical practice guideline recommendations, individual preferences, and individual goals. Observational studies can provide valuable knowledge to inform guidelines, decisions, and policy.Objectives: The American Thoracic Society (ATS) created a multidisciplinary ad hoc committee to develop a research statement to clarify the role of observational studies-alongside randomized controlled trials (RCTs)-in informing clinical decisions in pulmonary, critical care, and sleep medicine.Methods: The committee examined the strengths of observational studies assessing causal effects, how they complement RCTs, factors that impact observational study quality, perceptions of observational research, and, finally, the practicalities of incorporating observational research into ATS clinical practice guidelines.Measurements and Main Results: There are strengths and weakness of observational studies as well as RCTs. Observational studies can provide evidence in representative and diverse patient populations. Quality observational studies should be sought in the development of ATS clinical practice guidelines, and medical decision-making in general, when 1) no RCTs are identified or RCTs are appraised as being of low- or very low-quality (replacement); 2) RCTs are of moderate quality because of indirectness, imprecision, or inconsistency, and observational studies mitigate the reason that RCT evidence was downgraded (complementary); or 3) RCTs do not provide evidence for outcomes that a guideline committee considers essential for decision-making (e.g., rare or long-term outcomes; "sequential").Conclusions: Observational studies should be considered in developing clinical practice guidelines and in making clinical decisions.

Rectus femoris transfer in children with cerebral palsy: comparing a propensity score-matched observational study to a randomized controlled trial.

AIM: To test whether an observational study employing propensity score matching could accurately estimate the causal treatment effects of rectus femoris transfer (RFT) as part of single-event multilevel surgery (SEMLS) in ambulatory children with cerebral palsy. METHOD: We used a large clinical database to derive a propensity score for treatment assignment (SEMLS±RFT) and used this score to generate a matched patient cohort. We compared the causal treatment effects estimated from this matched cohort with a previously published randomized controlled trial (RCT). RESULTS: The treated arms of the observational study and RCT were well matched. There were 129 limbs (81 males) with a mean age of 10 years 7 months (4y 7mo) in the treated arm of the observational study, and 129 limbs (68 males) with a mean age of 10 years 2 months (3y 9mo) in the control arm of the observational study. Differences between the observational study and RCT cohorts were clinically meaningless for knee flexion kinematics (1-4°), timing of knee angle extrema (<3% gait cycle), and speed (<5mm/s). Postoperative changes in the observational study matched those from the RCT. All but one of the observational study confidence intervals were completely contained within the corresponding RCT confidence interval; there were no meaningful differences in magnitude or sign of key outcomes related to stiff knee gait. INTERPRETATION: Propensity score matching is an accurate method for estimating the causal treatment effects of RFT as part of an SEMLS. It seems reasonable to extend this approach to other common components of SEMLS treatment in this population. WHAT THIS PAPER ADDS: Propensity score matching is an accurate method for estimating the causal treatment effects of rectus femoris transfer (RFT) in ambulatory children with cerebral palsy (CP). The causal treatment effects for RFT surgery in ambulatory children diagnosed with CP were validated.