Participatory methods used in the evaluation of medical devices: a comparison of focus groups, interviews, and a survey.

BACKGROUND: Stakeholder engagement in evaluation of medical devices is crucial for aligning devices with stakeholders' views, needs, and values. Methods for these engagements have however not been compared to analyse their relative merits for medical device evaluation. Therefore, we systematically compared these three methods in terms of themes, interaction, and time-investment. METHODS: We compared focus groups, interviews, and an online survey in a case-study on minimally invasive endoscopy-guided surgery for patients with intracerebral haemorrhage. The focus groups and interviews featured two rounds, one explorative focussing on individual perspectives, and one interactive focussing on the exchange of perspectives between participants. The comparison between methods was made in terms of number and content of themes, how participants interact, and hours invested by all researchers. RESULTS: The focus groups generated 34 themes, the interviews 58, and the survey 42. Various improvements for the assessment of the surgical procedure were only discussed in the interviews. In focus groups, participants were inclined to emphasise agreement and support, whereas the interviews consisted of questions and answers. The total time investment for researchers of focus groups was 95 h, of interviews 315 h, and survey 81 h. CONCLUSIONS: Within the context of medical device evaluation, interviews appeared to be the most appropriate method for understanding stakeholder views since they provide a scope and depth of information that is not generated by other methods. Focus groups were useful to rapidly bring views together. Surveys enabled a quick exploration. Researchers should account for these methodological differences and select the method that is suitable for their research aim.

Early technology review: towards an expedited pathway.

OBJECTIVES: Evidence development for medical devices is often focused on satisfying regulatory requirements with the result that health professional and payer expectations may not be met, despite considerable investment in clinical trials. Early engagement with payers and health professionals could allow companies to understand these expectations and reflect them in clinical study design, increasing chances of positive coverage determination and adoption into clinical practice. METHODS: An example of early engagement through the EXCITE International model using an early technology review (ETR) is described which includes engagement with payers and health professionals to better inform companies to develop data that meet their expectations. ETR is based on an early evidence review, a framework of expectations that guides the process and identified gaps in evidence. The first fourteen ETRs were reviewed for examples of advice to companies that provided additional information from payers and health professionals that was thought likely to impact on downstream outcomes or strategic direction. Given that limitations were imposed by confidentiality, examples were genericized. RESULTS: Advice through early engagement can inform evidence development that coincides with expectations of payers and health professionals through a structured, objective, evidence-based approach. This could reduce the risk of business-related adverse outcomes such as failure to secure a positive coverage determination and/or acceptance by expert health professionals. CONCLUSIONS: Early engagement with key stakeholders exemplified by the ETR approach offers an alternative to the current approach of focusing on regulatory expectations. This could reduce the time to reimbursement and clinical adoption and benefit patient outcomes and/or health system efficiencies.

Prophylactic Mesh Placement During Formation of an End-colostomy: Long-term Randomized Controlled Trial on Effectiveness and Safety.

OBJECTIVE: The aim of this study was to determine if prophylactic mesh placement is an effective, safe, and cost-effective procedure to prevent parastomal hernia (PSH) formation in the long term. BACKGROUND: A PSH is the most frequent complication after stoma formation. Prophylactic placement of a mesh has been suggested to prevent PSH, but long-term evidence to support this approach is scarce. METHODS: In this multicentre superiority trial patients undergoing the formation of a permanent colostomy were randomly assigned to either retromuscular polypropylene mesh reinforcement or conventional colostomy formation. Primary endpoint was the incidence of a PSH after 5 years. Secondary endpoints were morbidity, mortality, quality of life, and cost-effectiveness. RESULTS: A total of 150 patients were randomly assigned to the mesh group (n = 72) or nonmesh group (n = 78). For the long-term follow-up, 113 patients were analyzed, and 37 patients were lost to follow-up. After a median follow-up of 60 months (interquartile range: 48.6-64.4), 49 patients developed a PSH, 20 (27.8%) in the mesh group and 29 (37.2%) in the nonmesh group ( P = 0.22; RD: -9.4%; 95% CI: -24, 5.5). The cost related to the meshing strategy was € 2.239 lower than the nonmesh strategy (95% CI: 491.18, 3985.49), and quality-adjusted life years did not differ significantly between groups ( P = 0.959; 95% CI: -0.066, 0.070). CONCLUSIONS: Prophylactic mesh placement during the formation of an end-colostomy is a safe procedure but does not reduce the incidence of PSH after 5 years of follow-up. It does, however, delay the onset of PSH without a significant difference in morbidity, mortality, or quality of life, and seems to be cost-effective.

Study protocol for a multicenter randomised controlled trial on the (cost)effectiveness of biopsy combined with same-session MR-guided LITT versus biopsy alone in patients with primary irresectable glioblastoma (EMITT trial).

BACKGROUND: Glioblastoma (GBM) is the most common primary, malignant brain tumour with a 5-year survival of 5%. If possible, a glioblastoma is resected and further treated with chemoradiation therapy (CRT), but resection is not feasible in about 30% of cases. Current standard of care in these cases is a biopsy followed by CRT. Magnetic resonance (MR) imaging-guided laser interstitial thermal therapy (LITT) has been suggested as a minimally invasive alternative when surgery is not feasible. However, high-quality evidence directly comparing LITT with standard of care is lacking, precluding any conclusions on (cost-)effectiveness. We therefore propose a multicenter randomized controlled study to assess the (cost-)effectiveness of MR-guided LITT as compared to current standard of care (EMITT trial). METHODS AND ANALYSIS: The EMITT trial will be a multicenter pragmatic randomized controlled trial in the Netherlands. Seven Dutch hospitals will participate in this study. In total 238 patients will be randomized with 1:1 allocation to receive either biopsy combined with same-session MR-guided LITT therapy followed by CRT or the current standard of care being biopsy followed by CRT. The primary outcomes will be health-related quality of life (HR-QoL) (non-inferiority) using EORTC QLQ-C30 + BN20 scores at 5 months after randomization and overall survival (superiority). Secondary outcomes comprise cost-effectiveness (healthcare and societal perspective) and HR-QoL of life over an 18-month time horizon, progression free survival, tumour response, disease specific survival, longitudinal effects, effects on adjuvant treatment, ablation percentage and complication rates. DISCUSSION: The EMITT trial will be the first RCT on the effectiveness of LITT in patients with glioblastoma as compared with current standard of care. Together with the Dutch Brain Tumour Patient association, we hypothesize that LITT may improve overall survival without substantially affecting patients' quality of life. TRIAL REGISTRATION: This trial is registered at ClinicalTrials.gov (NCT05318612).

MR-guided LITT therapy in patients with primary irresectable glioblastoma: a prospective, controlled pilot study.

PURPOSE: Laser interstitial thermal therapy (LITT) is increasingly being used in the treatment of brain tumors, whereas high-quality evidence of its effectiveness is lacking. This pilot examined the feasibility of conducting a randomized controlled trial (RCT) in patients with irresectable newly diagnosed glioblastoma (nGBM), and generated data on technical feasibility and safety. METHODS: We included patients with irresectable nGBM with KPS ≥ 70 and feasible trajectories to ablate ≥ 70% of the tumor volume. Patients were initially randomized to receive either biopsy combined with LITT or biopsy alone, followed by chemoradiation (CRT). Randomization was stopped after 9 patients as the feasibility endpoint with respect to willingness to be randomized was met. Main endpoints were feasibility of performing an RCT, technical feasibility of LITT and safety. Follow-up was 3 months. RESULTS: A total of 15 patients were included, of which 10 patients received a biopsy followed by LITT and 5 patients a biopsy. Most patients were able to complete the follow-up procedures (93% clinical, 86% questionnaires, 78% MRI). Patients were planned within 3 weeks after consultation (median 12 days, range 8-16) and no delay was observed in referring patients for CRT (median 37 days, range 28-61). Two CD ≥ 3 complications occurred in the LITT arm and none in the biopsy arm. CONCLUSION: An RCT to study the effectiveness of LITT in patients with an irresectable nGBM seems feasible with acceptable initial safety data. The findings from this pilot study helped to further refine the design of a larger full-scale multicenter RCT in the Netherlands. Protocol and study identifier: The current study is registered at clinicaltrials.gov (EMITT pilot study, NTR: NCT04596930).

Dealing With Uncertainty in Early Health Technology Assessment: An Exploration of Methods for Decision Making Under Deep Uncertainty.

OBJECTIVES: In early stages, the consequences of innovations are often unknown or deeply uncertain, which complicates early health economic modeling (EHEM). The field of decision making under deep uncertainty uses exploratory modeling (EM) in situations when the system model, input probabilities/distributions, and consequences are unknown or debated. Our aim was to evaluate the use of EM for early evaluation of health technologies. METHODS: We applied EM and EHEM to an early evaluation of minimally invasive endoscopy-guided surgery (MIS) for acute intracerebral hemorrhage and compared these models to derive differences, merits, and drawbacks of EM. RESULTS: EHEM and EM differ fundamentally in how uncertainty is handled. Where in EHEM the focus is on the value of technology, while accounting for the uncertainty, EM focuses on the uncertainty. EM aims to find robust strategies, which give relatively good outcomes over a wide range of plausible futures. This was reflected in our case study. EHEM provided cost-effectiveness thresholds for MIS effectiveness, assuming fixed MIS costs. EM showed that a policy with a population in which most patients had severe intracerebral hemorrhage was most robust, regardless of MIS effectiveness, complications, and costs. CONCLUSIONS: EHEM and EM were found to complement each other. EM seems most suited in the very early phases of innovation to explore existing uncertainty and many potential strategies. EHEM seems most useful to optimize promising strategies, yet EM methods are complex and might only add value when stakeholders are willing to consider multiple solutions to a problem and adopt flexible research and adoption strategies.

Care Pathway Analysis to Inform the Earliest Stages of Technology Development: Scoping Oncological Indications in Need of Innovation.

OBJECTIVES: Identifying unmet needs for innovative solutions across disease contexts is challenging but important for directing funding and research efforts and informing early-stage decisions during the innovation process. Our aim was to study the merits of care pathway analysis to scope disease contexts and guide the development of innovative devices. We used oncologic surgery as a case study, for which many intraoperative imaging techniques are under development. METHODS: Care pathway analysis is a mapping process, which produces graphical maps of clinical pathways using important outcomes and subsequent consequences. We performed care pathway analyses for glioblastoma, breast, bladder, prostate, renal, pancreatic, and oral cavity cancer. Differences between a "perfect" care pathway and the current care pathway in terms of percentage of inadequate margins, associated recurrences, quality of life, and 5-year overall survival were calculated to determine unmet needs. Data from The Netherlands Cancer Registry and literature were used. RESULTS: Care pathway analysis showed that highest percentages of inadequate margins were found in oral cavity cancer (72.5%), glioblastoma (48.7%), and pancreatic cancer (43.9%). Inadequate margins showed the strongest increase in recurrences in cancer of oral cavity, and bladder (absolute increases of 43.5% and 41.2%, respectively). Impact on survival was largest for bladder and oral cavity cancer with positive margins. CONCLUSIONS: Care pathway analysis provides overviews of current clinical paths in multiple indications. Disease contexts can be compared via effectiveness gaps that show the potential need for innovative solutions. This information can be used as basis for stakeholder involvement processes to prioritize care pathways in need of innovation.

Antibiotics for acute otitis media in children.

BACKGROUND: Acute otitis media (AOM) is one of the most common diseases in childhood for which antibiotics are commonly prescribed; a systematic review reported a pooled prevalence of 85.6% in high-income countries. This is an update of a Cochrane Review first published in the Cochrane Library in 1997 and updated in 1999, 2005, 2009, 2013 and 2015. OBJECTIVES: To assess the effects of antibiotics for children with AOM. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, Current Contents, CINAHL, LILACS and two trial registers. The date of the search was 14 February 2023. SELECTION CRITERIA: We included randomised controlled trials comparing 1) antimicrobial drugs with placebo, and 2) immediate antibiotic treatment with expectant observation (including delayed antibiotic prescribing) in children with AOM. DATA COLLECTION AND ANALYSIS: Two review authors independently screened trials for inclusion and extracted data using the standard methodological procedures recommended by Cochrane. Our primary outcomes were: 1) pain at various time points (24 hours, two to three days, four to seven days, 10 to 14 days), and 2) adverse effects likely to be related to the use of antibiotics. Secondary outcomes were: 1) abnormal tympanometry findings, 2) tympanic membrane perforation, 3) contralateral otitis (in unilateral cases), 4) AOM recurrences, 5) serious complications related to AOM and 6) long-term effects (including the number of parent-reported AOM symptom episodes, antibiotic prescriptions and health care utilisation as assessed at least one year after randomisation). We used the GRADE approach to rate the overall certainty of evidence for each outcome of interest. MAIN RESULTS: Antibiotics versus placebo We included 13 trials (3401 children and 3938 AOM episodes) from high-income countries, which we assessed at generally low risk of bias. Antibiotics do not reduce pain at 24 hours (risk ratio (RR) 0.89, 95% confidence interval (CI) 0.78 to 1.01; 5 trials, 1394 children; high-certainty evidence), or at four to seven days (RR 0.76, 95% CI 0.50 to 1.14; 7 trials, 1264 children), but result in almost a third fewer children having pain at two to three days (RR 0.71, 95% CI 0.58 to 0.88; number needed to treat for an additional beneficial outcome (NNTB) 20; 7 trials, 2320 children; high-certainty evidence), and likely result in two-thirds fewer having pain at 10 to 12 days (RR 0.33, 95% CI 0.17 to 0.66; NNTB 7; 1 trial, 278 children; moderate-certainty evidence). Antibiotics increase the risk of adverse events such as vomiting, diarrhoea or rash (RR 1.38, 95% CI 1.16 to 1.63; number needed to treat for an additional harmful outcome (NNTH) 14; 8 trials, 2107 children; high-certainty evidence). Antibiotics reduce the risk of children having abnormal tympanometry findings at two to four weeks (RR 0.83, 95% CI 0.72 to 0.96; NNTB 11; 7 trials, 2138 children), slightly reduce the risk of experiencing tympanic membrane perforations (RR 0.43, 95% CI 0.21 to 0.89; NNTB 33; 5 trials, 1075 children) and halve the risk of contralateral otitis episodes (RR 0.49, 95% CI 0.25 to 0.95; NNTB 11; 4 trials, 906 children). However, antibiotics do not reduce the risk of abnormal tympanometry findings at six to eight weeks (RR 0.89, 95% CI 0.70 to 1.13; 3 trials, 953 children) and at three months (RR 0.94, 95% CI 0.66 to 1.34; 3 trials, 809 children) or late AOM recurrences (RR 0.94, 95% CI 0.79 to 1.11; 6 trials, 2200 children). Severe complications were rare, and the evidence suggests that serious complications do not differ between children treated with either antibiotics or placebo. Immediate antibiotics versus expectant observation We included six trials (1556 children) from high-income countries. The evidence suggests that immediate antibiotics may result in a reduction of pain at two to three days (RR 0.53, 95% CI 0.35 to 0.79; NNTB 8; 1 trial, 396 children; low-certainty evidence), but probably do not reduce the risk of pain at three to seven days (RR 0.75, 95% CI 0.50 to 1.12; 4 trials, 959 children; moderate-certainty evidence), and may not reduce the risk of pain at 11 to 14 days (RR 0.91, 95% CI 0.75 to 1.10; 1 trial, 247 children; low-certainty evidence). Immediate antibiotics increase the risk of vomiting, diarrhoea or rash (RR 1.87, 95% CI 1.39 to 2.51; NNTH 10; 3 trials, 946 children; high-certainty evidence). Immediate antibiotics probably do not reduce the proportion of children with abnormal tympanometry findings at four weeks and evidence suggests that immediate antibiotics may not reduce the risk of tympanic membrane perforation and AOM recurrences. No serious complications occurred in either group. AUTHORS' CONCLUSIONS: This review reveals that antibiotics probably have no effect on pain at 24 hours, a slight effect on pain in the days following and only a modest effect on the number of children with tympanic perforations, contralateral otitis episodes and abnormal tympanometry findings at two to four weeks compared with placebo in children with AOM. In high-income countries, most cases of AOM spontaneously remit without complications. The benefits of antibiotics must be weighed against the possible harms: for every 14 children treated with antibiotics, one child experienced an adverse event (such as vomiting, diarrhoea or rash) that would not have occurred if antibiotics were withheld. For most children with mild disease in high-income countries, an expectant observational approach seems justified. Therefore, clinical management should emphasise advice about adequate analgesia and the limited role for antibiotics.

Ventilation tubes (grommets) for otitis media with effusion (OME) in children.

BACKGROUND: Otitis media with effusion (OME) is an accumulation of fluid in the middle ear cavity, common amongst young children. It may cause hearing loss which, when persistent, may lead to developmental delay, social difficulty and poor quality of life. Management includes watchful waiting, autoinflation, medical and surgical treatment. Insertion of ventilation tubes has often been used as the preferred treatment. OBJECTIVES: To evaluate the effects (benefits and harms) of ventilation tubes (grommets) for OME in children. SEARCH METHODS: We searched the Cochrane ENT Register, CENTRAL, Ovid MEDLINE, Ovid Embase, Web of Science, ClinicalTrials.gov, ICTRP and additional sources for published and unpublished trials on 20 January 2023. SELECTION CRITERIA: We included randomised controlled trials (RCTs) and quasi-RCTs in children (6 months to 12 years) with OME for ≥ 3 months. We included studies that compared ventilation tube (VT) insertion with five comparators: no treatment, watchful waiting (ventilation tubes inserted later, if required), myringotomy, hearing aids and other non-surgical treatments. DATA COLLECTION AND ANALYSIS: We used standard Cochrane methods. Our primary outcomes were determined following a multi-stakeholder prioritisation exercise and were: 1) hearing; 2) OME-specific quality of life; 3) persistent tympanic membrane perforation (as a severe adverse effect of the surgery). Secondary outcomes were: 1) persistence of OME; 2) other adverse effects (including tympanosclerosis, VT blockage and pain); 3) receptive language skills; 4) speech development; 5) cognitive development; 6) psychosocial skills; 7) listening skills; 8) generic health-related quality of life; 9) parental stress; 10) vestibular function; 11) episodes of acute otitis media. We used GRADE to assess the certainty of evidence for key outcomes. Although we included all measures of hearing assessment, the proportion of children who returned to normal hearing was our preferred method, due to challenges in interpreting the results of mean hearing thresholds. MAIN RESULTS: We included 19 RCTs (2888 children). We considered most of the evidence to be very uncertain, due to wide confidence intervals for the effect estimates, few participants, and a risk of performance and detection bias. Here we report our key outcomes at the longest reported follow-up. There were some limitations to the evidence. No studies investigated the comparison of ventilation tubes versus hearing aids. We did not identify any data on disease-specific quality of life; however, many studies were conducted before the development of specific tools to assess this in otitis media. Short-acting ventilation tubes were used in most studies and thus specific data on the use of long-acting VTs is limited. Finally, we did not identify specific data on the effects of VTs in children at increased risk of OME (e.g. with craniofacial syndromes). Ventilation tubes versus no treatment (four studies) The odds ratio (OR) for a return to normal hearing after 12 months was 1.13 with VTs (95% confidence interval (CI) 0.46 to 2.74; 54% versus 51%; 1 study, 72 participants; very low-certainty evidence). At six months, VTs may lead to a large reduction in persistent OME (risk ratio (RR) 0.30, 95% CI 0.14 to 0.65; 20.4% versus 68.0%; 1 study, 54 participants; low-certainty evidence). The evidence is very uncertain about the chance of persistent tympanic membrane perforation with VTs at 12 months (OR 0.85, 95% CI 0.38 to 1.91; 8.3% versus 9.7%; 1 RCT, 144 participants). Early ventilation tubes versus watchful waiting (six studies) There was little to no difference in the proportion of children whose hearing returned to normal after 8 to 10 years (i.e. by the age of 9 to 13 years) (RR for VTs 0.98, 95% CI 0.94 to 1.03; 93% versus 95%; 1 study, 391 participants; very low-certainty evidence). VTs may also result in little to no difference in the risk of persistent OME after 18 months to 6 years (RR 1.21, 95% CI 0.84 to 1.74; 15% versus 12%; 3 studies, 584 participants; very low-certainty evidence). We were unable to pool data on persistent perforation. One study showed that VTs may increase the risk of perforation after a follow-up duration of 3.75 years (RR 3.65, 95% CI 0.41 to 32.38; 1 study, 391 participants; very low-certainty evidence) but the actual number of children who develop persistent perforation may be low, as demonstrated by another study (1.26%; 1 study, 635 ears; very low-certainty evidence). Ventilation tubes versus non-surgical treatment (one study) One study compared VTs to six months of antibiotics (sulphisoxazole). No data were available on return to normal hearing, but final hearing thresholds were reported. At four months, the mean difference was -5.98 dB HL lower (better) for those receiving VTs, but the evidence is very uncertain (95% CI -9.21 to -2.75; 1 study, 125 participants; very low-certainty evidence). No evidence was identified regarding persistent OME. VTs may result in a low risk of persistent perforation at 18 months of follow-up (no events reported; narrative synthesis of 1 study, 60 participants; low-certainty evidence). Ventilation tubes versus myringotomy (nine studies) We are uncertain whether VTs may slightly increase the likelihood of returning to normal hearing at 6 to 12 months, since the confidence intervals were wide and included the possibility of no effect (RR 1.22, 95% CI 0.59 to 2.53; 74% versus 64%; 2 studies, 132 participants; very low-certainty evidence). After six months, persistent OME may be reduced for those who receive VTs compared to laser myringotomy, but the evidence is very uncertain (OR 0.27, 95% CI 0.19 to 0.38; 1 study, 272 participants; very low-certainty evidence). At six months, the risk of persistent perforation is probably similar with the use of VTs or laser myringotomy (narrative synthesis of 6 studies, 581 participants; moderate-certainty evidence). AUTHORS' CONCLUSIONS: There may be small short- and medium-term improvements in hearing and persistence of OME with VTs, but it is unclear whether these persist after longer follow-up. The RCTs included do not allow us to say when (or how much) VTs improve hearing in any specific child. However, interpretation of the evidence is difficult: many children in the control groups recover spontaneously or receive VTs during follow-up, VTs may block or extrude, and OME may recur. The limited evidence in this review also affects the generalisability/applicability of our findings to situations involving children with underlying conditions (e.g. craniofacial syndromes) or the use of long-acting tubes. Consequently, RCTs may not be the best way to determine whether an intervention is likely to be effective in any individual child. Instead, we must better understand the different OME phenotypes to target interventions to children who will benefit most, and avoid over-treating when spontaneous resolution is likely.

The value of reducing arthroscopic partial meniscectomy in the treatment of degenerative meniscus tears: a budget impact analysis.

AIMS: Numerous studies have shown that arthroscopic partial meniscectomy (APM) is not (cost-) effective in patients with symptoms attributed to a degenerative meniscus tear. We aimed to assess the budget impact of reducing APM in routine clinical practice in this population. MATERIALS AND METHODS: A patient-level state transition model was developed to simulate patients recently diagnosed with a degenerative meniscus tear. Three strategies were compared: "current guideline" (i.e., postpone surgery to at least 3 months after diagnosis), "APM at any time" (i.e., APM available directly after diagnosis), and "nonsurgical" (i.e., APM no longer performed). Total societal costs over 5 years were calculated to determine the budget impact. Probabilistic and deterministic sensitivity analyses were conducted to address uncertainty. RESULTS: The average cost per patient over 5 years were EUR 5,077, EUR 4,577, and EUR 4,218, for the "APM at any time," "current guideline," and "nonsurgical" strategy, respectively. Removing APM from the treatment mix (i.e., 30,000 patients per year) in the Netherlands, resulted in a reduction in health care expenditures of EUR 54 million (95 percent confidence interval [CI] EUR 38 million-EUR 70 million) compared to the "current guideline strategy" and EUR 129 million (95 percent CI EUR 102 million-EUR 156 million) compared to the "APM at any time" strategy. Sensitivity analyses showed that uncertainty did not alter our conclusions. CONCLUSIONS: Substantial costs can be saved when APM is no longer performed to treat symptoms attributed to degenerative meniscus tears in the Netherlands. It is therefore recommended to further reduce the use of APM to treat degenerative meniscus tears.

Arthroscopic partial meniscectomy for the degenerative meniscus tear: a comparison of patients included in RCTs and prospective cohort studies.

BACKGROUND AND PURPOSE: Concerns exist regarding the generalizability of results from randomized controlled trials (RCTs) evaluating arthroscopic partial meniscectomy (APM) to treat degenerative meniscus tears. It has been suggested that study populations are not representative of subjects selected for surgery in daily clinical practice. Therefore, we aimed to compare patients included in trials and prospective cohort studies that received APM for a degenerative meniscus tear. PATIENTS AND METHODS: Individual participant data from 4 RCTs and 2 cohort studies undergoing APM were collected. 1,970 patients were analyzed: 605 patients included in RCTs and 1,365 included in the cohorts. We compared patient and disease characteristics, knee pain, overall knee function, and health-related quality of life at baseline between the RCT and cohort groups using standardized differences, ratios comparing the variance of continuous covariates, and graphical methods such as quantile-quantile plots, side-by-side boxplots, and non-parametric density plots. RESULTS: Differences between RCT and the cohort were observed primarily in age (younger patients in the cohort; standardized difference: 0.32) and disease severity, with the RCT group having more severe symptoms (standardized difference: 0.38). While knee pain, overall knee function, and quality of life generally showed minimal differences between the 2 groups, it is noteworthy that the largest observed difference was in knee pain, where the cohort group scored 7 points worse (95% confidence interval 5-9, standardized difference: 0.29). CONCLUSION: Patients in RCTs were largely representative of those in cohort studies regarding baseline scores, though variations in age and disease severity were observed. Younger patients with less severe osteoarthritis were more common in the cohort; however, trial participants still appear to be broadly representative of the target population.

Beyond the RCT: When are Randomized Trials Unnecessary for New Therapeutic Devices, and What Should We Do Instead?

OBJECTIVE: The aim of this study was to develop an evidence-based framework for evaluation of therapeutic devices, based on ethical principles and clinical evidence considerations. SUMMARY BACKGROUND DATA: Nearly all medical products which do not work solely through chemical action are regulated as medical devices. Their huge range of purposes, mechanisms of action and risks pose challenges for regulation. High-profile implantable device failures have fuelled concerns about the level of clinical evidence needed for market approval. Calls for more rigorous evaluation lack clarity about what kind of evaluation is appropriate, and are commonly interpreted as meaning more randomized controlled trials (RCTs). These are valuable where devices are genuinely new and claim to offer measurable therapeutic benefits. Where this is not the case, RCTs may be inappropriate and wasteful. METHODS: Starting with a set of ethical principles and basic precepts of clinical epidemiology, we developed a sequential decision-making algorithm for identifying when an RCT should be performed to evaluate new therapeutic devices, and when other methods, such as observational study designs and registry-based approaches, are acceptable. RESULTS: The algorithm clearly defines a group of devices where an RCT is deemed necessary, and the associated framework indicates that an IDEAL 2b study should be the default clinical evaluation method where it is not. CONCLUSIONS: The algorithm and recommendations are based on the principles of the IDEAL-D framework for medical device evaluation and appear eminently practicable. Their use would create a safer system for monitoring innovation, and facilitate more rapid detection of potential hazards to patients and the public.

IDEAL-D Framework for Device Innovation: A Consensus Statement on the Preclinical Stage.

OBJECTIVE: To extend the IDEAL framework for device innovation, IDEAL-D, to include the preclinical stage of development (stage 0). BACKGROUND: In previous work, the IDEAL collaboration has proposed frameworks for new surgical techniques and complex therapeutic technologies, the central tenet being that development and evaluation can and should proceed together in an ordered and logical manner that balances innovation and safety. METHODS: Following agreement at the IDEAL Collaboration Council, a multidisciplinary working group was formed comprising 12 representatives from healthcare, academia, industry, and a patient advocate. The group conducted a series of discussions following the principles used in the development of the original IDEAL framework. Importantly, IDEAL aims for maximal transparency, optimal validity in the evaluation of primary effects, and minimization of potential risk to patients or others. The proposals were subjected to further review and editing by members of the IDEAL Council before a final consensus version was adopted. RESULTS: In considering which studies are required before a first-in-human study, we have: (1) classified devices according to what they do and the risks they carry, (2) classified studies according to what they show about the device, and (3) made recommendations based on the principle that the more invasive and high risk a device is, the greater proof required of their safety and effectiveness before progression to clinical studies (stage 1). CONCLUSIONS: The proposed recommendations for preclinical evaluation of medical devices represent a proportionate and pragmatic approach that balances the de-risking of first-in-human translational studies against the benefits of rapid translation of new devices into clinical practice.

Development of a diagnostic model to identify patients at high risk for cerebellopontine angle lesions.

PURPOSE: To develop a diagnostic model to identify patients at high risk of a CPA lesion. METHODS: A consecutive cohort of patients with AAD referred by a general practitioner, who underwent their first MRI examination of the CPA between 2005 and 2015 was included. Demographics, symptoms, findings during physical examination, and pure-tone audiometry were used as potential predictors. The presence of a CPA lesion was used as outcome. RESULTS: We analyzed data of 2,214 patients, detecting 73 CPA lesions in 69 (3.1%) patients. The final model contained eleven variables, namely gender [male] [OR 1.055 (95% CI 0.885-1.905)], sudden onset of hearing loss [OR 0.768 (95% CI 0.318-0.992)], gradual onset of hearing loss [OR 1.069 (95% CI 0.500-1.450)], unilateral tinnitus [OR 0.682 (95% CI 0.374-0.999)], complaints of unilateral aural fullness [OR 1.006 (95% CI 0.783-2.155)], instability [OR 1.006 (95% CI 0.580-2.121)], headache [OR 0.959 (95% CI 0.059-1.090)], facial numbness [OR 2.746 (95% CI 0.548-11.085)], facial nerve dysfunction during physical examination [OR 1.024 (95% CI 0.280-3.702)], and asymmetry in BC at 1 kHz [OR 1.013 (95% CI 1.000-1.027)] and 4 kHz [OR 1.008 (95% CI 1.000-1.026)]. CONCLUSION: The proposed diagnostic model is a first step in selecting patients with a high risk of a CPA lesion among those with AAD. It needs to be externally validated prior to its implementation in clinical practice.

Auditory brainstem response prior to MRI compared to standalone MRI in the detection of vestibular schwannoma: A modelling study.

OBJECTIVES: To determine the cost-effectiveness of auditory brainstem response prior to MRI (ABR-MRI) compared to standalone MRI to diagnose vestibular schwannoma. DESIGN: A state transition model was developed to simulate costs and effects (quality-adjusted life years [QALY]) for both diagnostic strategies for patients suspected of a vestibular schwannoma. Model input was derived from literature, hospital databases and expert opinions. Scenario and sensitivity analyses addressed model uncertainty. RESULTS: Over a lifetime horizon, ABR-MRI resulted in a limited cost-saving of €68 or €98 per patient (dependent on MRI sequence) and a health loss of 0.005 QALYs over standalone MRI. ABR-MRI, however, did miss patients with other important pathology (2% of the population) that would have been detected when using standalone MRI. In total, €14 203 or €19 550 could be saved per lost QALY if ABR-MRI was used instead of standalone MRI. The results were sensitive to the detection rate of vestibular schwannoma and health-related quality of life of missed patients. CONCLUSION: The cost-saving with ABR-MRI does not seem to outweigh the number of missed patients with VS and other important pathologies that would have been detected when using standalone MRI.

Novel imaging techniques for intraoperative margin assessment in surgical oncology: A systematic review.

Inadequate margins continue to occur frequently in patients who undergo surgical resection of a tumor, suggesting that current intraoperative methods are not sufficiently reliable in determining the margin status. This clinical demand has inspired the development of many novel imaging techniques that could help surgeons with intraoperative margin assessment. This systematic review provides an overview of novel imaging techniques for intraoperative margin assessment in surgical oncology, and reports on their technical properties, feasibility in clinical practice and diagnostic accuracy. PubMed, Embase, Web of Science and the Cochrane library were systematically searched (2013-2018) for studies reporting on imaging techniques for intraoperative margin assessment. Patient and study characteristics, technical properties, feasibility characteristics and diagnostic accuracy were extracted. This systematic review identified 134 studies that investigated and developed 16 groups of techniques for intraoperative margin assessment: fluorescence, advanced microscopy, ultrasound, specimen radiography, optical coherence tomography, magnetic resonance imaging, elastic scattering spectroscopy, bio-impedance, X-ray computed tomography, mass spectrometry, Raman spectroscopy, nuclear medicine imaging, terahertz imaging, photoacoustic imaging, hyperspectral imaging and pH measurement. Most studies were in early developmental stages (IDEAL 1 or 2a, n = 98); high-quality stage 2b and 3 studies were rare. None of the techniques was found to be clearly superior in demonstrating high feasibility as well as high diagnostic accuracy. In conclusion, the field of imaging techniques for intraoperative margin assessment is highly evolving. This review provides a unique overview of the opportunities and limitations of the currently available imaging techniques.

Neurosurgical Intervention for Supratentorial Intracerebral Hemorrhage.

OBJECTIVE: The effect of surgical treatment for supratentorial spontaneous intracerebral hemorrhage (ICH) and whether it is modified by key baseline characteristics and timing remains uncertain. METHODS: We performed a systematic review and meta-analysis of randomized controlled trials of surgical treatment of supratentorial spontaneous ICH aimed at clot removal. We searched MEDLINE, Embase, and Cochrane databases up to February 21, 2019. Primary outcome was good functional outcome at follow-up; secondary outcomes were death and serious adverse events. We analyzed all types of surgery combined and minimally invasive approaches separately. We pooled risk ratios with 95% confidence intervals and assessed the modifying effect of age, Glasgow Coma Scale, hematoma volume, and timing of surgery with meta-regression analysis. RESULTS: We included 21 studies with 4,145 patients; 4 (19%) were of the highest quality. Risk ratio of good functional outcome after any type of surgery was 1.40 (95% confidence interval [CI] = 1.22-1.60, I2 = 46%, 20 studies), and after minimally invasive surgery it was 1.47 (95% CI = 1.26-1.72, I2 = 47%, 12 studies). For death, the risk ratio for any type of surgery was 0.77 (95% CI = 0.68-0.85, I2 = 23%, 21 studies), and for minimally invasive surgery it was 0.68 (95% CI = 0.56-0.83, I2 = 14%, 13 studies). Serious adverse events were reported infrequently. Surgery seemed more effective when performed sooner after symptom onset (p = 0.04, 12 studies). Age, Glasgow Coma Scale, and hematoma volume did not modify the effect of surgery. INTERPRETATION: Surgical treatment of supratentorial spontaneous ICH may be beneficial, in particular with minimally invasive procedures and when performed soon after symptom onset. Further well-designed randomized trials are needed to demonstrate whether (minimally invasive) surgery improves functional outcome after ICH and to determine the optimal time window of the treatment after symptom onset. ANN NEUROL 2020;88:239-250.

A cost-effectiveness analysis of three approaches for lymph node assessment in patients with low- and intermediate-risk endometrial cancer.

OBJECTIVE: To assess the cost-effectiveness of sentinel lymph node mapping compared to risk factor assessment and routine full lymph node dissection for the assessment of lymph nodes in patients with low- and intermediate-risk endometrioid endometrial cancer. METHODS: A decision-analytic model was designed to compare three lymph node assessment strategies in terms of costs and effects: 1) sentinel lymph node mapping; 2) post-operative risk factor assessment (adjuvant therapy based on clinical and histological risk factors); 3) full lymph node dissection. Input data were derived from systematic literature searches and expert opinion. QALYs were used as measure of effectiveness. The model was built from a healthcare perspective and the impact of uncertainty was assessed with sensitivity analyses. RESULTS: Base-case analysis showed that sentinel lymph node mapping was the most effective strategy for lymph node assessment in patients with low- and intermediate-risk endometrial cancer. Compared to risk factor assessment it was more costly, but the incremental cost effectiveness ratio stayed below a willingness-to-pay threshold of €20,000 with a maximum of €9637/QALY. Sentinel lymph node mapping was dominant compared to lymph node dissection since it was more effective and less costly. Sensitivity analyses showed that the outcome of the model was robust to changes in input values. With a willingness-to-pay threshold of €20,000 sentinel lymph node mapping remained cost-effective in at least 74.3% of the iterations. CONCLUSION: Sentinel lymph node mapping is the most cost-effective strategy to guide the need for adjuvant therapy in patients with low and intermediate risk endometrioid endometrial cancer.

A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint.

Randomized trials typically estimate average relative treatment effects, but decisions on the benefit of a treatment are possibly better informed by more individualized predictions of the absolute treatment effect. In case of a binary outcome, these predictions of absolute individualized treatment effect require knowledge of the individual's risk without treatment and incorporation of a possibly differential treatment effect (ie, varying with patient characteristics). In this article, we lay out the causal structure of individualized treatment effect in terms of potential outcomes and describe the required assumptions that underlie a causal interpretation of its prediction. Subsequently, we describe regression models and model estimation techniques that can be used to move from average to more individualized treatment effect predictions. We focus mainly on logistic regression-based methods that are both well-known and naturally provide the required probabilistic estimates. We incorporate key components from both causal inference and prediction research to arrive at individualized treatment effect predictions. While the separate components are well known, their successful amalgamation is very much an ongoing field of research. We cut the problem down to its essentials in the setting of a randomized trial, discuss the importance of a clear definition of the estimand of interest, provide insight into the required assumptions, and give guidance with respect to modeling and estimation options. Simulated data illustrate the potential of different modeling options across scenarios that vary both average treatment effect and treatment effect heterogeneity. Two applied examples illustrate individualized treatment effect prediction in randomized trial data.

The Use of Decision Analytic Modeling in the Evaluation of Surgical Innovations: A Scoping Review.

OBJECTIVES: The main objective of this review was to map how decision analytic models are used in surgical innovation (in which research phase, with what aim) and to understand how challenges related to the assessment of surgical interventions are incorporated. METHODS: We systematically searched PubMed, Embase, and the Cochrane Library for studies published in 2018. We included original articles using a decision analytic model to compare surgical strategies. We included modeling studies of surgical innovations. General, innovation, and modeling characteristics were extracted, as were outcomes, recommendations, and handling of challenges related to the assessment of surgical interventions (learning curve, incremental innovation, dynamic pricing, quality variation, organizational impact). RESULTS: We included 46 studies. The number of studies increased with each research phase, from 4% (n = 2) in the preclinical phase to 40% (n = 20) in phase 3 studies. Eighty-one studies were excluded because they investigated established surgical procedures, indicating that modeling is predominantly applied after the innovation process. Regardless of the research stage, the aim to determine cost-effectiveness was most frequently identified (n = 40, 87%), whereas exploratory aims (eg, exploring when a strategy becomes cost-effective) were less common (n = 9, 20%). Most challenges related to the assessment of surgical interventions were rarely incorporated in models (eg, learning curve [n = 1, 2%], organizational impact [n = 2, 4%], and incremental innovation [n = 1, 2%]), except for dynamic pricing (n = 10, 22%) and quality variation (n = 6, 13%). CONCLUSIONS: In surgical innovation, modeling is predominantly used in later research stages to assess cost-effectiveness. The exploratory use of modeling seems still largely overlooked in surgery; therefore, the opportunity to inform research and development may not be optimally used.