Participatory methods used in the evaluation of medical devices: a comparison of focus groups, interviews, and a survey.

BACKGROUND: Stakeholder engagement in evaluation of medical devices is crucial for aligning devices with stakeholders' views, needs, and values. Methods for these engagements have however not been compared to analyse their relative merits for medical device evaluation. Therefore, we systematically compared these three methods in terms of themes, interaction, and time-investment. METHODS: We compared focus groups, interviews, and an online survey in a case-study on minimally invasive endoscopy-guided surgery for patients with intracerebral haemorrhage. The focus groups and interviews featured two rounds, one explorative focussing on individual perspectives, and one interactive focussing on the exchange of perspectives between participants. The comparison between methods was made in terms of number and content of themes, how participants interact, and hours invested by all researchers. RESULTS: The focus groups generated 34 themes, the interviews 58, and the survey 42. Various improvements for the assessment of the surgical procedure were only discussed in the interviews. In focus groups, participants were inclined to emphasise agreement and support, whereas the interviews consisted of questions and answers. The total time investment for researchers of focus groups was 95 h, of interviews 315 h, and survey 81 h. CONCLUSIONS: Within the context of medical device evaluation, interviews appeared to be the most appropriate method for understanding stakeholder views since they provide a scope and depth of information that is not generated by other methods. Focus groups were useful to rapidly bring views together. Surveys enabled a quick exploration. Researchers should account for these methodological differences and select the method that is suitable for their research aim.

Impact of time-to-treatment on survival for advanced non-small cell lung cancer patients in the Netherlands: a nationwide observational cohort study.

BACKGROUND: The assumption that more rapid treatment improves survival of advanced non-small cell lung cancer (NSCLC) has not yet been proven. We studied the relation between time-to-treatment and survival in advanced stage NSCLC patients in a large multicentric nationwide retrospective cohort. Additionally, we identified factors associated with delay. METHOD: We selected 10 306 patients, diagnosed and treated between 2014 and 2019 for clinical stage III and IV NSCLC, from the Netherlands Cancer Registry that includes nationwide data from 109 Dutch hospitals. Associations between survival and time-to-treatment were tested with Cox proportional hazard regression analyses. Time-to-treatment was adjusted for multiple covariates including diagnostic procedures and type of therapy. Factors associated with delay were identified by multilevel logistic regression. RESULTS: Risk of death significantly decreased with longer time-to-treatment for stage III patients receiving only radiotherapy (adjusted HR, aHR >21 days: 0.59 (95% CI 0.48 to 0.73)) or any type of systemic therapy (aHR >49 days: 0.72 (95% CI 0.56 to 0.91)) and stage IV patients receiving chemotherapy and/or immunotherapy (aHR >21 days: 0.81 (95% CI 0.73 to 0.88)). No significant association was found for stage III patients treated with chemoradiotherapy and stage IV patients treated with targeted therapy. More complex diagnostic procedures often delay treatment. CONCLUSION: Although in general it is important to start treatment as early as possible, our study finds no evidence that a more rapid start of treatment improves outcomes in advanced stage NSCLC patients. The benefit of urgent treatment is probably confounded by unmeasured patient and tumour characteristics and, clinical urgency dictating timelines of treatment. Time-to-treatment and its impact should be continuously evaluated as therapeutic strategies continue to evolve and improve.

Patient-preferred outcomes in patients with vestibular schwannoma: a qualitative content analysis of symptoms, side effects and their impact on health-related quality of life.

PURPOSE: During counseling and management of patients with vestibular schwannoma (VS), the emphasis is shifting from tumour control and nerve preservation towards maintaining or improving health-related quality of life (HRQoL). Understanding the patients' perspective and impact of VS is, therefore, of utmost importance. The current study aimed to identify treatment outcomes preferred by patients and to explore the patient-reported VS symptoms and management-related side effects and their impact on HRQoL. METHODS: Patients with VS were contacted through the Dutch VS association Stichting Hoormij and questioned using a semi-structured, cross-sectional online survey. Patients were asked to report and rank symptoms and side effects, with their impact on HRQoL and frequency of occurrence. Results were structured through qualitative content analysis. Coded symptoms, side effects, impacts, frequencies, and patient-preferred outcomes were analysed and summarized with descriptive statistics. RESULTS: Of the 231 respondents, 71% were actively treated. Hearing (symptoms vs. side effects: 78.8% vs. 63.6%), balance (62.3%; 48.8%), and energy issues (33.8%; 32.6%) were the most frequently mentioned symptoms and management-related side effects. Fatigue, deafness, headaches, and hearing loss had the highest impact on HRQoL. The majority of patients identified hearing preservation (61%), balance preservation (38.5%), and reduced tinnitus (34.6%) to be the patient-preferred outcomes. CONCLUSION: This qualitative study demonstrates that in this population many patients with VS encounter participation difficulties in their daily physical and social activities and value hearing and balance preservation, reduced tinnitus, and restored energy as preferred outcomes as they are hampered by symptoms and side effects related to hearing, balance, and energy. Healthcare professionals should consider these key points and use these and the patient-preferred outcomes in consultation, shared decision making, treatment, and follow-up to optimize patient-centred care.

Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification.

OBJECTIVES: Early assessments of health technologies help to better align and integrate their development and assessment. Such assessments can take many forms and serve different purposes, hampering users in their selection of the most appropriate method for a specific goal. The aim of this scoping review was to structure the large set of methods according to their specific goal. METHODS: A scoping review was conducted using PubMed and reference lists of retrieved articles, to identify review studies with a methodological focus. From the included reviews, all individual methods were listed. Based on additional literature and examples, we extracted the specific goal of each method. All goals were clustered to derive a set of subclasses and methods were grouped into these subclasses. RESULTS: Of the 404 screened, 5 reviews were included, and 1 was added when searching reference lists. The reviews described 56 methods, of which 43 (77%) were included and classified as methods to (1) explore the nature and magnitude of the problem, (2) estimate the nature and magnitude of the expected (societal) value, (3) identify conditions for the potential value to materialize, and (4) help develop and design the type of research that is needed. CONCLUSIONS: The wide range of methods for exploring the societal value of health technologies at an early stage of development can be subdivided into a limited number of classes, distinguishing methods according to their specific objective. This facilitates selection of appropriate methods, depending on the specific needs and aims.

ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists' training.

OBJECTIVES: This study aims to define consensus-based criteria for acquiring and reporting prostate MRI and establishing prerequisites for image quality. METHODS: A total of 44 leading urologists and urogenital radiologists who are experts in prostate cancer imaging from the European Society of Urogenital Radiology (ESUR) and EAU Section of Urologic Imaging (ESUI) participated in a Delphi consensus process. Panellists completed two rounds of questionnaires with 55 items under three headings: image quality assessment, interpretation and reporting, and radiologists' experience plus training centres. Of 55 questions, 31 were rated for agreement on a 9-point scale, and 24 were multiple-choice or open. For agreement items, there was consensus agreement with an agreement ≥ 70% (score 7-9) and disagreement of ≤ 15% of the panellists. For the other questions, a consensus was considered with ≥ 50% of votes. RESULTS: Twenty-four out of 31 of agreement items and 11/16 of other questions reached consensus. Agreement statements were (1) reporting of image quality should be performed and implemented into clinical practice; (2) for interpretation performance, radiologists should use self-performance tests with histopathology feedback, compare their interpretation with expert-reading and use external performance assessments; and (3) radiologists must attend theoretical and hands-on courses before interpreting prostate MRI. Limitations are that the results are expert opinions and not based on systematic reviews or meta-analyses. There was no consensus on outcomes statements of prostate MRI assessment as quality marker. CONCLUSIONS: An ESUR and ESUI expert panel showed high agreement (74%) on issues improving prostate MRI quality. Checking and reporting of image quality are mandatory. Prostate radiologists should attend theoretical and hands-on courses, followed by supervised education, and must perform regular performance assessments. KEY POINTS: • Multi-parametric MRI in the diagnostic pathway of prostate cancer has a well-established upfront role in the recently updated European Association of Urology guideline and American Urological Association recommendations. • Suboptimal image acquisition and reporting at an individual level will result in clinicians losing confidence in the technique and returning to the (non-MRI) systematic biopsy pathway. Therefore, it is crucial to establish quality criteria for the acquisition and reporting of mpMRI. • To ensure high-quality prostate MRI, experts consider checking and reporting of image quality mandatory. Prostate radiologists must attend theoretical and hands-on courses, followed by supervised education, and must perform regular self- and external performance assessments.

Surgical removal versus retention for the management of asymptomatic disease-free impacted wisdom teeth.

BACKGROUND: Prophylactic removal of asymptomatic disease-free impacted wisdom teeth is the surgical removal of wisdom teeth in the absence of symptoms and with no evidence of local disease. Impacted wisdom teeth may be associated with pathological changes, such as pericoronitis, root resorption, gum and alveolar bone disease (periodontitis), caries and the development of cysts and tumours. When surgical removal is performed in older people, the risk of postoperative complications, pain and discomfort is increased. Other reasons to justify prophylactic removal of asymptomatic disease-free impacted third molars have included preventing late lower incisor crowding, preventing damage to adjacent structures such as the second molar or the inferior alveolar nerve, in preparation for orthognathic surgery, in preparation for radiotherapy or during procedures to treat people with trauma to the affected area. Removal of asymptomatic disease-free wisdom teeth is a common procedure, and researchers must determine whether evidence supports this practice. This review is an update of an review originally published in 2005 and previously updated in 2012 and 2016. OBJECTIVES: To evaluate the effects of removal compared with retention (conservative management) of asymptomatic disease-free impacted wisdom teeth in adolescents and adults. SEARCH METHODS: Cochrane Oral Health's Information Specialist searched the following databases: Cochrane Oral Health's Trials Register (to 10 May 2019), the Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library, 2019, Issue 4), MEDLINE Ovid (1946 to 10 May 2019), and Embase Ovid (1980 to 10 May 2019). The US National Institutes of Health Trials Registry (ClinicalTrials.gov)and the World Health Organization International Clinical Trials Registry Platform were searched for ongoing trials. No restrictions were placed on the language or date of publication when searching the electronic databases. . SELECTION CRITERIA: We included randomised controlled trials (RCTs), with no restriction on length of follow-up, comparing removal (or absence) with retention (or presence) of asymptomatic disease-free impacted wisdom teeth in adolescents or adults. We also considered quasi-RCTs and prospective cohort studies for inclusion if investigators measured outcomes with follow-up of five years or longer. DATA COLLECTION AND ANALYSIS: Eight review authors screened search results and assessed the eligibility of studies for inclusion according to the review inclusion criteria. Eight review authors independently and in duplicate conducted the risk of bias assessments. When information was unclear, we contacted the study authors for additional information. MAIN RESULTS: This review update includes the same two studies that were identified in our previous version of the review: one RCT with a parallel-group design, which was conducted in a dental hospital setting in the United Kingdom, and one prospective cohort study, which was conducted in the private sector in the USA. Primary outcome No eligible studies in this review reported the effects of removal compared with retention of asymptomatic disease-free impacted wisdom teeth on health-related quality of life Secondary outcomes We found only low- to very low-certainty evidence of the effects of removal compared with retention of asymptomatic disease-free impacted wisdom teeth for a limited number of secondary outcome measures. One prospective cohort study, reporting data from a subgroup of 416 healthy male participants, aged 24 to 84 years, compared the effects of the absence (previous removal or agenesis) against the presence of asymptomatic disease-free impacted wisdom teeth on periodontitis and caries associated with the distal aspect of the adjacent second molar during a follow-up period of three to over 25 years. Very low-certainty evidence suggests that the presence of asymptomatic disease-free impacted wisdom teeth may be associated with increased risk of periodontitis affecting the adjacent second molar in the long term. In the same study, which is at serious risk of bias, there is insufficient evidence to demonstrate a difference in caries risk associated with the presence or absence of impacted wisdom teeth. One RCT with 164 randomised and 77 analysed adolescent participants compared the effect of extraction with retention of asymptomatic disease-free impacted wisdom teeth on dimensional changes in the dental arch after five years. Participants (55% female) had previously undergone orthodontic treatment and had 'crowded' wisdom teeth. No evidence from this study, which was at high risk of bias, was found to suggest that removal of asymptomatic disease-free impacted wisdom teeth has a clinically significant effect on dimensional changes in the dental arch. The included studies did not measure any of our other secondary outcomes: costs, other adverse events associated with retention of asymptomatic disease-free impacted wisdom teeth (pericoronitis, root resorption, cyst formation, tumour formation, inflammation/infection) and adverse effects associated with their removal (alveolar osteitis/postoperative infection, nerve injury, damage to adjacent teeth during surgery, bleeding, osteonecrosis related to medication/radiotherapy, inflammation/infection). AUTHORS' CONCLUSIONS: Insufficient evidence is available to determine whether asymptomatic disease-free impacted wisdom teeth should be removed or retained. Although retention of asymptomatic disease-free impacted wisdom teeth may be associated with increased risk of periodontitis affecting adjacent second molars in the long term, the evidence is very low certainty. Well-designed RCTs investigating long-term and rare effects of retention and removal of asymptomatic disease-free impacted wisdom teeth, in a representative group of individuals, are unlikely to be feasible. In their continuing absence, high quality, long-term prospective cohort studies may provide valuable evidence in the future. Given the current lack of available evidence, patient values should be considered and clinical expertise used to guide shared decision-making with people who have asymptomatic disease-free impacted wisdom teeth. If the decision is made to retain these teeth, clinical assessment at regular intervals to prevent undesirable outcomes is advisable.

On the integration of early health technology assessment in the innovation process: reflections from five stakeholders.

Early health technology assessment (HTA), which includes all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty, has seen many applications in recent years. However, it is still unclear how such early value assessments can be integrated into the technology innovation process. This commentary contributes to the discussion on the purposes early HTA can serve. Similarities and differences in the perspectives of five stakeholders (i.e., the hospital, the patient, the assessor, the medical device industry, and the policy maker) on the purpose, value, and potential challenges of early HTA are described. All five stakeholders agreed that integrating early HTA in the innovation process has the possibility to shape and refine an innovation, and inform research and development decisions. The early assessment, using a variety of methodologies, can provide insights that are relevant for all stakeholders but several challenges, for example, feasibility and responsibility, need to be addressed before early HTA can become standard practice. For early evaluations to be successful, all relevant stakeholders including patients need to be involved. Also, nimble, flexible assessment methods are needed that fit the dynamics of medical technology. Best practices should be shared to optimize both the innovation process and the methods to perform an early value assessment.

Early Assessment of Proof-of-Problem to Guide Health Innovation.

BACKGROUND: Although the relevance of both push and pull factors is acknowledged in models of innovation, needs, broadly defined, are rarely considered, whereas supply-driven innovation in publicly funded health systems carries the risk that it may not match the underlying problems experienced by patients and consumers. OBJECTIVES: To explore a mixed-methods, multistakeholder approach that focuses on pertinent problems when assessing the potential value of an innovation as applied to a case of surgical innovation in meniscus surgery. METHODS: Through interviews of stakeholders (n = 11) we sought to identify current problems of meniscus surgery in the Netherlands. On the basis of the subsequent problem definitions, we used stakeholder and literature input to quantify the room for improvement and stakeholder engagement to uncover possible barriers and facilitators to the implementation of the proposed innovation. RESULTS: Despite being enthusiastic about the ingenuity of the proposed innovation and seeing some potential for cost saving, most stakeholders (n = 10) agreed that there are no major problems in current meniscus surgery meriting the innovation. They even discerned pragmatic barriers that would challenge the potential cost savings. CONCLUSIONS: By adopting a problem-oriented multistakeholder approach to early health technology assessment, we were able to estimate the potential value of an innovation in its social context, finding that, beyond the initial enthusiasm, the proposed innovation was unlikely to resolve the problems distinguished by the stakeholders. We concluded that our multistakeholder, mixed-methods approach to early health technology assessment is feasible and helps foster more demand-driven innovations.

Implementation of stepped care for patients with chronic fatigue syndrome in community-based mental health care: outcomes at post-treatment and long-term follow-up.

BACKGROUND: Cognitive behavioural therapy (CBT) is an evidence-based treatment for chronic fatigue syndrome (CFS). Stepped care for CFS, consisting of a minimal intervention followed by face-to-face CBT, was found efficacious when tested in a CFS specialist centre. Stepped care implemented in a community-based mental health centre (MHC) has not yet been evaluated. AIMS: (1) To test the effectiveness of stepped care for CFS implemented in a MHC at post-treatment and at long-term follow-up; and (2) compare post-treatment outcomes of implemented stepped care with treatment outcomes of a CFS specialist centre. METHOD: An uncontrolled study was used to test effectiveness of stepped care implemented in a MHC (n = 123). The outcomes of implemented care were compared with the outcomes of specialist care reported in previous studies (n = 583). Data on outcomes from implemented stepped care were gathered at post-treatment and at long-term follow-up. Mixed models were used as method of analysis. RESULTS: Fatigue decreased and physical functioning increased significantly following implemented stepped care (both p < .001). The follow-up was completed by 94 patients (78%) within 1-6 years after treatment. Treatment effects were sustained to follow-up. Patients in the MHC showed less improvement directly following stepped care compared with patients in a CFS specialist centre (p < .01). CONCLUSION: Implemented stepped care for CFS is effective with sustained treatment gains at long-term follow-up. There is room for improvement when compared with outcomes of a CFS specialist centre. Some suggestions are made on how to improve stepped care.

Optimal search strategies for identifying moderators and predictors of treatment effects in PubMed.

BACKGROUND: Treatment effects differ across patients. To guide selection of treatments for patients, it is essential to acknowledge these differences and identify moderators or predictors. Our aim was to generate optimal search strategies (commonly known as filters) for PubMed to retrieve papers identifying moderators and predictors of treatment effects. METHODS: Six journals were hand-searched for articles on moderators or predictors. Selected articles were randomly allocated to a development and validation set. Search terms were extracted from the development set and tested for their performance. Search filters were created from combinations of these terms and tested in the validation set. RESULTS: Of 4407 articles, 198 were considered to be relevant. The most sensitive filter in the development set '("Epidemiologic Methods" [MeSH] OR assign* OR control*[tiab] OR trial*[tiab]) AND therapy*[sh]' yielded in the validation set a sensitivity of 89% [88%-90%] and a specificity of 80% [79%-82%]. CONCLUSIONS: The search filters created in this study can help to efficiently retrieve evidence on moderators and predictors of treatment effect. Testing of the filters in multiple domains should reveal robustness across disciplines. These filters can facilitate the retrieval of evidence on moderators and predictors of treatment effects, helping the implementation of stratified or personalised health care.

Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes.

Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, health technology assessment agencies (HTAs) presently employ value assessment frameworks that are ill fitted to capture the range and diversity of stakeholder values and thereby risk compromising the legitimacy of their recommendations. We propose "evidence-informed deliberative processes" as an alternative framework with the aim to enhance this legitimacy. This framework integrates two increasingly popular and complementary frameworks for priority setting: multicriteria decision analysis and accountability for reasonableness. Evidence-informed deliberative processes are, on one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making through evidence-informed evaluation of the identified values. The framework has important implications for how HTA agencies should ideally organize their processes. First, HTA agencies should take the responsibility of organizing stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phases, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal. The framework should not be considered a blueprint for HTA agencies but rather an aspirational goal-agencies can take incremental steps toward achieving this goal.

The development of CHAMP: a checklist for the appraisal of moderators and predictors.

BACKGROUND: Personalized healthcare relies on the identification of factors explaining why individuals respond differently to the same intervention. Analyses identifying such factors, so called predictors and moderators, have their own set of assumptions and limitations which, when violated, can result in misleading claims, and incorrect actions. The aim of this study was to develop a checklist for critically appraising the results of predictor and moderator analyses by combining recommendations from published guidelines and experts in the field. METHODS: Candidate criteria for the checklist were retrieved through systematic searches of the literature. These criteria were evaluated for appropriateness using a Delphi procedure. Two Delphi rounds yielded a pilot checklist, which was tested on a set of papers included in a systematic review on reinforced home-based palliative care. The results of the pilot informed a third Delphi round, which served to finalize the checklist. RESULTS: Forty-nine appraisal criteria were identified in the literature. Feedback was obtained from fourteen experts from (bio)statistics, epidemiology and other associated fields elicited via three Delphi rounds. Additional feedback from other researchers was collected in a pilot test. The final version of our checklist included seventeen criteria, covering the design (e.g. a priori plausibility), analysis (e.g. use of interaction tests) and results (e.g. complete reporting) of moderator and predictor analysis, together with the transferability of the results (e.g. clinical importance). There are criteria both for individual papers and for bodies of evidence. CONCLUSIONS: The proposed checklist can be used for critical appraisal of reported moderator and predictor effects, as assessed in randomized or non-randomized studies using individual participant or aggregate data. This checklist is accompanied by a user's guide to facilitate implementation. Its future use across a wide variety of research domains and study types will provide insights about its usability and feasibility.

Lay and professional stakeholder involvement in scoping palliative care issues: Methods used in seven European countries.

BACKGROUND: Stakeholders are people with an interest in a topic. Internationally, stakeholder involvement in palliative care research and health technology assessment requires development. Stakeholder involvement adds value throughout research (from prioritising topics to disseminating findings). Philosophies and understandings about the best ways to involve stakeholders in research differ internationally. Stakeholder involvement took place in seven countries (England, Germany, Italy, Lithuania, the Netherlands, Norway and Poland). Findings informed a project that developed concepts and methods for health technology assessment and applied these to evaluate models of palliative care service delivery. AIMS: To report on stakeholder involvement in the INTEGRATE-HTA project and how issues identified informed project development. DESIGN: Using stakeholder consultation or a qualitative research design, as appropriate locally, stakeholders in seven countries acted as 'advisors' to aid researchers' decision making. Thematic analysis was used to identify key issues across countries. SETTING/PARTICIPANTS: A total of 132 stakeholders (82 professionals and 50 'lay' people) aged ⩾18 participated in individual face-to-face or telephone interviews, consultation meetings or focus groups. RESULTS: Different stakeholder involvement methods were used successfully to identify key issues in palliative care. A total of 23 issues common to three or more countries informed decisions about the intervention and comparator of interest, sub questions and specific assessments within the health technology assessment. CONCLUSION: Stakeholders, including patients and families undergoing palliative care, can inform project decision making using various involvement methods according to the local context. Researchers should consider local understandings about stakeholder involvement as views of appropriate and feasible methods vary. Methods for stakeholder involvement, especially consultation, need further development.

TAKING PATIENT HETEROGENEITY AND PREFERENCES INTO ACCOUNT IN HEALTH TECHNOLOGY ASSESSMENTS.

OBJECTIVES: The INTEGRATE-HTA project provided methodology to evaluate complex technologies. This study provides guidance on how to retrieve and critically appraise available evidence on moderators and predictors of treatment effects and on patient preferences for treatment outcomes as a source of complexity. METHODS: Search filters for PubMed were developed by hand-searching a large volume of articles reporting on relevant aspects. Search terms were retrieved from selected papers and algorithmically combined to find the optimal combination of search terms. For the development of the appraisal checklists literature was searched in PubMed and Google Scholar together with citation chasing. For the CHecklist for the Appraisal of Moderators and Predictors (CHAMP) a Delphi procedure was used to value a set of eligible appraisal criteria retrieved from the literature. RESULTS: Search filters were developed optimized for different accuracy measures. The final version of CHAMP consists of a seventeen questions covering the design, analysis, results and transferability of results of moderator and predictor analysis. The final checklist for appraisal of literature on patient preferences for treatment outcomes consist of six questions meant to help the user to identify relevant quality issues together with a guidance toward existing tools concerning the appraisal of specific preference elicitation methods. CONCLUSIONS: Incorporating knowledge on subgroups for whom a specific treatment will produce more benefit holds the promise of better targeting and, ultimately, enhancing overall effectiveness and efficiency of healthcare technology. Finally, incorporating information on preferences for treatment outcomes will foster health technology assessment that addresses outcomes that are important to patients.

STAKEHOLDER INVOLVEMENT THROUGHOUT HEALTH TECHNOLOGY ASSESSMENT: AN EXAMPLE FROM PALLIATIVE CARE.

OBJECTIVES: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA. METHODS: A 5-step "INTEGRATE-HTA Model" developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face-face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England. RESULTS: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use. CONCLUSION: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed.

The development of PubMed search strategies for patient preferences for treatment outcomes.

BACKGROUND: The importance of respecting patients' preferences when making treatment decisions is increasingly recognized. Efficiently retrieving papers from the scientific literature reporting on the presence and nature of such preferences can help to achieve this goal. The objective of this study was to create a search filter for PubMed to help retrieve evidence on patient preferences for treatment outcomes. METHODS: A total of 27 journals were hand-searched for articles on patient preferences for treatment outcomes published in 2011. Selected articles served as a reference set. To develop optimal search strategies to retrieve this set, all articles in the reference set were randomly split into a development and a validation set. MeSH-terms and keywords retrieved using PubReMiner were tested individually and as combinations in PubMed and evaluated for retrieval performance (e.g. sensitivity (Se) and specificity (Sp)). RESULTS: Of 8238 articles, 22 were considered to report empirical evidence on patient preferences for specific treatment outcomes. The best search filters reached Se of 100 % [95 % CI 100-100] with Sp of 95 % [94-95 %] and Sp of 97 % [97-98 %] with 75 % Se [74-76 %]. In the validation set these queries reached values of Se of 90 % [89-91 %] with Sp 94 % [93-95 %] and Se of 80 % [79-81 %] with Sp of 97 % [96-96 %], respectively. CONCLUSIONS: Narrow and broad search queries were developed which can help in retrieving literature on patient preferences for treatment outcomes. Identifying such evidence may in turn enhance the incorporation of patient preferences in clinical decision making and health technology assessment.

Surgical removal versus retention for the management of asymptomatic disease-free impacted wisdom teeth.

BACKGROUND: Prophylactic removal of asymptomatic disease-free impacted wisdom teeth is surgical removal of wisdom teeth in the absence of symptoms and with no evidence of local disease. Impacted wisdom teeth may be associated with pathological changes, such as pericoronitis, root resorption, gum and alveolar bone disease (periodontitis), caries and the development of cysts and tumours. When surgical removal is carried out in older people, the risk of postoperative complications, pain and discomfort is increased. Other reasons to justify prophylactic removal of asymptomatic disease-free impacted third molars have included preventing late lower incisor crowding, preventing damage to adjacent structures such as the second molar or the inferior alveolar nerve, in preparation for orthognathic surgery, in preparation for radiotherapy or during procedures to treat people with trauma to the affected area. Removal of asymptomatic disease-free wisdom teeth is a common procedure, and researchers must determine whether evidence supports this practice. This review is an update of an existing review published in 2012. OBJECTIVES: To evaluate the effects of removal compared with retention (conservative management) of asymptomatic disease-free impacted wisdom teeth in adolescents and adults. SEARCH METHODS: We searched the following electronic databases: Cochrane Oral Health's Trials Register (to 24 May 2016), the Cochrane Central Register of Controlled Trials (CENTRAL) (2016, Issue 4), MEDLINE Ovid (1946 to 24 May 2016) and Embase Ovid (1980 to 24 May 2016). We searched ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform for ongoing and unpublished studies to 24 May 2016. We imposed no restrictions on language or date of publication in our search of electronic databases. SELECTION CRITERIA: Studies comparing removal (or absence) with retention (or presence) of asymptomatic disease-free impacted wisdom teeth in adolescents or adults. We included randomised controlled trials (RCTs) with no restriction on length of follow-up, if available. We considered quasi-RCTs and prospective cohort studies for inclusion if investigators measured outcomes with follow-up of five years or longer. DATA COLLECTION AND ANALYSIS: Eight review authors screened search results and assessed the eligibility of studies for inclusion according to the review inclusion criteria. Eight review authors independently conducted risk of bias assessments in duplicate. When information was unclear, we contacted study authors for additional information. MAIN RESULTS: This review includes two studies. The previous review included one RCT with a parallel-group design, which was conducted in a dental hospital setting in the United Kingdom; our new search for this update identified one prospective cohort study conducted in the private sector in the USA. Primary outcome No eligible studies in this review reported the effects of removal compared with retention of asymptomatic disease-free impacted wisdom teeth on health-related quality of life Secondary outcomes We found only low to very low quality evidence of the effects of removal compared with retention of asymptomatic disease-free impacted wisdom teeth for a limited number of secondary outcome measures.One prospective cohort study, reporting data from a subgroup of 416 healthy male participants, aged 24 to 84 years, compared the effect of the absence (previous removal or agenesis) against the presence of asymptomatic disease-free impacted wisdom teeth on periodontitis and caries associated with the distal of the adjacent second molar during a follow-up period of three to over 25 years. Very low quality evidence suggests that the presence of asymptomatic disease-free impacted wisdom teeth may be associated with increased risk of periodontitis affecting the adjacent second molar in the long term. In the same study, which is at serious risk of bias, there is insufficient evidence to demonstrate a difference in caries risk associated with the presence or absence of impacted wisdom teeth.One RCT with 164 randomised and 77 analysed adolescent participants compared the effect of extraction with retention of asymptomatic disease-free impacted wisdom teeth on dimensional changes in the dental arch after five years. Participants (55% female) had previously undergone orthodontic treatment and had 'crowded' wisdom teeth. No evidence from this study, which was at high risk of bias, was found to suggest that removal of asymptomatic disease-free impacted wisdom teeth has a clinically significant effect on dimensional changes in the dental arch.The included studies did not measure our other secondary outcomes: costs, other adverse events associated with retention of asymptomatic disease-free impacted wisdom teeth (pericoronitis, root resorption, cyst formation, tumour formation, inflammation/infection) and adverse effects associated with their removal (alveolar osteitis/postoperative infection, nerve injury, damage to adjacent teeth during surgery, bleeding, osteonecrosis related to medication/radiotherapy, inflammation/infection). AUTHORS' CONCLUSIONS: Insufficient evidence is available to determine whether or not asymptomatic disease-free impacted wisdom teeth should be removed. Although asymptomatic disease-free impacted wisdom teeth may be associated with increased risk of periodontitis affecting adjacent second molars in the long term, the evidence is of very low quality. Well-designed RCTs investigating long-term and rare effects of retention and removal of asymptomatic disease-free impacted wisdom teeth, in a representative group of individuals, are unlikely to be feasible. In their continuing absence, high quality, long-term prospective cohort studies may provide valuable evidence in the future. Given the lack of available evidence, patient values should be considered and clinical expertise used to guide shared decision making with patients who have asymptomatic disease-free impacted wisdom teeth. If the decision is made to retain asymptomatic disease-free impacted wisdom teeth, clinical assessment at regular intervals to prevent undesirable outcomes is advisable.

Implementing evidence-based practice for patients with chronic fatigue syndrome.

The aim of our study was to explore whether community-based mental health care centres (MHCs) are able to implement and sustain cognitive behaviour therapy (CBT) for chronic fatigue syndrome (CFS) with the help of an implementation manual. We monitored the implementation process and treatment outcome data of three Dutch MHCs that implemented or sustained CBT for CFS, one in the context of a stepped care programme. We compared these data with findings of other treatment studies conducted in the context of CBT for CFS. All three MHCs included at least 40 patients with dropout rates between 15% and 35% from intention-to-treat to second assessment. Effect sizes ranged between 0.88 and 1.76 for changes in fatigue severity and 0.43 and 1.23 for changes in physical functioning. With one exception, these outcomes were within the range of our benchmark. Contrary to original expectations, we provided additional implementation support to the two MHCs new with CBT for CFS. We concluded that our implementation manual does not seem to substitute external support for team leaders and associated professions during initial implementation of CBT for CFS but may have the potential to make this assistance more efficient. Particular attention should be paid to challenges of implementing stepped care for CFS. KEY PRACTITIONERS MESSAGE: Implementation of CBT for CFS in community-based MHCs was monitored. External support was provided in addition to an implementation manual during initial implementation of CBT for CFS. Participating MHCs were generally capable of successfully implementing and delivering CBT for CFS. Implementation of low-intensity interventions for CFS might better be postponed until therapists have sufficient experience with conventional CBT for CFS.

An Overview of Stakeholders, Methods, Topics, and Challenges in Participatory Approaches Used in the Development of Medical Devices: A Scoping Review.

BACKGROUND: There is a wide variety of participatory approaches to involve stakeholders in the development of medical devices, but there is no comprehensive overview of these approaches. We therefore studied what participatory approaches are used in the development of medical devices as well as the most important characteristics and challenges of these approaches. METHODS: We conducted a scoping review and searched PubMed, Embase and Web of Science for articles published between July 2014 and July 2019. Papers were included if they presented original research featuring any form of stakeholder participation in the development of medical devices. We used The Spectrum of Public Participation to categorise the approach of each paper. Subsequently, we described the characteristics of each approach: the stakeholders involved, data collection methods, and topics addressed. We also identified challenges of the approaches as reported by researchers. RESULTS: 277 papers were included, which could be categorised into three levels of participation: collaboration, involvement, and consultation. Patients and healthcare professionals are frequently engaged in all approaches. The most often used methods are workshops in the collaboration approach papers, and interviews in the involvement and consultation approach papers. Topics addressed in all approaches are: the problem, device requirements, design choices, testing, and procedural aspects of involvement. Reported challenges entail issues related to sampling, analysis, social dynamics, feasibility, and the limited number of topics that can be addressed. CONCLUSION: Participatory approaches reported in literature can be categorised in three overarching approaches that have comparable methodological characteristics. This suggests that if researchers want to apply a participatory approach it is not necessary to adopt a pre-determined approach, such as 'participatory action research' (PAR). Instead, they can independently determine the degree of participation, stakeholders, methods, topics, and strategies to account for challenges, making sure the participatory approach fits their research question and context.

Impact of time-to-treatment on survival for early-stage non-small cell lung cancer in The Netherlands-a nationwide observational cohort study.

Background: Varied outcomes on the relation between time-to-treatment and survival in early-stage non-small cell lung cancer (NSCLC) patients are reported. We examined this relation in a large multicentric retrospective cohort study and identified factors associated with extended time-to-treatment. Methods: We included 9,536 patients with clinical stage I-II NSCLC, diagnosed and treated in 2014-2019, from the Netherlands Cancer Registry that includes nation-wide data. Time-to-treatment was defined as the number of days between first outpatient visit for suspected lung cancer and start of treatment. The effect of extended time-to-treatment beyond the first quartile and survival was studied with Cox proportional hazard regression. Analyses were stratified for stage and type of therapy. Time-to-treatment was adjusted for multiple covariates including performance status and socioeconomic status. Factors associated with treatment delay were identified by multilevel logistic regression. Results: Median time-to-treatment was 47 days [interquartile range (IQR): 34-65] for stage I and 46 days (IQR: 34-62) for stage II. The first quartile extended to 33 days for both stages. Risk of death increased significantly with extended time-to-treatment for surgical treatment of clinical stage II patients [adjusted hazard ratio (aHR) >33 days: 1.36, 95% confidence intervals (CI): 1.09-1.70], but not in stage II patients treated with radiotherapy or in stage I patients. Causes of prolonged time-to-treatment were multifactorial including diagnostic tests, such as endoscopic ultrasound (EUS) or endobronchial ultrasound (EBUS). Conclusions: Clinical stage II patients benefit from fast initiation of surgical treatment. Surprisingly this appears to be accounted for by patients who are clinically stage II but pathologically stage I. Further study is needed on characterizing these patients and the significance of lymph node- or distant micrometastasis in guiding time-to-treatment and treatment strategy.