How much is enough? Proposing achievement thresholds for core EPAs of graduating medical students in Canada.

PURPOSE: The transition towards Competency-Based Medical Education at the Cumming School of Medicine was accelerated by the reduced clinical time caused by the COVID-19 pandemic. The purpose of this study was to define a standard protocol for setting Entrustable Professional Activity (EPA) achievement thresholds and examine their feasibility within the clinical clerkship. METHODS: Achievement thresholds for each of the 12 AFMC EPAs for graduating Canadian medical students were set by using sequential rounds of revision by three consecutive groups of stakeholders and evaluation experts. Structured communication was guided by a modified Delphi technique. The feasibility/consequence models of these EPAs were then assessed by tracking their completion by the graduating class of 2021. RESULTS: The threshold-setting process resulted in set EPA achievement levels ranging from 1 to 8 across the 12 AFMC EPAs. Estimates were stable after the first round for 9 of 12 EPAs. 96.27% of EPAs were successfully completed by clerkship students despite the shortened clinical period. Feasibility was predicted by the slowing rate of EPA accumulation overtime during the clerkship. CONCLUSION: The process described led to consensus on EPA achievement thresholds. Successful completion of the assigned thresholds was feasible within the shortened clerkship.[Box: see text].

Development of an International Standard Set of Value-Based Outcome Measures for Patients With Chronic Kidney Disease: A Report of the International Consortium for Health Outcomes Measurement (ICHOM) CKD Working Group.

Value-based health care is increasingly promoted as a strategy for improving care quality by benchmarking outcomes that matter to patients relative to the cost of obtaining those outcomes. To support the shift toward value-based health care in chronic kidney disease (CKD), the International Consortium for Health Outcomes Measurement (ICHOM) assembled an international working group of health professionals and patient representatives to develop a standardized minimum set of patient-centered outcomes targeted for clinical use. The considered outcomes and patient-reported outcome measures were generated from systematic literature reviews. Feedback was sought from patients and health professionals. Patients with very high-risk CKD (stages G3a/A3 and G3b/A2-G5, including dialysis, kidney transplantation, and conservative care) were selected as the target population. Using an online modified Delphi process, outcomes important to all patients were selected, such as survival and hospitalization, and to treatment-specific subgroups, such as vascular access survival and kidney allograft survival. Patient-reported outcome measures were included to capture domains of health-related quality of life, which were rated as the most important outcomes by patients. Demographic and clinical variables were identified to be used as case-mix adjusters. Use of these consensus recommendations could enable institutions to monitor, compare, and improve the quality of their CKD care.

Consensus Guidelines of Irreversible Electroporation for Pancreatic Tumors: Protocol Standardization Using the Modified Delphi Technique.

Since no uniform treatment protocol for pancreatic irreversible electroporation (IRE) exists, the heterogeneity throughout literature complicates the comparison of results. To reach agreement among experts, a consensus study was performed. Eleven experts, recruited according to predefined criteria regarding previous IRE publications, participated anonymously in three rounds of questionnaires according to a modified Delphi technique. Consensus was defined as having reached ≥80% agreement. Response rates were 100, 64, and 64% in rounds 1 to 3, respectively; consensus was reached in 93%. Pancreatic IRE should be considered for stage III pancreatic cancer and inoperable recurrent disease after previous local treatment. Absolute contraindications are ventricular arrhythmias, implantable stimulation devices, congestive heart failure NYHA class 4, and severe ascites. The inter-electrode distance should be 10 to 20 mm and the exposure length should be 15 mm. After 10 test pulses, 90 treatment pulses of 1,500 V/cm should be delivered continuously, with a 90-µs pulse length. The first postprocedural contrast-enhanced computed tomography should take place 1 month post-IRE, and then every 3 months. This article provides expert recommendations regarding patient selection, procedure, and follow-up for IRE treatment in pancreatic malignancies through a modified Delphi consensus study. Future studies should define the maximum tumor diameter, response evaluation criteria, and the optimal number of preoperative FOLFIRINOX cycles.

Assessing primary health care provider and organization readiness to address family violence in Alberta, Canada: development of a Delphi consensus readiness tool.

BACKGROUND: Family violence, which includes intimate partner abuse, child abuse, and elder abuse, is a serious public health concern. Primary healthcare (PHC) offers a vital opportunity to identify and address family violence, yet barriers prevent the effective implementation of family violence interventions in PHC settings. The purpose of this study is to improve family violence identification and response in Alberta's PHC settings by exploring readiness factors. METHODS: An integrated knowledge translation approach, combining implementation science and participatory action research, was employed to develop a readiness assessment tool for addressing family violence within PHC settings in Alberta. The research involved three phases: phase 1 involved a rapid evidence assessment, phase 2 engaged a panel of healthcare and family violence experts to explore readiness components in the Alberta context, and phase 3 utilized a 3-round Delphi consensus-building process to refine readiness indicators. RESULTS: Phase 1 findings from a rapid evidence assessment highlighted five main models/tools for assessing readiness to implement family violence interventions in PHC settings. In phase 2, additional concepts were identified through exploration with healthcare and family violence expert panel members, resulting in a total of 16 concepts for assessing family violence readiness within the Alberta PHC context. The 3-round Delphi consensus-building process in Phase 3 involved nine panelists, who collectively agreed on the inclusion of all concepts and indicators, yielding a total of 60 items for the proposed readiness assessment tool for addressing family violence in PHC within Alberta. CONCLUSION: The current study lays the groundwork for future family violence intervention programs, offering insights into key components that promote readiness for implementing comprehensive programs and supporting PHC organizations in effectively addressing family violence.

Standards for Scaffolding in Health Sciences Programmes: A Delphi Consensus Study.

Introduction: Health sciences programmes operate in complex, unpredictable contexts, underscoring the need for comprehensive scaffolding of the learning processes. Yet, the scaffolding approaches remain fragmented, and lack a shared approach to how programmes could integrate scaffolding across the curricula. The literature argues that standards result in the comprehensive implementation of educational practices. There are no reported standards related to scaffolding practices in these programmes. OBJECTIVES: To develop standards for scaffolding in health sciences programmes utilising a consensus approach through a modified Delphi Technique. METHODS: Following the recommendations on Conducting and REporting of DElphi Studies (CREDES), an online modified Delphi technique was applied. Evidence on the application of scaffolding in health sciences programmes, obtained through an integrative review, was synthesised to draft standards. Using purposive and snowball sampling, an international panel from diverse geographical and professional backgrounds refined and validated the standards. Descriptive statistics was utilised to analyse demographic data and consensus agreements to include standards and criteria. Qualitative analysis of textual comments ensured the synthesis and inclusion of critical divergent views and additions. RESULTS: A total of 22 experts from around the globe agreed to participate in the study and one did not complete Delphi surveys. Most experts (n = 18) held a PhD; and an average of 19 years of teaching in health sciences programmes. Four standards and 27 criteria were included after achieving consensus during the two Delphi surveys rounds. The included standards focused on four areas: structuring and sequencing educational activities, resources/tools for scaffolding, structuring the programme and instructional strategies to support learning. CONCLUSION: The principle-based standards developed in this study could direct and support scaffolding practices in health sciences programmes. The standards' emphases on macro-, meso- and micro-scaffolding present numerous opportunities for designing and applying contextually sensitive scaffolding strategies at every level of curriculum implementation.

Comparison of the Structured Consent Process Using Modified Delphi Technique with the Standard Process in Obtaining Informed Consent for Procedures in ENT by PHASE III MBBS Students.

Communication skill is a core competency and the training must begin in the undergraduate period itself. The Phase III MBBS students during their ENT posting are required to obtain informed consent for procedures and surgeries in ENT which forms the basis for efficient communication skills in house-surgency (internship) and residency. Informed consent taking is an important aspect and in the clinical postings, the teaching of communication skills along with history taking and physical examination can go a long way in making a strong foundation to good doctor patient relationships. This study aimed to compare the structured consent process using modified Delphi technique with the standard process in obtaining informed consent for procedures in ENT by PHASE III MBBS students. The need to sensitise the MBBS students on appropriate consent taking procedures with familiarisation of the essential elements of the Kalamazoo consensus statement were raised, accepted and final OSCE assessment attributes decided by the modified Delphi technique. The Modified Delphi technique is a unique means to obtain opinions of experts across the field in various spheres so as to identify lacunae if any in the existing teaching with means to reach a valid and reliable consensus. Our study included Phase III MBBS students posted to the Department of ENT, Govt Medical College Kozhikode during Jan-Feb 2022 wherein one batch of 30 students were taught with 2 classes on informed consent taking by the structured process obtained after Modified Delphi technique and included as; "MD" group and another batch of 30 students from the entire batch taught by the standard process was included as; "T" group respectively. After completion of the clinical postings an assessment was carried out with OSCE stations in Mar 2022; wherein 10 students were evaluated for each of the 6 common ENT procedures, 5 students from "T" group and 5 students from "MD" group respectively. Median total score of MD group was 6.5 (3.25-8) and median score of T group was 4.5 (2.25-6.75). The difference in mean ranks of these scores was statistically significant, p < 0.0001. The feedback assessment using the questionnaire with Likert scale had all 30(100%) students recommend this method of structured consent taking for enhancement of communications skills. However 20% of the students were not satisfied with the teaching learning method expressing the need for more time allocation and demonstrations. Informed consent taking requires the appropriate training in the undergraduate period itself as seen by the improved OSCE scores on assessment after teaching by the structured consent taking process as well as from the feedback of the students.

Chemotherapy administration checklist for patients receiving chemotherapy: Development and validation.

BACKGROUND: With the increase in the number of new cancer cases, the number of patients receiving chemotherapy increases. The administration of chemotherapy is a very significant task. Unfortunately, the availability of a chemotherapy administration checklist still lags behind. The aim is to develop a chemotherapy administration checklist for patients receiving chemotherapy. MATERIALS AND METHODS: This study was conducted at the oncology day care and in-patient department (IPD) wards of the All India Institute of Medical Sciences, Jodhpur, Rajasthan. Methodological research design was used in this study. It took place in five phases: preliminary phase includes literature review, focussed group discussion, assessment of current practices, generation of item pool, and preparation of preliminary draft; then its validation by modified Delphi technique, pilot testing; final try out, and evaluation phase. A total of 260 patients were enrolled. RESULTS: The content validity index was 0.97. To perform factor analysis and principal component analysis KMO and Bartlett's test of sphericity was used, which allows the data for factor analysis to yield eight components. A total of 26 items were formulated. The scoring was done dichotomously; a score of 1 is for done and 0 for not done and not applicable. Cronbach's alpha was used to find out the internal consistency of the checklist was found to be 0.72. For interrater reliability, the Cohen kappa's value was found to be 0.91. CONCLUSIONS: Chemotherapy administration checklist was a valid and reliable checklist. This checklist is feasible and easy to incorporate into clinical practices.

Classification of Patients for Whom Benefit of Long-term Opioid Therapy No Longer Outweighs Harm: Protocol for a Delphi Study.

BACKGROUND: Patients with chronic pain prescribed long-term opioid therapy may come to a point where the benefits of the therapy are outweighed by the risks and tapering is indicated. At the 2019 Veterans Health Administration State of the Art Conference, there was an acknowledgment of a lack of clinical guidance with regard to treating this subset of patients. Some of the participants believed clinicians and patients would both benefit from a new diagnostic entity describing this situation. OBJECTIVE: The aim of this study was to determine if a new diagnostic entity was needed and what the criteria of the diagnostic entity would be. Given the ability of the Delphi method to synthesize input from a broad range of experts, we felt this technique was the most appropriate for this study. METHODS: We designed a modified Delphi technique involving 3 rounds. The first round is a series of open-ended questions asking about the necessity of this diagnostic entity, how this condition is different from opioid use disorder, and what its possible diagnostic criteria would be. After synthesizing the responses collected, a second round will be conducted to ask participants to rate the different responses offered by their peers. These ratings will be collected and analyzed, and will generate a preliminary definition for this clinical phenomena. In the third round, we will circulate this definition with the aim of achieving consensus. RESULTS: The modified Delphi study was initiated in July of 2020 and analysis is currently underway. CONCLUSIONS: This protocol has been approved by the Internal Review Board at the Connecticut Veterans Affairs and the study is in process. This protocol may assist other researchers conducting similar studies. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/33310.

Operationalising a modified Delphi study to progress quality care process nursing metrics for acute care.

Background: Despite representing the largest occupational group within the healthcare workforce, evidence suggests that due to the complexity of nursing practice, nurses' contribution remains 'invisible'. Quality Care Metrics aligned to standards can offer valuable numerical information that quantify input, output and dimensions of nursing care processes in complex clinical and interprofessional milieus. Aims and objectives: Progress an evidence-based metric system to measure the quality and clinical safety of nursing care within acute care in Ireland. The objectives were to: classify quality care process nursing metrics and corresponding indicators pertinent to acute care; reach agreement on a selected set of robust metrics and corresponding indicators; and implement the findings of the study. Design: A modified four-round Delphi study. Methods: The modified Delphi study integrated a four-round survey of 422 nurses, face-to-face meetings with a patient representative and key stakeholders within acute services with a final consensus meeting inclusive of a panel of 26 expert nurse clinicians. Results: There was consensus on 11 quality care process nursing metrics and 53 corresponding indicators for the acute care setting. Despite the rating of 'critical' in the Delphi rounds, a concern was reported by participants on the subjective nature of three of the developed metrics: 'patient experience', 'patient engagement' and 'professional and ethical approach to care' based on the absence of objective measurement tools that include patient input. Conversely, this led to the conundrum for the panel of experts at the final consensus meeting who were divided in their views on objectively observing, recording and subsequent auditing of those three developed metrics in real-time clinical practice. Conclusion: This paper describes the operationalisation of a modified Delphi technique that progressed a set of 11 quality care process metrics and 53 corresponding indicators. The challenge now is the implementation of these quality care process metrics so that nurses' contribution to patient-centred care is tangible in acute care.

Nursing-sensitive quality indicators for pernicious placenta previa in obstetrics: A Delphi study based across Chinese institutions.

AIM: To identify a set of scientific, systematic and clinically applicable nursing-sensitive quality indicators for pernicious placenta previa (PPP). DESIGN: A modified Delphi-Consensus Technique. METHODS: According to literature retrieval published between 2009-2019, 38 nursing-sensitive quality indicators were chosen and a questionnaire was designed. An online survey was conducted in 20 hospitals in China, and data of experts' opinions were collected and analysed by improved Delphi method. RESULTS: 38 nursing-sensitive quality indicators were identified. The response rates in the two rounds of expert consultation were 97.4% and 100%, and the authoritative coefficients were 0.89 and 0.92. The coefficients of variation ranged from 0.04-0.28. The nursing-sensitive quality indicators were successfully constructed based on the modified Delphi technique.

Developing a tool for measuring the disaster resilience of healthcare rescuers: a modified Delphi study.

BACKGROUND: Disaster resilience is an essential personal characteristics of health rescue workers to respond to disasters in an effective manner, and maintain a state of adaptation after deployment. It is essential for disaster managers to recruit, assess, and prepare healthcare rescuers with this characteristic. A specific tool for measuring the disaster resilience of healthcare rescuers has yet to be devised. OBJECTIVE: The purpose of this study was to establish the content validity of a tool for measuring the disaster resilience of disaster rescue workers. METHODS: A modified Delphi approach was employed. Experts in disaster work and research were invited to rate the domains and items of a prototype tool for measuring disaster resilience in healthcare rescue workers. The panel of experts rated the relevance of the items using a 4-point Likert scale. The median and interquartile range, as well as the level of agreement, were calculated for each item using the Kendall coefficient W, to assess the consensus of the experts. The content validity index (CVI) was calculated to assess the content validity of this tool. RESULTS: A total of 22 and 21 experts were involved in the first and second rounds of this modified Delphi study (response rate of 91.7 and 95.5%), respectively. After two rounds of expert query, an eight-domain and 27-item disaster resilience measuring tool was established. The median range of all of the included items was 3.50 to 4.00 and the interquartile range was 0.00 to 1.00, and all items achieved ≥85% agreement. The Kendall coordination coefficient W was 0.21 and 0.33 in the first and second rounds, respectively, with P < 0.01. The I-CVI ranged from 0.85 to 1.0, while the S-CVI/UA and S-CVI /Ave were 0.69 and 0.97, respectively. CONCLUSION: Consensus was reached on a disaster resilience measuring tool covering 27 items. The content validity of this tool for measuring the disaster resilience of healthcare rescuers was excellent. This tool is validated and ready to be tested in a pilot study to assess its psychometric properties.

Homeopathic clinical case reports: Development of a supplement (HOM-CASE) to the CARE clinical case reporting guideline.

OBJECTIVE: Develop a criteria catalog serving as a guideline for authors to improve the quality of reporting clinical case reports in homeopathy. METHOD: An online Delphi process was initiated with a panel of 19 homeopathic experts from Europe, the USA and India. Homeopathy specific item selection took place in three rounds of adjusting. The selected items can be used as an extension of the CARE clinical case reporting guideline. RESULTS: Eight homeopathy specific 'core' items were selected from a list of 31 suggested items; (1) the clinical history from a homeopathic perspective; (2) the type of homeopathy; detailed description of the medication--(3) nomenclature, (4) manufacture, (5) galenic form+dosage; outcomes--(6) objective evidence if available, (7) occurrence homeopathic aggravation, (8) assessment possible causal attribution of changes to the homeopathic treatment. A further 4 items were recommended for consideration as optional items when case reports are used for specific, in particular educational, purposes. The 8 core items can be used, merged into 6 items, as a homeopathy specific (HOM-CASE) extension to the CARE clinical case reporting guideline items 6, 9 and 10. CONCLUSION: Use of the HOM-CASE guideline extension will contribute to transparent and accurate reporting and can significantly improve the quality and reliability of clinical case reports in homeopathy.

Expert validation of a teamwork assessment rubric: A modified Delphi study.

BACKGROUND: Teamwork is a 'soft skill' employability competence desired by employers. Poor teamwork skills in healthcare have an impact on adverse outcomes. Teamwork skills are rarely the focus of teaching and assessment in undergraduate courses. The TeamUP Rubric is a tool used to teach and evaluate undergraduate students' teamwork skills. Students also use the rubric to give anonymised peer feedback during team-based academic assignments. The rubric's five domains focus on planning, environment, facilitation, conflict management and individual contribution; each domain is grounded in relevant theory. Students earn marks for their teamwork skills; validity of the assessment rubric is critical. QUESTION: To what extent do experts agree that the TeamUP Rubric is a valid assessment of 'teamwork skills'? DESIGN: Modified Delphi technique incorporating Feminist Collaborative Conversations. PARTICIPANTS: A heterogeneous panel of 35 professionals with recognised expertise in communications and/or teamwork. METHODS: Three Delphi rounds using a survey that included the rubric were conducted either face-to-face, by telephone or online. Quantitative analysis yielded item content validity indices (I-CVI); minimum consensus was pre-set at 70%. An average of the I-CVI also yielded sub-scale (domain) (D-CVI/Ave) and scale content validity indices (S-CVI/Ave). After each Delphi round, qualitative data were analysed and interpreted; Feminist Collaborative Conversations by the research team aimed to clarify and confirm consensus about the wording of items on the rubric. RESULTS: Consensus (at 70%) was obtained for all but one behavioural descriptor of the rubric. We modified that descriptor to address expert concerns. The TeamUP Rubric (Version 4) can be considered to be well validated at that level of consensus. The final rubric reflects underpinning theory, with no areas of conceptual overlap between rubric domains. CONCLUSION: The final TeamUP Rubric arising from this study validly measures individual student teamwork skills and can be used with confidence in the university setting.

What is agenda setting in the clinical encounter? Consensus from literature review and expert consultation.

OBJECTIVE: To establish consensus on the core domains of agenda setting in consultations. METHODS: We reviewed the healthcare literature and, using a modified Delphi technique to embrace both patient and clinician perspectives, conducted an iterative online survey, with 30 experts in health communication. Participants described agenda setting and rated the importance of proposed domains. Consensus was determined where the group median was ≥5 on a 7-point Likert-like response scale, and the interquartile range fell to within one point on this scale. RESULTS: Relevant publications were identified in three overlapping bodies of healthcare literature. Survey respondents considered that agenda setting involved a process whereby patients and clinicians establish a joint focus for both their conversation and their working relationship. Consensus was obtained on six core domains: identifying patient talk topics, identifying clinician talk topics, agreement of shared priorities, establishing conversational focus, collaboration and engagement. New terminology--agenda mapping and agenda navigation--is proposed. CONCLUSION: We identified core agenda setting domains that embraced patient and clinician perspectives. PRACTICE IMPLICATIONS: An integrated conceptualization of agenda setting may now be used by researchers and educators in both clinician and patient focused interventions.

Developing injury indicators for First Nations and Inuit children and youth in Canada: a modified Delphi approach.

INTRODUCTION: The purpose of this research was to take the initial step in developing valid indicators that reflect the injury issues facing First Nations and Inuit children and youth in Canada. METHODS: Using a modified-Delphi process, relevant expert and community stakeholders rated each indicator on its perceived usefulness and ability to prompt action to reduce injury among children and youth in indigenous communities. The Delphi process included 5 phases and resulted in a refined set of 27 indicators. RESULTS: Indicators related to motorized vehicle collisions, mortality and hospitalization rates were rated the most useful and most likely to prompt action. These were followed by indicators for community injury prevention training and response systems, violent and inflicted injury, burns and falls, and suicide. CONCLUSION: The results suggest that a broad-based modified-Delphi process is a practical and appropriate method, within the OCAP™ (Ownership, Control, Access and Possession) principles, for developing a proposed set of indicators for injury prevention activity focused on First Nations and Inuit children and youth. Following additional work to validate and populate the indicators, it is anticipated that communities will utilize them to monitor injury and prompt decisions and action to reduce injuries among children and youth.

TITRE: Élaboration d'indicateurs de blessures chez les enfants et les jeunes des Premières nations et inuits au Canada grâce à une méthode de Delphes modifiée. INTRODUCTION: Le but de cette recherche était de franchir la première étape de l'élaboration d'indicateurs valides et représentatifs des blessures chez les enfants et les jeunes des Premières nations et Inuits du Canada. MÉTHODOLOGIE: À l'aide d'une méthode de Delphes modifiée, des intervenants des collectivités et des experts ont évalué chaque indicateur en fonction de son utilité perçue et de sa capacité à inciter à l'action dans le but de réduire les blessures chez les enfants et les jeunes des collectivités autochtones. Le processus s'est déroulé en 5 étapes et a permis d'obtenir un ensemble de 27 indicateurs. RÉSULTATS: Les indicateurs évalués comme étant les plus utiles et les plus susceptibles d'inciter à l'action ont été ceux liés au taux de collision de véhicules à moteur, au taux de mortalité et au taux d'hospitalisation. Ont été sélectionnés ensuite les indicateurs de la formation en prévention des blessures et programmes communautaires d'intervention, ceux des blessures intentionnelles, ceux des brûlures et des chutes et enfin ceux du suicide. CONCLUSION: Les résultats indiquent qu'utiliser une méthode de Delphes modifiée avec participation élargie est efficace et pertinent pour élaborer un ensemble d'indicateurs visant à guider les activités de prévention des blessures chez les enfants et les jeunes des Premières nations et Inuits, tout en respectant les principes PCAP™ (propriété, contrôle, accès et possession). Une fois achevés les travaux complémentaires de validation des indicateurs et la collecte des données associées, ces indicateurs vont pouvoir servir aux collectivités pour la surveillance des blessures et pour la prise de décisions et de mesures efficaces de réduction des blessures chez les enfants et les jeunes.