Improving health system responses when patients are harmed: a protocol for a multistage mixed-methods study.

INTRODUCTION: At least 10% of hospital admissions in high-income countries, including Australia, are associated with patient safety incidents, which contribute to patient harm ('adverse events'). When a patient is seriously harmed, an investigation or review is undertaken to reduce the risk of further incidents occurring. Despite 20 years of investigations into adverse events in healthcare, few evaluations provide evidence of their quality and effectiveness in reducing preventable harm.This study aims to develop consistent, informed and robust best practice guidance, at state and national levels, that will improve the response, learning and health system improvements arising from adverse events. METHODS AND ANALYSIS: The setting will be healthcare organisations in Australian public health systems in the states of New South Wales, Queensland, Victoria and the Australian Capital Territory. We will apply a multistage mixed-methods research design with evaluation and in-situ feasibility testing. This will include literature reviews (stage 1), an assessment of the quality of 300 adverse event investigation reports from participating hospitals (stage 2), and a policy/procedure document review from participating hospitals (stage 3) as well as focus groups and interviews on perspectives and experiences of investigations with healthcare staff and consumers (stage 4). After triangulating results from stages 1-4, we will then codesign tools and guidance for the conduct of investigations with staff and consumers (stage 5) and conduct feasibility testing on the guidance (stage 6). Participants will include healthcare safety systems policymakers and staff (n=120-255) who commission, undertake or review investigations and consumers (n=20-32) who have been impacted by adverse events. ETHICS AND DISSEMINATION: Ethics approval has been granted by the Northern Sydney Local Health District Human Research Ethics Committee (2023/ETH02007 and 2023/ETH02341).The research findings will be incorporated into best practice guidance, published in international and national journals and disseminated through conferences.

Consumer and provider perceptions of the specialist unit model of care: A qualitative study.

BACKGROUND: Specialist care units cater to targeted cohorts of patients, applying evidence-based practice to people with a specific condition (e.g., dementia) or meeting other specific criteria (e.g., children). This paper aimed to collate perceptions of local consumers and health providers around specialist care units, as a model of care that may be considered for a new local healthcare facility. METHODS: This was a qualitative study using two-hour workshops and interviews to collect data. Participants were consumers and health providers in the planned facility's catchment: 49 suburbs in metropolitan Australia. Consumers and health providers were recruited through advertisements and emails. An initial survey collected demographic details. Consumers and health providers participated in separate two-hour workshops in which a scenario around the specialist unit model was presented and discussion on benefits, barriers and enablers of the model was led by researchers. Detailed notes were taken for analysis. RESULTS: Five consumer workshops (n = 22 participants) and five health provider workshops (n = 42) were conducted. Participants were representative of this culturally diverse region. Factors identified by participants as relevant to the specialist unit model of care included: accessibility; a perceived narrow scope of practice; coordination with other services; resources and infrastructure; and awareness and expectations of the units. Some factors identified as risks or barriers when absent were identified as strengths and enablers when present by both groups of participants. CONCLUSIONS: Positive views of the model centred on the higher perceived quality of care received in the units. Negative views centred on a perceived narrow scope of care and lack of flexibility. Consumers hinted, and providers stated explicitly, that the model needed to be complemented by an integrated model of care model to enable continuity of care and easy transfer of patients into and out of the specialist unit.

Rates of adherence to cancer treatment guidelines in Australia and the factors associated with adherence: A systematic review.

Adherence to cancer treatment clinical practice guidelines (CPGs) varies enormously across Australia, despite being associated with improved patient outcomes. This systematic review aims to characterize adherence rates to active-cancer treatment CPGs in Australia and related factors to inform future implementation strategies. Five databases were systematically searched, abstracts were screened for eligibility, a full-text review and critical appraisal of eligible studies performed, and data extracted. A narrative synthesis of factors associated with adherence was conducted, and the median adherence rates within cancer streams calculated. A total of 21,031 abstracts were identified. After duplicates were removed, abstracts screened, and full texts reviewed, 20 studies focused on adherence to active-cancer treatment CPGs were included. Overall adherence rates ranged from 29% to 100%. Receipt of guideline recommended treatments was higher for patients who were younger (diffuse large B-cell lymphoma [DLBCL], colorectal, lung, and breast cancer); female (breast and lung cancer), and male (DLBCL and colorectal cancer); never smokers (DLBCL and lung cancer); non-Indigenous Australians (cervical and lung cancer); with less advanced stage disease (colorectal, lung, and cervical cancer), without comorbidities (DLBCL, colorectal, and lung cancer); with good-excellent Eastern Cooperative Oncology Group performance status (lung cancer); living in moderately accessible places (colon cancer); and; treated in metropolitan facilities (DLBLC, breast and colon cancer). This review characterized active-cancer treatment CPG adherence rates and associated factors in Australia. Future targeted CPG implementation strategies should account for these factors, to redress unwarranted variation particularly in vulnerable populations, and improve patient outcomes (Prospero number: CRD42020222962).

Cochlear implant aesthetics and its impact on stigma, social interaction and quality of life: a mixed-methods study protocol.

INTRODUCTION: Awareness of the benefits of cochlear implants is low, and barriers such as fear of surgery and ongoing rehabilitation have been noted. Perceived stigma associated with hearing loss also plays a key role, with many adults not wanting to appear old or be identified as a person with a disability. In effect, a cochlear implant makes deafness visible. New technologies have led to a smaller external profile for some types of cochlear implants, but qualitative assessments of benefit have not been explored. This study will examine cochlear implant aesthetics and cosmetics, and its impact on perceived stigma, social interactions, communication and quality of life. A particular focus will be the examination of totally implantable device concepts. A secondary aim is to understand what research techniques are best suited and most appealing for cochlear implant recipients, to assist in future study design and data collection methods. METHODS AND ANALYSIS: This study utilises a mixed-methods design. Three datasets will be collected from each participant with an expected sample size of 10-15 participants to allow for data saturation of themes elicited. Each participant will complete a demographic questionnaire, a quickfire survey (a short concise questionnaire on a topic of research familiarity and preference) and a semi-structured interview. Questionnaire and quickfire survey data will be analysed using descriptive statistics. Interviews will be transcribed and analysed thematically. All participants will be adults with more than 1 year of experience using cochlear implants. ETHICS AND DISSEMINATION: This study has been granted ethical approval from Macquarie University (HREC: 520211056232432) and meets the requirements set out in the National Statement on Ethical Conduct in Human Research. Study findings will be disseminated widely through international peer-reviewed journal articles, public and academic presentations, plain language summaries for participants and an executive summary for the project funder. This work was supported by Cochlear Limited (Cochlear Ltd). The funder will have no role in conducting or reporting on the study.

Developing Methods to Support Collaborative Learning and Co-creation of Resilient Healthcare-Tips for Success and Lessons Learned From a Norwegian Hospital Cancer Care Study.

BACKGROUND: There is a growing attention on the role of patients and stakeholders in resilience, but there is lack of knowledge and methods on how to support collaborative learning between stakeholders and co-creation of resilient healthcare. The aim of this article was to demonstrate how the methodological process of a consensus process for exploring aspects of next of kin involvement in hospital cancer care can be replicated as an effort to promote resilient healthcare through co-creation with multiple stakeholders in hospitals. METHODS: The study applied a modified nominal group technique process developed by synthesizing research findings across 4 phases of a research project with a mixed-methods approach. The process culminated in a 1-day meeting with 20 stakeholder participants (5 next of kin representatives, 10 oncology nurses, and 5 physicians) from 2 Norwegian university hospitals. RESULTS: The consensus method established reflexive spaces with collective sharing of experiences between the 2 hospitals and between the next of kin and healthcare professionals. The method promoted collaborative learning processes including identification and reflection upon new ideas for involvement, and reduction of the gap between healthcare professionals' and next of kin experiences and expectations for involvement. Next of kin were considered as important resources for resilient performance, if involved with a proactive approach. The consensus process identified both successful and unsuccessful collaborative practices and resulted in a co-designed guide for healthcare professionals to support next of kin involvement in hospital cancer care. CONCLUSIONS: This study expands the body of knowledge on methods development that is relevant for collaborative learning and co-creation of resilient healthcare. This study demonstrated that the consensus methods process can be used for creating reflexive spaces to support collaborative learning and co-creation of resilience in cancer care. Future research within the field of collaborative learning should explore interventions that include a larger number of stakeholders.

Preoperative Anticoagulation Management in Everyday Clinical Practice: An International Comparative Analysis of Work-as-Done Using the Functional Resonance Analysis Method.

OBJECTIVES: Preoperative anticoagulation management (PAM) is a complex, multidisciplinary process important to patient safety. The Functional Resonance Analysis Method (FRAM) is a novel method to study how complex processes usually go right at the frontline (labeled Safety-II) and how this relates to predefined procedures. This study aimed to assess PAM in everyday practice and explore the usability and utility of FRAM. METHODS: The study was conducted at an Australian and European Cardiothoracic Surgery Department. A FRAM model of work-as-imagined was developed using (inter)national guidelines. Semistructured interviews with 18 involved professionals were used to develop models reflecting work-as-done at both sites, which were presented to staff for validation. Workload in hours was estimated per process step. RESULTS: In both centers, work-as-done differed from work-as-imagined, such as in the division of tasks among disciplines (e.g., nurses/registrars rather than medical specialists), but control mechanisms had been developed locally to ensure safe care (e.g., crosschecking with other clinicians). Centers had organized the process differently, revealing opportunities for improvement regarding patient information and clustering of clinic visits. Presenting FRAM models to staff initiated discussion on improvement of functions in the model that are vital for success. Overall workload was estimated at 47 hours per site. CONCLUSIONS: This FRAM analysis provided insight into PAM from the perspective of frontline clinicians, revealing essential functions, interdependencies and variability, and the relation with guidelines. Future studies are warranted to study the potential of FRAM, such as for guiding improvements in complex systems.

Variability in clinicians' understanding and reported methods of identifying high-risk surgical patients: a qualitative study.

BACKGROUND: High-risk patients presenting for surgery require complex decision-making and perioperative management. However, given there is no gold standard for identifying high-risk patients, doing so may be challenging for clinicians in practice. Before a gold standard can be established, the state of current practice must be determined. This study aimed to understand how working clinicians define and identify high-risk surgical patients. METHODS: Clinicians involved in the care of high-risk surgical patients at a public hospital in regional Australia were interviewed as part of an ongoing study evaluating a new shared decision-making process for high-risk patients. The new process, Patient-Centred Advanced Care Planning (PC-ACP) engages patients, families, and clinicians from all relevant specialties in shared decision-making in line with the patient's goals and values. The semi-structured interviews were conducted before the implementation of the new process and were coded using a modified form of the 'constant comparative method' to reveal key themes. Themes concerning patient risk, clinician's understanding of high risk, and methods for identifying high-risk surgical patients were extricated for close examination. RESULTS: Thirteen staff involved in high-risk surgery at the hospital at which PC-ACP was to be implemented were interviewed. Analysis revealed six sub-themes within the major theme of factors related to patient risk: (1) increase in high-risk patients, (2) recognising frailty, (3) risk-benefit balance, (4) suitability and readiness for surgery, (5) avoiding negative outcomes, and (6) methods in use for identifying high-risk patients. There was considerable variability in clinicians' methods of identifying high-risk patients and regarding their definition of high risk. This variability occurred even among clinicians within the same disciplines and specialties. CONCLUSIONS: Although clinicians were confident in their own ability to identify high-risk patients, they acknowledged limitations in recognising frail, high-risk patients and predicting and articulating possible outcomes when consenting these patients. Importantly, little consistency in clinicians' reported methods for identifying high-risk patients was found. Consensus regarding the definition of high-risk surgical patients is necessary to ensure rigorous decision-making.

A mobile methods pilot study of surgical spaces: 'fit for purpose? Organisational productivity and workforce wellbeing in workspaces in hospital' (FLOURISH).

BACKGROUND: Good workspace design is key to the quality of work, safety, and wellbeing for workers, yet we lack vital knowledge about optimal hospital design to meet healthcare workforce needs. This study used novel mobile methods to examine the concept of Work-as-Done and the effect of workspace-use on healthcare professional practice, productivity, health and safety in an Australian university hospital. METHODS: This pilot study took place in one gastroenterological surgical unit between 2018 and 2019. Data collection involved 50 h of observations and informal conversations, followed by interpretation of five architectural plans and 45 photographs. Fieldnotes were thematically analysed and corroborated by analysis of visual data using a predefined taxonomy. RESULTS: Six themes were identified, revealing spaces that both support and hinder Work-as-Done. Fit-for-purpose spaces facilitated effective communication between staff, patients and families, conferred relative comfort and privacy, and supported effective teamwork. Unfit-for-purpose spaces were characterised by disruptions to work practices, disharmony among team members, and physical discomfort for staff. Staff employed workarounds to manage unfit-for-purpose spaces. CONCLUSION: The results identified negative impacts of negotiating unfit-for-purpose workspaces on the work and wellbeing of staff. While the use of workarounds and adaptations enable staff to maintain everyday working practices, they can also lead to unexpected consequences. Results indicated the need to identify and support fit-for-purpose spaces and minimize the detrimental qualities of unfit-for-purpose spaces. This study showed that mobile methods were suitable for examining Work-as-Done in a fast-moving, adaptive hospital setting.

Using the WHO International Classification of patient safety framework to identify incident characteristics and contributing factors for medical or surgical complication deaths.

This study aimed to operationalise and use the World Health Organisation's International Classification for Patient Safety (ICPS) to identify incident characteristics and contributing factors of deaths involving complications of medical or surgical care in Australia. A sample of 500 coronial findings related to patient deaths following complications of surgical or medical care in Australia were reviewed using a modified-ICPS (mICPS). Over two-thirds (69.0%) of incidents occurred during treatment and 27.4% occurred in the operating theatre. Clinical process and procedures (55.9%), medication/IV fluids (11.2%) and healthcare-associated infection/complications (10.4%) were the most common incident types. Coroners made recommendations in 44.0% of deaths and organisations undertook preventive actions in 40.0% of deaths. This study demonstrated that the ICPS was able to be modified for practical use as a human factors taxonomy to identify sequences of incident types and contributing factors for patient deaths. Further testing of the mICPS is warranted.

Fit for purpose? OrganisationaL prOdUctivity and woRkforce wellbeIng in workSpaces in Hospital (FLOURISH): a multimethod qualitative study protocol.

INTRODUCTION: Workspaces are socially constructed environments where social relationships are produced, reproduced, challenged and transformed. Their primary function is to support high-quality service delivery to the benefit of 'patients'. They are also settings where employees can work effectively, in a safe and healthy environment, delivering a high-quality service according to a 'Work-As-Done' rather than a 'Work-As-Imagined' model. However, hospital design is currently based on a managerial understanding of work accomplishments, often falling short of understanding what is actually happening on the ground. Furthermore, the research landscape lacks rigorous assessment of these complex sociological and health research concepts, either within the Australian context where this protocol is set, or internationally.This paper describes an innovative protocol aimed at examining healthcare employees' and organisations' concerns and beliefs in workspace design. It outlines research investigating the effect of workspace use on productivity, health and safety and worker satisfaction, to clarify Work-As-Done, while creating healthy and more fulfilling environments. METHODS: This is a proof-of-concept study, taking place between June 2018 and April 2019, employing a multimethod, qualitative approach for in-depth assessment of one Australian, private, university hospital environment, using as its 'case' the Gastroenterology Surgical Unit. It involves (1) observations and informal interviews (shadowing) with employees and patients as they traverse hospital spaces and (2) visual data of spatial use. Fieldnotes will be analysed thematically, and visual data analysed using a predefined schematic framework (a visual taxonomy). Overarching themes and categories will be considered corroboratively, mixing visual and textual data to build an iterative and dynamic picture. ETHICS AND DISSEMINATION: Ethical considerations will be discussed, while approval has been granted by the University's Human Research Ethics Committee (HREC/5201800282), along with Governance approved by the Health Clinical Research Executive (CRG2018005). Study results will be disseminated through publications, research conferences and public reports.

Improving decision making in acute healthcare through implementation of an intensive care unit (ICU) intervention in Australia: a multimethod study.

OBJECTIVE: To evaluate the implementation of an intensive care unit (ICU) intervention designed to establish rules for making ICU decisions about postsurgery beds. DESIGN: Preintervention/postintervention case study using a multimethod approach, involving two phases of staff interviews, process mapping and collection of administrative data. SETTING: ICU in a 700-bed regional tertiary care hospital in Australia. PARTICIPANTS: 31 interview participants. Phases 1 and 2 participants drawn from three groups of staff: bedside nursing staff in the ICU, ICU specialist doctors and senior management staff involved in oversight of ICU operations. Phase 2 included an additional participant group: staff from surgery and emergency departments. INTERVENTION: Implementation of an ICU escalation plan and introduction of a multidisciplinary morning meeting to determine ICU bed status in accordance with the plan. MAIN OUTCOME MEASURES: Interview data consisted of preintervention staff perceptions of ICU workplace cohesiveness with bed pressure, and postintervention staff perceptions of the escalation plan and ICU performance. Administrative data consisted of bed status (red, amber or green), monthly number of planned elective surgeries requiring an ICU bed and monthly number of elective surgeries cancelled due to unavailability of ICU beds. RESULTS: Improved internal communication, decision making and cohesion within the ICU and better coordination between ICU and other hospital departments. Significant reduction in elective surgeries cancelled due to unavailability of ICU beds, χ21=24.9, p<0.0001. CONCLUSIONS: By establishing rules for decision making around ICU bed allocation, the intervention improved internal professional relationships within the ICU as well as between the ICU and external departments and reduced the number of elective surgeries cancelled.

Determinants of health and wellbeing in refractory epilepsy and surgery: The Patient Reported, ImpleMentation sciEnce (PRIME) model.

This paper offers a new way of understanding the course of a chronic, neurological condition through a comprehensive model of patient-reported determinants of health and wellbeing: The Patient Reported ImpleMentation sciEnce (PRIME) model is the first model of its kind to be based on patient-driven insights for the design and implementation of initiatives that could improve tertiary, primary, and community healthcare services for patients with refractory epilepsy, and has broad implications for other disorders; PRIME focuses on: patient-reported determinants of health and wellbeing, pathways through care, gaps in treatment and other system delays, patient need and expectation, and barriers and facilitators to high-quality care provision; PRIME highlights that in the context of refractory epilepsy, patients value appropriate, clear, and speedy referrals from primary care practitioners and community neurologists to specialist healthcare professionals based in tertiary epilepsy centers. Many patients also want to share in decisions around treatment and care, and gain a greater understanding of their debilitating disease, so as to find ways to self-manage their illness more effectively and plan for the future. Here, PRIME is presented using refractory epilepsy as the exemplar case, while the model remains flexible, suitable for adaptation to other settings, patient populations, and conditions; PRIME comprises six critical levels: 1) The Individual Patient Model; 2) The Patient Relationships Model; 3) The Patient Care Pathways Model; 4) The Patient Transitions Model; 5) The Pre- and Postintervention Model; and 6) The Comprehensive Patient Model. Each level is dealt with in detail, while Levels 5 and 6 are presented in terms of where the gaps lie in our current knowledge, in particular in relation to patients' journeys through healthcare, system intersections, and individuals adaptive behavior following resective surgery, as well as others' views of the disease, such as family members.

Effectiveness of an electronic patient-centred self-management tool for gout sufferers: a cluster randomised controlled trial protocol.

INTRODUCTION: Gout is increasing despite effective therapies to lower serum urate concentrations to 0.36 mmol/L or less, which, if sustained, significantly reduces acute attacks of gout. Adherence to urate-lowering therapy (ULT) is poor, with rates of less than 50% 1 year after initiation of ULT. Attempts to increase adherence in gout patients have been disappointing. We aim to evaluate the effectiveness of use of a personal, self-management, 'smartphone' application (app) to achieve target serum urate concentrations in people with gout. We hypothesise that personalised feedback of serum urate concentrations will improve adherence to ULT. METHODS AND ANALYSIS: Setting and designPrimary care. A prospective, cluster randomised (by general practitioner (GP) practices), controlled trial. PARTICIPANTS: GP practices will be randomised to either intervention or control clusters with their patients allocated to the same cluster. INTERVENTION: The intervention group will have access to the Healthy.me app tailored for the self-management of gout. The control group patients will have access to the same app modified to remove all functions except the Gout Attack Diary. PRIMARY AND SECONDARY OUTCOMES: The proportion of patients whose serum urate concentrations are less than or equal to 0.36 mmol/L after 6 months. Secondary outcomes will be proportions of patients achieving target urate concentrations at 12 months, ULT adherence rates, serum urate concentrations at 6 and 12 months, rates of attacks of gout, quality of life estimations and process and economic evaluations. The study is designed to detect a ≥30% improvement in the intervention group above the expected 50% achievement of target serum urate at 6 months in the control group: power 0.80, significance level 0.05, assumed 'dropout' rate 20%. ETHICS AND DISSEMINATION: This study has been approved by the University of New South Wales Human Research Ethics Committee. Study findings will be disseminated in international conferences and peer-reviewed journal. TRIAL REGISTRATION NUMBER: ACTRN12616000455460.

Improving outcomes from high-risk surgery: a multimethod evaluation of a patient-centred advanced care planning intervention.

INTRODUCTION: Patients who are frail, have multiple comorbidities or have a terminal illness often have poor outcomes from surgery. However, sole specialists may recommend surgery in these patients without consultation with other treating clinicians or allowing for patient goals. The Patient-Centred Advanced Care Planning (PC-ACP) model of care provides a framework in which a multidisciplinary advanced care plan is devised to incorporate high-risk patients' values and goals. Decision-making is performed collaboratively by patients, their family, surgeons, anaesthetists, intensivists and surgical case managers. This study aims to evaluate the feasibility of this new model of care, and to determine potential benefits to patients and clinicians. METHODS AND ANALYSIS: After being assessed for frailty, patients will complete a patient-clinician information engagement survey pretreatment and at 6 months follow-up. Patients (and/or family members) will be interviewed about their experience of care pretreatment and at 3 and 6 months follow-ups. Clinicians will complete a survey on workplace attitudes and engagement both preimplementation and postimplementation of PC-ACP and be interviewed, following each survey, on the implementation of PC-ACP. We will use process mapping to map the patient journey through the surgical care pathway to determine areas of improvement and to identify variations in patient experience. ETHICS AND DISSEMINATION: This study has received ethical approval from Townsville Hospital and Health Service HREC (HREC/16/QTHS/100). Results will be communicated to the participating hospital, presented at conferences and submitted for publication in a peer-reviewed MEDLINE-indexed journal.

Macquarie Surgical Innovation Identification Tool (MSIIT): a study protocol for a usability and pilot test.

INTRODUCTION: Medicine relies on innovation to continually improve. However, innovation is potentially risky, and not all innovations are successful. Therefore, it is important to identify innovations prospectively and provide support, to make innovation as safe and effective as possible. The Macquarie Surgical Innovation Identification Tool (MSIIT) is a simple checklist designed as a practical tool for hospitals to identify planned surgical innovations. This project aims to test the usability and pilot the use of the MSIIT in a surgical setting. METHODS AND ANALYSIS: The project will run in two phases at two Australian hospitals, one public and one private. Phase I will involve interviews, focus groups and a survey of hospital administrators and surgical teams to assess the usability and system requirements for the use of the MSIIT. Current practice regarding surgical innovation within participating hospitals will be mapped, and the best implementation strategy for MSIIT completion will be established. Phase II will involve trialling the MSIIT for each surgery within the trial period by various surgical personnel. Follow-up interviews, focus groups and a survey will be conducted with trial participants to collect feedback on their experience of using the MSIIT during the trial period. Comparative data on rates of surgical innovation during the trial period will also be gathered from existing hospital systems and compared to the rates identified by the MSIIT. ETHICS AND DISSEMINATION: Ethical approval has been obtained. The results of this study will be presented to interested health services and other stakeholders, presented at conferences and published in a peer-reviewed MEDLINE-indexed journal.

Determination of health-care teamwork training competencies: a Delphi study.

OBJECTIVE: The purpose of this study was to determine the optimum content of a 1-day classroom-based crew resource management (CRM) course for health-care personnel working in ad hoc teams in complex, time-critical hospital departments such as surgery, intensive care or emergency. DESIGN: A two-round modified Delphi panel. PARTICIPANTS: selected teamwork competency components suitable for inclusion in 1 day of training from a list developed via literature review. Participants Fifteen experts in health care, CRM and training. MAIN OUTCOME MEASURE: Knowledge, skill and attitude competency components for a 1-day CRM course. RESULTS: Of the 110 knowledge, skill and attitude CRM competency components, 40 components were selected by greater than 70% of respondents, whereas the remaining 62 components were selected by fewer than 55% of respondents. These 40 competency components ranged across five competency domains: communication, task management, situational awareness, decision-making and leadership, and provided a consensus on the most critical areas for inclusion in training for health-care personnel. CONCLUSIONS: This new competency model is now available for use. Although the sample size was limited, a high degree of consensus was reached after only two rounds. A modified Delphi technique within the context of competencies first refined from the literature was a useful and cost-effective method for determining the content of a 1-day CRM training course for health-care workers.