The cirrhosis care Alberta (CCAB) protocol: implementing an evidence-based best practice order set for the management of liver cirrhosis - a hybrid type I effectiveness-implementation trial.

BACKGROUND: Liver cirrhosis is a leading cause of morbidity, premature mortality and acute care utilization in patients with digestive disease. In the province of Alberta, hospital readmission rates for patients with cirrhosis are estimated at 44% at 90 days. For hospitalized patients, multiple care gaps exist, the most notable stemming from i) the lack of a structured approach to best practice care for cirrhosis complications, ii) the lack of a structured approach to broader health needs and iii) suboptimal preparation for transition of care into the community. Cirrhosis Care Alberta (CCAB) is a 4-year multi-component pragmatic trial which aims to address these gaps. The proposed intervention is initiated at the time of hospitalization through implementation of a clinical information system embedded electronic order set for delivering evidence-based best practices under real-world conditions. The overarching objective of the CCAB trial is to demonstrate effectiveness and implementation feasibility for use of the order set in routine patient care within eight hospital sites in Alberta. METHODS: A mixed methods hybrid type I effectiveness-implementation design will be used to evaluate the effectiveness of the order set intervention. The primary outcome is a reduction in 90-day cumulative length of stay. Implementation outcomes such as reach, adoption, fidelity and maintenance will also be evaluated alongside other patient and service outcomes such as readmission rates, quality of care and cost-effectiveness. This theory-based trial will be guided by Normalization Process Theory, Consolidated Framework on Implementation Research (CFIR) and the Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) Framework. DISCUSSION: The CCAB project is unique in its breadth, both in the comprehensiveness of the multi-component order set and also for the breadth of its roll-out. Lessons learned will ultimately inform the feasibility and effectiveness of this approach in "real-world" conditions as well as adoption and adaptation of these best practices within the rest of Alberta, other provinces in Canada, and beyond. TRIAL REGISTRATION: ClinicalTrials.gov: NCT04149223, November 4, 2019.

Diagnosing technical competence in six bedside procedures: comparing checklists and a global rating scale in the assessment of resident performance.

PURPOSE: To compare procedure-specific checklists and a global rating scale in assessing technical competence. METHOD: Two trained raters used procedure-specific checklists and a global rating scale to independently evaluate 218 video-recorded performances of six bedside procedures of varying complexity for technical competence. The procedures were completed by 47 residents participating in a formative simulation-based objective structured clinical examination at the University of Calgary in 2011. Pass/fail (competent/not competent) decisions were based on an overall global assessment item on the global rating scale. Raters provided written comments on performances they deemed not competent. Checklist minimum passing levels were set using traditional standard-setting methods. RESULTS: For each procedure, the global rating scale demonstrated higher internal reliability and lower interrater reliability than the checklist. However, interrater reliability was almost perfect for decisions on competence using the overall global assessment (Kappa range: 0.84-1.00). Clinically significant procedural errors were most often cited as reasons for ratings of not competent. Using checklist scores to diagnose competence demonstrated acceptable discrimination: The area under the curve ranged from 0.84 (95% CI 0.72-0.97) to 0.93 (95% CI 0.82-1.00). Checklist minimum passing levels demonstrated high sensitivity but low specificity for diagnosing competence. CONCLUSIONS: Assessment using a global rating scale may be superior to assessment using a checklist for evaluation of technical competence. Traditional standard-setting methods may establish checklist cut scores with too-low specificity: High checklist scores did not rule out incompetence. The role of clinically significant errors in determining procedural competence should be further evaluated.