Effects of Practicing With and Obtaining Crowdsourced Feedback From the Video-Based Communication Assessment App on Resident Physicians' Adverse Event Communication Skills: Pre-post Trial.

BACKGROUND: US residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide individualized feedback to residents about communication skills, and may not assure that residents acquire the skills desired by patients. The Video-based Communication Assessment (VCA) app is a novel tool for simulating communication scenarios for practice and obtaining crowdsourced assessments and feedback on physicians' communication skills. We previously established that crowdsourced laypeople can reliably assess residents' error disclosure skills with the VCA app. However, its efficacy for error disclosure training has not been tested. OBJECTIVE: We aimed to evaluate the efficacy of using VCA practice and feedback as a stand-alone intervention for the development of residents' error disclosure skills. METHODS: We conducted a pre-post study in 2020 with pathology, obstetrics and gynecology, and internal medicine residents at an academic medical center in the United States. At baseline, residents each completed 2 specialty-specific VCA cases depicting medical errors. Audio responses were rated by at least 8 crowdsourced laypeople using 6 items on a 5-point scale. At 4 weeks, residents received numerical and written feedback derived from layperson ratings and then completed 2 additional cases. Residents were randomly assigned cases at baseline and after feedback assessments to avoid ordinal effects. Ratings were aggregated to create overall assessment scores for each resident at baseline and after feedback. Residents completed a survey of demographic characteristics. We used a 2×3 split-plot ANOVA to test the effects of time (pre-post) and specialty on communication ratings. RESULTS: In total, 48 residents completed 2 cases at time 1, received a feedback report at 4 weeks, and completed 2 more cases. The mean ratings of residents' communication were higher at time 2 versus time 1 (3.75 vs 3.53; P<.001). Residents with prior error disclosure experience performed better at time 1 compared to those without such experience (ratings: mean 3.63 vs mean 3.46; P=.02). No differences in communication ratings based on specialty or years in training were detected. Residents' communication was rated higher for angry cases versus sad cases (mean 3.69 vs mean 3.58; P=.01). Less than half of all residents (27/62, 44%) reported prior experience with disclosing medical harm to patients; experience differed significantly among specialties (P<.001) and was lowest for pathology (1/17, 6%). CONCLUSIONS: Residents at all training levels can potentially improve error disclosure skills with VCA practice and feedback. Error disclosure curricula should prepare residents for responding to various patient affects. Simulated error disclosure may particularly benefit trainees in diagnostic specialties, such as pathology, with infrequent real-life error disclosure practice opportunities. Future research should examine the effectiveness, feasibility, and acceptability of VCA within a longitudinal error disclosure curriculum.

Video-Based Communication Assessment of Physician Error Disclosure Skills by Crowdsourced Laypeople and Patient Advocates Who Experienced Medical Harm: Reliability Assessment With Generalizability Theory.

BACKGROUND: Residents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios. OBJECTIVE: We aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app. METHODS: Internal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents' error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople. RESULTS: Internal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates' and crowdsourced laypeople's ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75. CONCLUSIONS: Crowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.

EVIDENCE REQUIRED BY HEALTH TECHNOLGY ASSESSMENT AND REIMBURSEMENT BODIES EVALUATING DIAGNOSTIC OR PROGNOSTIC ALGORITHMS THAT INCLUDE OMICS DATA.

OBJECTIVES: Multi-analyte assays with algorithmic analyses (MAAAs) use combinations of circulating and clinical markers including omics-based sources for diagnostic and/or prognostic purposes. Assessing MAAAs is challenging under existing health technology assessment (HTA) methods or practices. We undertook a scoping review to explore the HTA methods used for MAAAs to identify the criteria used for clinical research and reimbursement purposes. METHODS: This review included only non-companion (stand-alone) tests that are actionable and that have been evaluated by leading HTA or insurer/reimbursement bodies up to September 2017. RESULTS: Twenty-five reports and articles evaluating seventeen MAAAs were examined, most of which have been developed in oncology. The two main models used were the EUnetHTA Core model and the Evaluation of Genomic Applications in Practice and Prevention ACCE framework. Clinical validity and utility criteria were used, as were economic, ethical, legal, and social aspects. Economic evidence on MAAAs was scarce, and there is no consensus on whether the perspectives used are sufficiently broad to include all relevant stakeholders. CONCLUSIONS: Clinical utility and efficiency were the most used criteria, with stronger evidence needed linking the use of the algorithm with the clinical outcomes in real-life practice. HTA bodies must as well consider questions related to the analytical validity of MAAAs or with organizational aspects. The two main models, the EUnetHTA Core model and the ACCE framework, could be adapted to the assessment of MAAAs.

Social/economic costs and health-related quality of life in patients with cystic fibrosis in Europe.

OBJECTIVES: Our goal was to provide data on the economic burden and health-related quality of life (HRQOL) of patients with cystic fibrosis (CF) and their caregivers in Europe. METHODS: A cross-sectional study was carried out on adults and children with CF in eight European countries. Patients completed an anonymous questionnaire regarding their socio-demographic characteristics, use of healthcare services and presence of a caregiver. Costs were calculated with a bottom-up approach using unit costs from each participating country, and HRQOL was assessed using EQ-5D. The principal caregiver also answered a questionnaire on their characteristics, HRQOL and burden. RESULTS: A total of 905 patients with CF was included (399 adults and 506 children). The total average annual cost per patient varied from €21,144 in Bulgaria to €53,256 in Germany. Adults had higher direct healthcare costs than children, but children had much higher informal care costs (P < 0.0001). Total costs increased with patients' level of dependence. In adults, mean utility fell between 0.640 and 0.870, and the visual analogue scale ranged from 46.0 to 69.7. There was no difference in caregiver HRQOL regardless of whether they cared for an adult or a child. However, caregivers who looked after a child had a significantly higher burden (P = 0.0013). CONCLUSIONS: Our study highlights the burden of CF in terms of costs and decreased HRQOL for both patients and their caregivers throughout Europe.

Social/economic costs and health-related quality of life in patients with fragile X syndrome in Europe.

OBJECTIVE: To estimate the social/economic costs of fragile X syndrome (FXS) in Europe and to assess the health-related quality of life (HRQOL) of patients and caregivers. METHODS: A cross-sectional study was conducted in a sample of European countries. Patients were recruited through patients' associations. Data on their resource use and absence from the labour market were retrospectively obtained from an online questionnaire. Costs were estimated by a bottom-up approach and the EuroQol-5 Domain (EQ-5D) questionnaire was used to measure patients' and caregivers' HRQOL. RESULTS: Five countries were included in the analysis. The mean annual cost of FXS per patient varied from €4951 in Hungary to €58,862 in Sweden. Direct non-healthcare costs represented the majority of costs in all countries but there were differences in the share incurred by formal and informal care among those costs. Costs were also shown to differ between children and adults. Mean EQ-5D utility score for adult patients varied from 0.52 in France (n = 42) to 0.73 in Hungary (n = 2), while for caregivers this score was consistently inferior to 0.87. CONCLUSION: Our findings underline that, although its prevalence is low, FXS is costly from a societal perspective. They support the development of tailored policies to reduce the consequences of FXS on both patients and their relatives.