Advancing discussion of ethics in mixed methods health services research.

BACKGROUND: To describe the ethical issues and experiences of scientists conducting mixed methods health services research and to advance empirical and conceptual discussion on ethical integrity in mixed methods health research. METHODS: The study was conducted with 64 scholars, faculty and consultants from the NIH-funded Mixed Methods Research Training Program (MMRTP) for the Health Sciences. This was a cross-sectional study. Survey results were analyzed using descriptive statistics to characterize responses and open coding to summarize strategies about eight ethical mixed methods research issues. Respondents completed an online survey to elicit experiences related to eight ethical issues (informed consent, confidentiality, data management, burden, safety, equitable recruitment, communication, and dissemination) and strategies for addressing them. RESULTS: Only about one-third of respondents thought their research ethics training helped them plan, conduct, or report mixed methods research. The most frequently occurring ethical issues were participant burden, dissemination and equitable recruitment (> 70% endorsement). Despite occurring frequently, < 50% of respondents rated each ethical issue as challenging. The most challenging ethical issues were related to managing participant burden, communication, and dissemination. Strategies reported to address ethical issues were largely not specific or unique to mixed methods with the exception of strategies to mitigate participant burden and, to a lesser degree, to facilitate equitable recruitment and promote dissemination of project results. CONCLUSIONS: Mixed methods health researchers reported encountering ethical issues often yet varying levels of difficulty and effectiveness in the strategies used to mitigate ethical issues. This study highlights some of the unique challenges faced by mixed methods researchers to plan for and appropriately respond to arising ethical issues such as managing participant burden and confidentiality across data sources and utilizing effective communication and dissemination strategies particularly when working with a multidisciplinary research team. As one of the first empirical studies to examine mixed methods research ethics, our findings highlight the need for greater attention to ethics in health services mixed methods research and training.

Rapid assessment of communication consistency: sentiment analysis of public health briefings during the COVID-19 pandemic.

Background: A key component of the initial public health response to the COVID-19 pandemic involved the use of mass media briefings led by public health officials to communicate updates during a time of great uncertainty and rapidly changing information. This study aims to examine the consistency of communications expressed during the public health briefings to generate novel insights about the type, direction, and strength of public health messages. The data source included 131 readily accessible public health briefings alongside the provincial and national new confirmed case counts during the first two waves of rapidly increasing cases during the pandemic in Alberta, Canada. We employed sentiment analysis as a text mining technique to explore the types and frequency of words in public health briefings conveying positive and negative sentiments. Using statistical analyses and data visualizations, we examined how public health messaging shifted with case trends. Results: Our findings indicate consistent public health messaging in terms of sentiments regardless of case count fluctuations, an association of specific words with conveying positive and negative sentiments, and a focus on particular message patterns at different points during the first two waves of the COVID-19 pandemic. Conclusion: Our findings demonstrate the practical implications and methodological advantages of using sentiment analysis as a data analytics tool for rapidly and objectively assessing the consistency of health communications during a public health crisis.

Competency-based achievement system: using formative feedback to teach and assess family medicine residents' skills.

PROBLEM ADDRESSED: Family medicine residency programs require innovative means to assess residents' competence in "soft" skills (eg, patient-centred care, communication, and professionalism) and to identify residents who are having difficulty early enough in their residency to provide remedial training. OBJECTIVE OF PROGRAM: To develop a method to assess residents' competence in various skills and to identify residents who are having difficulty. PROGRAM DESCRIPTION: The Competency-Based Achievement System (CBAS) was designed to measure competence using 3 main principles: formative feedback, guided self-assessment, and regular face-to-face meetings. The CBAS is resident driven and provides a framework for meaningful interactions between residents and advisors. Residents use the CBAS to organize and review their feedback, to guide their own assessment of their progress, and to discern their future learning needs. Advisors use the CBAS to monitor, guide, and verify residents' knowledge of and competence in important skills. CONCLUSION: By focusing on specific skills and behaviour, the CBAS enables residents and advisors to make formative assessments and to communicate their findings. Feedback indicates that the CBAS is a user-friendly and helpful system to assess competence.

Assessing competency-based evaluation course impacts: A mixed methods case study.

The contexts in which evaluators develop and apply their expertise are increasingly complex; evaluator education needs to provide robust opportunities to support and assess the progressive, lifelong development of relevant knowledge and skills. This mixed methods case study begins to address the dearth of empirical evidence assessing the impacts and learner experiences of competency-based approaches to evaluator education. A decade-in-the-making doctoral evaluation course based on the Canadian Evaluation Society's Competencies for Canadian Evaluation Practice created an opportune study setting. We applied a systems perspective to Stufflebeam's Context, Input, Process, and Product (CIPP) model to frame the case study analysis and presentation of the complex impacts generated by pre/post course competency self-assessments and reflections, integrated with an end-of-course focus group. Seven insights shed new light on the effective course design and implementation features for developing intended and unintended evaluator competencies. We conclude with a discussion of the theoretical, practical, and methodological implications for effective competency-based evaluator education.