Virtual OSCE Delivery and Quality Assurance During a Pandemic: Implications for the Future.

Background: During 2020, the COVID-19 pandemic caused worldwide disruption to the delivery of clinical assessments, requiring medicals schools to rapidly adjust their design of established tools. Derived from the traditional face-to-face Objective Structured Clinical Examination (OSCE), the virtual OSCE (vOSCE) was delivered online, using a range of school-dependent designs. The quality of these new formats was evaluated remotely through virtual quality assurance (vQA). This study synthesizes the vOSCE and vQA experiences of stakeholders from participating Australian medical schools based on a Quality framework. Methods: This study utilized a descriptive phenomenological qualitative design. Focus group discussions (FGD) were held with 23 stakeholders, including examiners, academics, simulated patients, professional staff, students and quality assurance examiners. The data was analyzed using a theory-driven conceptual Quality framework. Results: The vOSCE was perceived as a relatively fit-for purpose assessment during pandemic physical distancing mandates. Additionally, the vOSCE was identified as being value-for-money and was noted to provide procedural benefits which lead to an enhanced experience for those involved. However, despite being largely delivered fault-free, the current designs are considered limited in the scope of skills they can assess, and thus do not meet the established quality of the traditional OSCE. Conclusions: Whilst virtual clinical assessments are limited in their scope of assessing clinical competency when compared with the traditional OSCE, their integration into programs of assessment does, in fact, have significant potential. Scholarly review of stakeholder experiences has elucidated quality aspects that can inform iterative improvements to the design and implementation of future vOSCEs.

"Could You Work in My Team?": Exploring How Professional Clinical Role Expectations Influence Decision-Making of Assessors During Exit-Level Medical School OSCEs.

Decision-making in clinical assessment, such as exit-level medical school Objective Structured Clinical Examinations (OSCEs), is complex. This study utilized an empirical phenomenological qualitative approach with thematic analysis to explore OSCE assessors' perceptions of the concept of a "prototypical intern" expressed during focus group discussions. Topics discussed included the concept of a prototypical intern, qualities to be assessed, and approaches to clinical assessment decision-making. The thematic analysis was then applied to a theoretical framework (Cultural Historical Activity Theory-CHAT) that explored the complexity of making assessment decisions amidst potentially contradicting pressures from academic and clinical perspectives. Ten Australasian medical schools were involved with 15 experienced and five less experienced assessors participating. Thematic analysis of the data revealed four major themes in relation to how the prototypical intern concept influences clinical assessors' judgements: (a) Suitability of marking rubric based on assessor characteristics and expectations; (b) Competence as final year student vs. performance as a prototypical intern; (c) Safety, trustworthiness and reliability as constructs requiring assessment and (d) Contradictions in decision making process due to assessor differences. These themes mapped well within the interaction between two proposed activity systems in the CHAT model: academic and clinical. More clinically engaged and more experienced assessors tend to fall back on a heuristic, mental construct of a "prototypical intern," to calibrate judgements, particularly, in difficult situations. Further research is needed to explore whether consensus on desirable intern qualities and their inclusion into OSCE marksheets decreases the cognitive load and increases the validity of assessor decision making.

Evaluation of learning from Practical Obstetric Multi-Professional Training and its impact on patient outcomes in Australia using Kirkpatrick's framework: a mixed methods study.

OBJECTIVES: The aim of this study was to evaluate the implementation of the Practical Obstetric Multi-Professional Training (PROMPT) simulation using the Kirkpatrick's framework. We explored participants' acquisition of knowledge and skills, its impact on clinical outcomes and organisational change to integrate the PROMPT programme as a credentialing tool. We also aimed to assess participants' perception of usefulness of PROMPT in their clinical practice. STUDY DESIGN: Mixed methods approach with a pre-test/post-test design. SETTING: Healthcare network providing obstetric care in Victoria, Australia. PARTICIPANTS: Medical and midwifery staff attending PROMPT between 2013 and 2015 (n=508); clinical outcomes were evaluated in two cohorts: 2011-2012 (n=15 361 births) and 2014-2015 (n=12 388 births). INTERVENTION: Attendance of the PROMPT programme, a simulation programme taught in multidisciplinary teams to facilitate teaching emergency obstetric skills. MAIN OUTCOME MEASURE: Clinical outcomes compared before and after embedding PROMPT in educational practice. SECONDARY OUTCOME MEASURE: Assessment of knowledge gained by participants through a qualitative analysis and description of process of embedding PROMPT in educational practice. RESULTS: There was a change in the management of postpartum haemorrhage by early recognition and intervention. The key learning themes described by participants were being prepared with a prior understanding of procedures and equipment, communication, leadership and learning in a safe, supportive environment. Participants reported a positive learning experience and increase in confidence in managing emergency obstetric situations through the PROMPT programme, which was perceived as a realistic demonstration of the emergencies. CONCLUSION: Participants reported an improvement of both clinical and non-technical skills highlighting principles of teamwork, communication, leadership and prioritisation in an emergency situation. An improvement was observed in management of postpartum haemorrhage, but no significant change was noted in clinical outcomes over a 2-year period after PROMPT. However, the skills acquired by medical and midwifery staff justify embedding PROMPT in educational programmes.

Migrant fathers and their attitudes to potential male hormonal contraceptives.

The purpose of this study was to assess potential uptake of male hormonal contraception (MHC) in migrant fathers in a post-partum setting, and to compare them to Australian-born fathers. It was a cross-sectional study of a convenience sample from the post-natal ward of a tertiary level obstetric hospital. Seventy-six English-speaking fathers born in South-East Asia or on the Indian subcontinent were surveyed and their responses compared with those provided by 118 Australian-born participants from a previous study. The main outcome measures were acceptability of potential male hormonal contraception on a 5-point scale, and preferred mode of administration. Information on past and future intended contraceptive use, including existing male forms of contraception, was also compared with data collected from the group of Australian-born men. Only 13.6% (95%CI: 5.8-21.4) of migrant fathers would definitely or probably consider trying MHC compared with 47.5% (95%CI: 38.5-56.5) of Australian-born fathers (chi-square, p < 0.001). There were significant differences in desired mode of administration in potential 'triers' from both groups, as well as in attitudes to existing contraception. Two-yearly injection was the most popular method of administration in migrants, with 38.3% of 'triers' listing it as their first choice (compared with 21.4% in published data on Australian-born men; chi-square, p = 0.038). We conclude that migrant groups are less enthusiastic about novel potential MHC. The influence of education on acceptance of this contraceptive possibility needs to be assessed.

Will Australian men use male hormonal contraception? A survey of a postpartum population.

AIM: To survey the attitudes of a population of Australian men to potential use of male hormonal contraception (MHC). DESIGN: Survey of male partners of women who had recently given birth. Men were approached while visiting their female partners on the ward. PARTICIPANTS: 118 out of 148 Australian-born English-speaking men who were approached. SETTING: Postnatal ward of Monash Medical Centre (a public teaching hospital in Melbourne), between October 2000 and April 2001. MAIN OUTCOME MEASURE: Attitudes towards potential use of MHC, rated on a five-point scale. RESULTS: 89/118 men surveyed (75.4%; 95% Cl, 67.7%-83.2%) indicated that they would consider trying MHC if it were available. The three most popular choices for method of administration of MHC were (in descending order) an oral pill, a three-monthly injection, or a two-yearly injection. A statistically significant association was found between acceptability of vasectomy and acceptability of MHC (70.5% of men who indicated they would try MHC [MHC "triers"] found vasectomy acceptable versus 44.5% of MHC "non-triers"; P = 0.011). Triers reported a higher rate of approval of MHC by their female partners than non-triers (79.8% v 13.8%, respectively; P < 0.0001). CONCLUSIONS: MHC appears to be acceptable to a majority of Australian men when surveyed in a postpartum context. Attitudes of men towards existing male contraception, as well as the attitudes of their partners, appear to exert a strong influence on acceptability of MHC.