Reliability, validity and acceptability of an online clinical reasoning simulator for medical students: An international pilot.

INTRODUCTION: Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reasoning skills using real-life cases. METHODS: In 2020, we conducted an international, multicentre pilot study using 20 clinical cases with 2457 final-year medical students from 21 schools worldwide. Psychometric analysis was performed (n = 1502 students completing at least 80% of cases). Classical estimates of reliability for three test domains (hypothesis generation, hypothesis argumentation and knowledge application) were calculated using Cronbach's alpha and McDonald's omega coefficients. Validity evidence was obtained by confirmatory factor analysis (CFA) and measurement alignment (MA). Items from the knowledge application domain were analysed using cognitive diagnostic modelling (CDM). Acceptability was evaluated by an anonymous student survey. RESULTS: Reliability estimates were high with narrow confidence intervals. CFA revealed acceptable goodness-of-fit indices for the proposed three-factor model. CDM analysis demonstrated good absolute test fit and high classification accuracy estimates. Student survey responses showed high levels of acceptability. CONCLUSION: Our findings suggest that Practicum Script is a useful resource for strengthening students' clinical reasoning skills and ability to manage uncertainty.

Barriers and facilitators to establishing a clinical academic career in clinical education research in the UK: A focus group study.

In a rapidly changing healthcare environment, we need a robust evidence base to inform effective education and training. This study aimed to examine factors perceived to determine career progression in clinical education research in the UK. Six online focus groups were conducted, with 35 participants from a range of medical, dental, nursing, and allied health professions who identified as aspiring or early career clinical education researchers. Transcripts underwent thematic analysis. Two themes and associated subthemes were constructed to illustrate perceived factors impacting on career development: (1) A cultural challenge from clinical norms. Challenges included differences between the epistemological assumptions of biomedical and clinical research, and the underlying philosophy of education research, which is more closely aligned with the knowledge generation of the social sciences. This led to difficulty communicating the impact of education research to patient care. There were also blurred boundaries between education delivery and research, with the latter lacking a clearly defined group identity. (2) Structures, systems and relationships for career progression. Practical considerations included time and funding (or lack thereof), the opportunity to undertake formal training, networking and role models. This research highlights a number of systemic barriers and facilitators to careers in clinical education research and offers targets of intervention to enable a sustainable academic workforce in clinical education research.

High prevalence of severe sleep cycle disruption in de novo acromegaly and underdiagnosis by common clinical screening tools: A prospective, observational, cross-sectional study.

CONTEXT: Although sleep disordered breathing (SDB) is well-recognised in acromegaly, most studies have reported heterogeneous, often heavily treated, groups and few have performed detailed sleep phenotyping at presentation. OBJECTIVE: To study SDB using the gold standard of polysomnography, in the largest group of newly-diagnosed, treatment-naïve patients with acromegaly. SETTING AND PATIENTS: 40 patients [22 males, 18 females; mean age 54 years (range 23-78)], were studied to: (i) establish the prevalence and severity of SDB (ii) assess the reliability of commonly employed screening tools [Epworth Sleepiness Scale (ESS) and overnight oxygen desaturation index (DI)] to detect SDB (iii) determine the extent to which sleep architecture is disrupted. RESULTS: Obstructive sleep apnoea (OSA), defined by the apnoea-hypopnoea index (AHI), was present in 79% of subjects (mild, n = 12; moderate, n = 5; severe, n = 14). However, in these individuals with OSA by AHI criteria, ESS (positive in 35% [n = 11]) and DI (positive in 71%: mild, n = 11; moderate, n = 6; severe, n = 5) markedly underestimated its prevalence/extent. Seventy-eight percent of patients exhibited increased arousal, with marked disruption of the sleep cycle, despite most (82%) having normal total time asleep. Fourteen patients spent longer in stage 1 sleep. Deeper sleep stages were severely attenuated in many subjects (reduced stage 2, n = 18; reduced slow wave sleep, n = 24; reduced rapid eye movement sleep, n = 32). CONCLUSION: Our study provides strong support for clinical guidelines that recommend screening for sleep apnoea syndrome in patients with newly-diagnosed acromegaly. Importantly, however, it highlights shortcomings in commonly recommended screening tools (questionnaires, desaturation index) and demonstrates the added value of polysomnography to allow timely detection of obstructive sleep apnoea and associated sleep cycle disruption.

Undergraduate dermatology education: The importance of curriculum review.

In this letter, we highlight the considerable diversity in undergraduate dermatology training in the United Kingdom and acknowledge the barriers faced in implementing the revised national undergraduate curriculum provided by the British Association of Dermatologists (BAD). We provide a pragmatic approach of ensuring that our dermatology placement aligns with the BAD national undergraduate curriculum and Medical Licensing Assessment (MLA) content map, in the face of limited clinician time and placement length. We urge other medical schools to adopt our approach of curriculum mapping, particularly in light of the upcoming MLA, and sustainable educational resource development.

Twelve tips for introducing very short answer questions (VSAQs) into your medical curriculum.

Most undergraduate written examinations use multiple-choice questions, such as single best answer questions (SBAQs) to assess medical knowledge. In recent years, a strong evidence base has emerged for the use of very short answer questions (VSAQs). VSAQs have been shown to be an acceptable, reliable, discriminatory, and cost-effective assessment tool in both formative and summative undergraduate assessments. VSAQs address many of the concerns raised by educators using SBAQs including inauthentic clinical scenarios, cueing and test-taking behaviours by students, as well as the limited feedback SBAQs provide for both students and teachers. The widespread use of VSAQs in medical assessment has yet to be adopted, possibly due to lack of familiarity and experience with this assessment method. The following twelve tips have been constructed using our own practical experience of VSAQs alongside supporting evidence from the literature to help medical educators successfully plan, construct and implement VSAQs within medical curricula.

Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis.

BACKGROUND: The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels. METHODS: A literature search was conducted in April 2020 using eight electronic databases: MEDLINE, APA PsycInfo, SCOPUS, Web of Science, EThOS, OpenGrey, PROSPERO, and the UCAT website. Data were extracted from selected studies and tabulated as results matrices. A narrative synthesis was performed. RESULTS: Twenty-four studies satisfied our inclusion criteria, 23 of which were deemed to be of good quality (using the Newcastle-Ottawa Scale). For over 70% of univariate data points, the UCAT exerted no statistically significant predictive validity; for the remainder, predictive power was weak. The cognitive total and verbal reasoning subtests had the largest evidence base as weakly positive predictors of academic performance. The SJT subtest was a weak predictor of professional behaviour during medical school. Studies specific to dental schools demonstrated variable findings across the five studies. Only 1 study looked at post-graduate outcome measures and demonstrated that the UCAT was not a predictor of health- or conduct-related fitness to practice declarations at GMC registration. CONCLUSIONS: These data provide some support for the use of cognitive total and verbal reasoning subtests as part of medical school selection. Further research is needed to investigate outcomes beyond professional registration and for dental students.

Is Academic Attainment or Situational Judgment Test Performance in Medical School Associated With the Likelihood of Disciplinary Action? A National Retrospective Cohort Study.

PURPOSE: Disciplinary action imposed on physicians indicates their fitness to practice medicine is impaired and patient safety is potentially at risk. This national retrospective cohort study sought to examine whether there was an association between academic attainment or performance on a situational judgment test (SJT) in medical school and the risk of receiving disciplinary action within the first 5 years of professional practice in the United Kingdom. METHOD: The authors included data from the UK Medical Education Database for 34,865 physicians from 33 U.K. medical schools that started the UK Foundation Programme (similar to internship) between 2014 and 2018. They analyzed data from 2 undergraduate medical assessments used in the United Kingdom: the Educational Performance Measure (EPM), which is based on academic attainment, and SJT, which is an assessment of professional attributes. The authors calculated hazard ratios (HRs) for EPM and SJT scores. RESULTS: The overall rate of disciplinary action was low (65/34,865, 0.19%) and the mean time to discipline was 810 days (standard deviation [SD] = 440). None of the physicians with fitness to practice concerns identified as students went on to receive disciplinary action after they qualified as physicians. The multivariate survival analysis demonstrated that a score increase of 1 SD (approximately 7.6 percentage points) on the EPM reduced the hazard of disciplinary action by approximately 50% (HR = 0.51; 95% confidence interval [CI]: 0.38, 0.69; P < .001). There was not a statistically significant association between the SJT score and the hazard of disciplinary action (HR = 0.84; 95% CI: 0.62, 1.13; P = .24). CONCLUSIONS: An increase in EPM score was significantly associated with a reduced hazard of disciplinary action, whereas performance on the SJT was not. Early identification of increased risk of disciplinary action may provide an opportunity for remediation and avoidance of patient harm.

A remote access mixed reality teaching ward round.

BACKGROUND: Heterogeneous access to clinical learning opportunities and inconsistency in teaching is a common source of dissatisfaction among medical students. This was exacerbated during the COVID-19 pandemic, with limited exposure to patients for clinical teaching. METHODS: We conducted a proof-of-concept study at a London teaching hospital using mixed reality (MR) technology (HoloLens2™) to deliver a remote access teaching ward round. RESULTS: Students unanimously agreed that use of this technology was enjoyable and provided teaching that was otherwise inaccessible. The majority of participants gave positive feedback on the MR (holographic) content used (n = 8 out of 11) and agreed they could interact with and have their questions answered by the clinician leading the ward round (n = 9). Quantitative and free text feedback from students, patients and faculty members demonstrated that this is a feasible, acceptable and effective method for delivery of clinical education. DISCUSSION: We have used this technology in a novel way to transform the delivery of medical education and enable consistent access to high-quality teaching. This can now be integrated across the curriculum and will include remote access to specialist clinics and surgery. A library of bespoke MR educational resources will be created for future generations of medical students and doctors to use on an international scale.

Question-Based Collaborative Learning for Constructive Curricular Alignment.

INTRODUCTION: We designed a curriculum mapping tool which enables medical students to access intended learning outcomes (ILOs) on their iPads in the workplace. Students were encouraged to use the online curriculum map in a specially planned teaching session: question-based collaborative learning (QBCL). The aim of the session was to empower medical students to constructively align their experiential learning with the learning outcomes of the undergraduate curriculum. In doing so, our session aimed to provide students with a greater understanding of the curriculum, improve their insights into assessment and their question-writing abilities. METHODS: The QBCL pre-session preparation involved reviewing a patient with a presentation that aligned to the year-specific ILOs. During a 150 minute QBCL session, students received training on how to write high quality multiple choice questions (MCQs) delivered by a faculty member of Imperial College School of Medicine. They then worked collaboratively in groups and created MCQs based on their clinical encounters. Their questions were tagged to the relevant learning objective and submitted online via the curriculum map. The student-generated MCQs were analyzed using an adjusted version of Bloom's taxonomy. We also conducted a quantitative evaluation of the session. RESULTS: One hundred and sixty-three questions were submitted, with 81% of questions being tagged to ILOs considered to show evidence of learning consistent with the "Apply" tier of Bloom's taxonomy. The majority of students agreed that the session was interactive (80%), thought-provoking (77%) and improved their team-working skills (70%). It gave them a greater understanding of the undergraduate curriculum (65%), improved their question-writing and insight into assessments (76%), and provided an opportunity to learn from their peers (86%). Students agreed that this session covered a variety of cases (82%) and deepened their understanding of medical conditions and presentations (87%). CONCLUSION: We encouraged students to actively interact with the curriculum map. Students were able to achieve their own constructive alignment by writing assessment items based on real patients and linking them to the appropriate intended learning outcomes.