Utility of large language models for creating clinical assessment items.

PURPOSE: To compare student performance, examiner perceptions and cost of GPT-assisted (generative pretrained transformer-assisted) clinical and professional skills assessment (CPSAs) items against items created using standard methods. METHODS: We conducted a prospective, controlled, double-blinded comparison of CPSA items developed using GPT-assistance with those created through standard methods. Two sets of six practical cases were developed for a formative assessment sat by final year medical students. One clinical case in each set was created with GPT-assistance. Students were assigned to one of the two sets. RESULTS: The results of 239 participants were analysed in the study. There was no statistically significant difference in item difficulty, or discriminative ability between GPT-assisted and standard items. One hundred percent (n = 15) of respondents to an examiner feedback questionnaire felt GPT-assisted cases were appropriately difficult and realistic. GPT-assistance resulted in significant labour cost savings, with a mean reduction of 57% (880 GBP) in labour cost per case when compared to standard case drafting methods. CONCLUSIONS: GPT-assistance can create CPSA items of comparable quality with significantly less cost when compared to standard methods. Future studies could evaluate GPT's ability to create CPSA material in other areas of clinical practice, aiming to validate the generalisability of these findings.

The influence of candidates' race on examiners' ratings in standardised assessments of clinical practice.

PURPOSE: Delivering fair and reliable summative assessments in medical education assumes examiner decision making is devoid of bias. We investigated whether candidate racial appearances influenced examiner ratings in undergraduate clinical exams. METHODS: We used an internet-based design. Examiners watched a randomised set of six videos of three different white candidates and three different non-white (Asian, black and Chinese) candidates taking a clinical history at either fail, borderline or pass grades. We compared the median and interquartile range (IQR) of the paired difference between scores for the white and non-white candidates at each performance grade and tested for statistical significance. RESULTS: 160 Examiners participated. At the fail grade, the black and Chinese candidates scored lower than the white candidate, with median paired differences of -2.5 and -1 respectively (both p < 0.001). At the borderline grade, the black and Chinese candidates scored higher than the white candidate, with median paired differences of +2 and +3, respectively (both p < 0.001). At the passing grade, the Asian candidate scored lower than the white candidate (median paired difference -1, p < 0.001). CONCLUSION: The racial appearance of candidates appeared to influence the scores awarded by examiners, but not in a uniform manner.

Health care professionals' knowledge and attitudes towards antibiotic prescribing for the treatment of urinary tract infections: A systematic review.

PURPOSE: Previous models identify knowledge and attitudes that influence prescribing behaviour. The present study focuses on antibiotic prescribing for urinary tract infections (UTIs) to describe levels of health care professionals' knowledge and attitude factors in this area and how those levels are assessed. METHODS: A systematic search was conducted to identify studies assessing the identified knowledge or attitude factors influencing health care professionals' antibiotic prescribing for urinary tract infections up to September 2022. Study quality was assessed using the Newcastle-Ottawa scale. Data were extracted about the types of factors assessed, the levels indicated and how those levels were assessed. Data were synthesized using counts, and levels were categorized as 'poor', 'moderate', 'high' or 'very high'. RESULTS: Seven studies were identified, six of which relied entirely on closed-ended items. Levels of knowledge factors assessed were poor, for example, their 'knowledge of condition' and 'knowledge of task environment' were poor. Levels of the attitude factors assessed varied, for example, while health care professionals expressed moderate confidence in providing optimal patient care and appropriate attitude of fear towards the problem of antibiotic resistance, they expressed a poor attitude of complacency by giving into patient pressure to prescribe an antibiotic. CONCLUSIONS: Present evidence suggests that clinicians have poor levels of knowledge and varying levels of attitudes about antibiotic prescribing for UTIs. However, few studies were identified, and assessments were largely limited to closed-ended types of questions. Future studies that assess more factors and employ open-ended question types could better inform future interventions to optimize antibiotic prescribing.

Mixed method evaluation of a learning from excellence programme for community health workers in Neno, Malawi.

BACKGROUND: Community Health Workers (CHWs) play an essential role in linking communities to facility-based healthcare. However, CHW programmes have often been hampered by low levels of staff motivation, and new tools aimed at improving staff motivation and work environment are needed. One such intervention is the "Learning from Excellence" (LfE) programme. We aimed to assess feasibility, outputs, and impact of a co-designed LfE programme on CHW motivation, in Neno District. METHODS: We conducted a convergent mixed-method evaluation of the LfE programme. Co-design of the programme and forms took place between October 2019 and January 2020. LfE forms submitted between September and November 2020 were analysed using descriptive statistics and memos summarising answers to the open-ended question. To investigate experiences with LfE we conducted in-depth semi-structured interviews with key stakeholders, CHWs, and site supervisors, which were analysed thematically. A pre-post intervention questionnaire was developed to assess the impact of the co-designed LfE intervention on CHW motivation and perceived supervision. Outcomes were triangulated into a logic model. RESULTS: In total 555 LfE forms were submitted, with 34.4% of CHWs in Neno District submitting at least one LfE report. Four themes were identified in the interviews: LfE implementation processes, experience, consequences, and recommendations. A total of 50 CHWs participated in the questionnaire in January 2020 and 46 of them completed the questionnaire in December 2020. No statistically significant differences were identified between pre-and post-LfE measurements for both motivation (Site F: p = 0.86; Site G: p = 0.31) and perceived supervision (Site F: p = 0.95; Site G: p = 0.45). A logic model, explaining how the LfE programme could impact CHWs was developed. CONCLUSIONS: Many CHWs participated in the LfE intervention between September 2020 and November 2020. LfE was welcomed by CHWs and stakeholders as it allowed them to appreciate excellent work in absence of other opportunities to do so. However, no statistically significant differences in CHW motivation and perceived supervision were identified. While the intervention was feasible in Neno District, we identified several barriers and facilitators for implementation. We developed a logic model to explain contextual factors, and mechanisms that could lead to LfE outcomes for CHWs in Neno District. The developed logic model can be used by those designing and implementing interventions like LfE for health workers.

Digital adaptation of the Standing up for Myself intervention in young people and adults with intellectual disabilities: the STORM feasibility study.

Background: Stigma contributes to the negative social conditions persons with intellectual disabilities are exposed to, and it needs tackling at multiple levels. Standing Up for Myself is a psychosocial group intervention designed to enable individuals with intellectual disabilities to discuss stigmatising encounters in a safe and supportive setting and to increase their self-efficacy in managing and resisting stigma. Objectives: To adapt Standing Up for Myself to make it suitable as a digital intervention; to evaluate the feasibility and acceptability of Digital Standing Up for Myself and online administration of outcome measures in a pilot; to describe usual practice in the context of the coronavirus disease 2019 pandemic to inform future evaluation. Design: Adaptation work followed by a single-arm pilot of intervention delivery. Setting and participants: Four third and education sector organisations. Individuals with mild-to-moderate intellectual disabilities, aged 16+, members of existing groups, with access to digital platforms. Intervention: Digital Standing Up for Myself intervention. Adapted from face-to-face Standing Up for Myself intervention, delivered over four weekly sessions, plus a 1-month follow-up session. Outcomes: Acceptability and feasibility of delivering Digital Standing Up for Myself and of collecting outcome and health economic measures at baseline and 3 months post baseline. Outcomes are mental well-being, self-esteem, self-efficacy in rejecting prejudice, reactions to discrimination and sense of social power. Results: Adaptation to the intervention required changes to session duration, group size and number of videos; otherwise, the content remained largely the same. Guidance was aligned with digital delivery methods and a new group member booklet was produced. Twenty-two participants provided baseline data. The intervention was started by 21 participants (four groups), all of whom were retained at 3 months. Group facilitators reported delivering the intervention as feasible and suggested some refinements. Fidelity of the intervention was good, with over 90% of key components observed as implemented by facilitators. Both facilitators and group members reported the intervention to be acceptable. Group members reported subjective benefits, including increased confidence, pride and knowing how to deal with difficult situations. Digital collection of all outcome measures was feasible and acceptable, with data completeness ≥ 95% for all measures at both time points. Finally, a picture of usual practice has been developed as an intervention comparator for a future trial. Limitations: The pilot sample was small. It remains unclear whether participants would be willing to be randomised to a treatment as usual arm or whether they could be retained for 12 months follow-up. Conclusions: The target number of groups and participants were recruited, and retention was good. It is feasible and acceptable for group facilitators with some training and supervision to deliver Digital Standing Up for Myself. Further optimisation of the intervention is warranted. Future work: To maximise the acceptability and reach of the intervention, a future trial could offer the adapted Digital Standing Up for Myself, potentially alongside the original face-to-face version of the intervention. Study registration: This study was registered as ISRCTN16056848. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/149/03) and is published in full in Public Health Research; Vol. 12, No. 1. See the NIHR Funding and Awards website for further award information.

People with intellectual disabilities (or 'learning disabilities' in United Kingdom language) are more likely to experience poor physical and mental health than the general population. Stigma (negative stereotypes, prejudice and discrimination) has been linked to lower self-esteem, quality of life, and mental and physical ill health. Efforts to empower people with intellectual disabilities themselves to challenge stigma with a view to improving well-being, health and self-esteem are lacking. In 2017, we developed Standing Up for Myself, a brief group-based programme for people with mild-to-moderate intellectual disabilities aged 16+ to address this gap. As this study got underway, face-to-face meetings were suspended due to the coronavirus disease 2019 pandemic. We used the opportunity to assess whether Standing Up for Myself could be delivered through web-based meetings. We adapted Standing Up for Myself for digital delivery, with close input from advisors with intellectual disabilities and experienced group facilitators. We then tested the digital version in charity and education settings to evaluate if Digital Standing Up for Myself could be delivered as planned and how acceptable it was to group facilitators and participants. Four groups, with a total of 22 members, signed up to try Digital Standing Up for Myself. One participant dropped out before starting Standing Up for Myself, and the other 21 continued until the end of the programme. Retention and attendance were good; participants on average attended four of the five sessions. Ninety per cent of the core programme requirements were fully delivered as detailed in the Digital Standing Up for Myself manual. Problems with technology were manageable, although facilitators found using the Standing Up for Myself Wiki platform (an online platform for storage and sharing of resources) difficult, particularly when sharing video content. Facilitators felt acceptable levels of privacy were achieved and there were no reports of undue distress. All facilitators and many group members said they would recommend Digital Standing Up for Myself to others. Group members shared how the programme benefitted them, noting increased awareness about disabilities, and for some increased confidence, pride and independence. Some had learnt how to stand up for themselves and manage difficult situations and took pride in this. Completing outcome and health cost measures via web-based meetings was acceptable and data were largely fully complete and useable.

Is the awarding gap at UK medical schools influenced by ethnicity and medical school attended? A retrospective cohort study.

OBJECTIVES: To better characterise the Awarding Gap (AG) between black, Asian and other minority ethnic (BAME) and white students in UK undergraduate medical education by examining how it affects eight minority ethnicity subgroups (Bangladeshi, black, Chinese, Indian, Pakistani, mixed, other Asian background and other ethnic background) and whether the AG varies by medical school attended. DESIGN: Retrospective cohort study. SETTING: Data extracted from the UK Medical Education Database on students enrolled at 33 UK medical schools in the academic years starting 2012, 2013 and 2014. PARTICIPANTS: 16 020 'Home' tuition fee status students who sat the University Clinical Aptitude Test on entry to university and obtained a UK Foundation Programme (UKFP) application score on exit. PRIMARY OUTCOME MEASURE: UKFP Z-scores on exit from medical school. RESULTS: There were significant differences in UKFP Z-scores between ethnicity subgroups. After white students, mixed ethnicity students performed best (coefficient -0.15 standard deviations [SD]) compared with white students, (95% confidence interval [CI] -0.23 to -0.08, p<0.001) and Pakistani students scored lowest (coefficient -0.53 SD, 95% CI -0.60 to -0.46, p<0.001). In pairwise comparisons of scores between all nine individual ethnicity subgroups, 15/36 were statistically significant. The AG varied considerably across medical schools. The largest gap showed the coefficient for BAME was -0.83 SD compared with white students (95% CI -1.18 to -0.49, p<0.001), while the smallest demonstrated no statistically significant difference in performance between BAME and white students (+0.05 SD, 95% CI -0.32 to 0.42, p=0.792). CONCLUSIONS: BAME students are significantly disadvantaged by the current UK medical education system. There are clear differences in medical school outcomes between students from different ethnicity subgroups, and the size of the AG also varies by medical school attended. Urgent and effective action must be taken to address the AG and achieve an equal learning environment for our future doctors.

Knowledge Attainment and Engagement Among Medical Students: A Comparison of Three Forms of Online Learning.

Objective: This study compared knowledge attainment and student enjoyment and engagement between clinical case vignette, patient-testimony videos and mixed reality (MR) teaching via the Microsoft HoloLens 2, all delivered remotely to third year medical students. The feasibility of conducting MR teaching on a large scale was also assessed. Setting & Participants: Medical students in Year 3 at Imperial College London participated in three online teaching sessions, one in each format. All students were expected to attend these scheduled teaching sessions and to complete the formative assessment. Inclusion of their data used as part of the research trial was optional. Primary and Secondary Outcome Measures: The primary outcome measure was performance on a formative assessment, which served to compare knowledge attainment between three forms of online learning. Moreover, we aimed to explore student engagement with each form of learning via a questionnaire, and also feasibility of applying MR as a teaching tool on a large scale. Comparisons between performances on the formative assessment between the three groups were investigated using a repeated measures two-way ANOVA. Engagement and enjoyment were also analysed in the same manner. Results: A total of 252 students participated in the study. Knowledge attainment of students using MR was comparable with the other two methods. Participants reported higher enjoyment and engagement (p<0.001) for the case vignette method, compared with MR and video-based teaching. There was no difference in enjoyment or engagement ratings between MR and the video-based methods. Conclusion: This study demonstrated that the implementation of MR is an effective, acceptable, and feasible way of teaching clinical medicine to undergraduate students on a large scale. However, case-based tutorials were found to be favoured most by students. Future work could further explore the best uses for MR teaching within the medical curriculum.

The impact of patient skin colour on diagnostic ability and confidence of medical students.

Previous literature has explored unconscious racial biases in clinical education and medicine, finding that people with darker skin tones can be underrepresented in learning resources and managed differently in a clinical setting. This study aimed to examine whether patient skin colour can affect the diagnostic ability and confidence of medical students, and their cognitive reasoning processes. We presented students with 12 different clinical presentations on both white skin (WS) and non-white skin (NWS). A think aloud (TA) study was conducted to explore students' cognitive reasoning processes (n = 8). An online quiz was also conducted where students submitted a diagnosis and confidence level for each clinical presentation (n = 185). In the TA interviews, students used similar levels of information gathering and analytical reasoning for each skin type but appeared to display increased uncertainty and reduced non-analytical reasoning methods for the NWS images compared to the WS images. In the online quiz, students were significantly more likely to accurately diagnose five of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema and meningococcal disease) on WS compared to NWS (p < 0.01). With regards to students' confidence, they were significantly more confident diagnosing eight of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema, meningococcal disease, urticaria, chickenpox and Kawasaki disease) on WS when compared to NWS (p < 0.01). These findings highlight the need to improve teaching resources to include a greater diversity of skin colours exhibiting clinical signs, to improve students' knowledge and confidence, and ultimately, to avoid patients being misdiagnosed due to the colour of their skin.

Twelve tips for introducing very short answer questions (VSAQs) into your medical curriculum.

Most undergraduate written examinations use multiple-choice questions, such as single best answer questions (SBAQs) to assess medical knowledge. In recent years, a strong evidence base has emerged for the use of very short answer questions (VSAQs). VSAQs have been shown to be an acceptable, reliable, discriminatory, and cost-effective assessment tool in both formative and summative undergraduate assessments. VSAQs address many of the concerns raised by educators using SBAQs including inauthentic clinical scenarios, cueing and test-taking behaviours by students, as well as the limited feedback SBAQs provide for both students and teachers. The widespread use of VSAQs in medical assessment has yet to be adopted, possibly due to lack of familiarity and experience with this assessment method. The following twelve tips have been constructed using our own practical experience of VSAQs alongside supporting evidence from the literature to help medical educators successfully plan, construct and implement VSAQs within medical curricula.

Is the assumption of equal distances between global assessment categories used in borderline regression valid?

BACKGROUND: Standard setting for clinical examinations typically uses the borderline regression method to set the pass mark. An assumption made in using this method is that there are equal intervals between global ratings (GR) (e.g. Fail, Borderline Pass, Clear Pass, Good and Excellent). However, this assumption has never been tested in the medical literature to the best of our knowledge. We examine if the assumption of equal intervals between GR is met, and the potential implications for student outcomes. METHODS: Clinical finals examiners were recruited across two institutions to place the typical 'Borderline Pass', 'Clear Pass' and 'Good' candidate on a continuous slider scale between a typical 'Fail' candidate at point 0 and a typical 'Excellent' candidate at point 1. Results were analysed using one-sample t-testing of each interval to an equal interval size of 0.25. Secondary data analysis was performed on summative assessment scores for 94 clinical stations and 1191 medical student examination outcomes in the final 2 years of study at a single centre. RESULTS: On a scale from 0.00 (Fail) to 1.00 (Excellent), mean examiner GRs for 'Borderline Pass', 'Clear Pass' and 'Good' were 0.33, 0.55 and 0.77 respectively. All of the four intervals between GRs (Fail-Borderline Pass, Borderline Pass-Clear Pass, Clear Pass-Good, Good-Excellent) were statistically significantly different to the expected value of 0.25 (all p-values < 0.0125). An ordinal linear regression using mean examiner GRs was performed for each of the 94 stations, to determine pass marks out of 24. This increased pass marks for all 94 stations compared with the original GR locations (mean increase 0.21), and caused one additional fail by overall exam pass mark (out of 1191 students) and 92 additional station fails (out of 11,346 stations). CONCLUSIONS: Although the current assumption of equal intervals between GRs across the performance spectrum is not met, and an adjusted regression equation causes an increase in station pass marks, the effect on overall exam pass/fail outcomes is modest.

Standard setting Very Short Answer Questions (VSAQs) relative to Single Best Answer Questions (SBAQs): does having access to the answers make a difference?

BACKGROUND: We investigated whether question format and access to the correct answers affect the pass mark set by standard-setters on written examinations. METHODS: Trained educators used the Angoff method to standard set two 50-item tests with identical vignettes, one in a single best answer question (SBAQ) format (with five answer options) and the other in a very short answer question (VSAQ) format (requiring free text responses). Half the participants had access to the correct answers and half did not. The data for each group were analysed to determine if the question format or having access to the answers affected the pass mark set. RESULTS: A lower pass mark was set for the VSAQ test than the SBAQ test by the standard setters who had access to the answers (median difference of 13.85 percentage points, Z = -2.82, p = 0.002). Comparable pass marks were set for the SBAQ test by standard setters with and without access to the correct answers (60.65% and 60.90% respectively). A lower pass mark was set for the VSAQ test when participants had access to the correct answers (difference in medians -13.75 percentage points, Z = 2.46, p = 0.014). CONCLUSIONS: When given access to the potential correct answers, standard setters appear to appreciate the increased difficulty of VSAQs compared to SBAQs.

Assessing the Utility of a Quality-of-Care Assessment Tool Used in Assessing Comprehensive Care Services Provided by Community Health Workers in South Africa.

Background: Few studies exist on the tools for assessing quality-of-care of community health worker (CHW) who provide comprehensive care, and for available tools, evidence on the utility is scanty. We aimed to assess the utility components of a previously-reported quality-of-care assessment tool developed for summative assessment in South Africa. Methods: In two provinces, we used ratings by 21 CHWs and three team leaders in two primary health care facilities per province regarding whether the tool covered everything that happens during their household visits and whether they were happy to be assessed using the tool (acceptability and face validity), to derive agreement index (≥85%, otherwise the tool had to be revised). A panel of six experts quantitatively validated 11 items of the tool (content validity). Content validity index (CVI), of individual items (I-CVI) or entire scale (S-CVI), should be >80% (excellent). For the inter-rater reliability (IRR), we determined agreement between paired observers' assigned quality-of-care messages and communication scores during 18 CHW household visits (nine households per site). Bland and Altman plots and multilevel model analysis, for clustered data, were used to assess IRR. Results: In all four CHW and team leader sites, agreement index was ≥85%, except for whether they were happy to be assessed using the tool, where it was <85% in one facility. The I-CVI of the 11 items in the tool ranged between 0.83 and 1.00. For the S-CVI, all six experts agreed on relevancy (universal agreement) in eight of 11 items (0.72) whereas the average of I-CVIs, was 0.95. The Bland-Altman plot limit of agreements between paired observes were -0.18 to 0.44 and -0.30 to 0.44 (messages score); and -0.22 to 0.45 and -0.28 to 0.40 (communication score). Multilevel modeling revealed an estimated reliability of 0.77 (messages score) and 0.14 (communication score). Conclusion: The quality-of-care assessment tool has a high face and content validity. IRR was substantial for quality-of-care messages but not for communication score. This suggests that the tool may only be useful in the formative assessment of CHWs. Such assessment can provide the basis for reflection and discussion on CHW performance and lead to change.

Instruments used to measure knowledge and attitudes of healthcare professionals towards antibiotic use for the treatment of urinary tract infections: A systematic review.

BACKGROUND: Urinary tract infections (UTIs) are the second most common condition (after upper respiratory tract infections) for which adults receive antibiotics, and this prevalence may contribute to antibiotic resistance. Knowledge and attitudes have been identified as potential determinants of antibiotic prescribing behaviour among healthcare professionals in the treatment and management of UTIs. An instrument that captures prescribers' baseline knowledge of and attitudes towards antibiotic prescribing for UTIs could inform interventions to enhance prescribing. The current systematic review evaluates the psychometric properties of instruments already available and describes the theoretical constructs they measure. METHODS: Five electronic databases were searched for published studies and instruments. The Consensus-based Standards for the selection of health status Measurement Instruments checklist was used to assess the psychometric quality reporting of the instruments. The items included in each instrument were mapped onto the theoretical constructs underlying knowledge and attitudes using a mixed-theoretical model developed for this study. RESULTS: Fourteen studies met the review inclusion criteria. All instruments were available for review. None of the instruments had all the psychometric properties evaluated. Most of the instruments sought to identify knowledge and/or attitude factors influencing antibiotic prescribing for UTIs rather than to measure/assess knowledge and attitudes. CONCLUSIONS: Few instruments for the assessment of knowledge and attitudes of healthcare professionals towards antibiotic use and UTI treatment are available. None of the instruments underwent the full development process to ensure that all psychometric properties were met. Furthermore, none of the instruments assessed all domains of knowledge and attitudes. Therefore, the ability of the instruments to provide a robust measurement of knowledge and attitudes is doubtful. There is a need for an instrument that fully and accurately measures the constructs of knowledge and attitude of healthcare professionals in the treatment of UTIs.

Using very short answer errors to guide teaching.

BACKGROUND: Student performance in examinations reflects on both teaching and student learning. Very short answer questions require students to provide a self-generated response to a question of between one and five words, which removes the cueing effects of single best answer format examinations while still enabling efficient machine marking. The aim of this study was to pilot a method of analysing student errors in an applied knowledge test consisting of very short answer questions, which would enable identification of common areas that could potentially guide future teaching. METHODS: We analysed the incorrect answers given by 1417 students from 20 UK medical schools in a formative very short answer question assessment delivered online. FINDINGS: The analysis identified four predominant types of error: inability to identify the most important abnormal value, over or unnecessary investigation, lack of specificity of radiology requesting and over-reliance on trigger words. CONCLUSIONS: We provide evidence that an additional benefit to the very short answer question format examination is that analysis of errors is possible. Further assessment is required to determine if altering teaching based on the error analysis can lead to improvements in student performance.

Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis.

BACKGROUND: The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels. METHODS: A literature search was conducted in April 2020 using eight electronic databases: MEDLINE, APA PsycInfo, SCOPUS, Web of Science, EThOS, OpenGrey, PROSPERO, and the UCAT website. Data were extracted from selected studies and tabulated as results matrices. A narrative synthesis was performed. RESULTS: Twenty-four studies satisfied our inclusion criteria, 23 of which were deemed to be of good quality (using the Newcastle-Ottawa Scale). For over 70% of univariate data points, the UCAT exerted no statistically significant predictive validity; for the remainder, predictive power was weak. The cognitive total and verbal reasoning subtests had the largest evidence base as weakly positive predictors of academic performance. The SJT subtest was a weak predictor of professional behaviour during medical school. Studies specific to dental schools demonstrated variable findings across the five studies. Only 1 study looked at post-graduate outcome measures and demonstrated that the UCAT was not a predictor of health- or conduct-related fitness to practice declarations at GMC registration. CONCLUSIONS: These data provide some support for the use of cognitive total and verbal reasoning subtests as part of medical school selection. Further research is needed to investigate outcomes beyond professional registration and for dental students.

A household-based community health worker programme for non-communicable disease, malnutrition, tuberculosis, HIV and maternal health: a stepped-wedge cluster randomised controlled trial in Neno District, Malawi.

BACKGROUND: Community health worker (CHW) programmes are a valuable component of primary care in resource-poor settings. The evidence supporting their effectiveness generally shows improvements in disease-specific outcomes relative to the absence of a CHW programme. In this study, we evaluated expanding an existing HIV and tuberculosis (TB) disease-specific CHW programme into a polyvalent, household-based model that subsequently included non-communicable diseases (NCDs), malnutrition and TB screening, as well as family planning and antenatal care (ANC). METHODS: We conducted a stepped-wedge cluster randomised controlled trial in Neno District, Malawi. Six clusters of approximately 20 000 residents were formed from the catchment areas of 11 healthcare facilities. The intervention roll-out was staggered every 3 months over 18 months, with CHWs receiving a 5-day foundational training for their new tasks and assigned 20-40 households for monthly (or more frequent) visits. FINDINGS: The intervention resulted in a decrease of approximately 20% in the rate of patients defaulting from chronic NCD care each month (-0.8 percentage points (pp) (95% credible interval: -2.5 to 0.5)) while maintaining the already low default rates for HIV patients (0.0 pp, 95% CI: -0.6 to 0.5). First trimester ANC attendance increased by approximately 30% (6.5pp (-0.3, 15.8)) and paediatric malnutrition case finding declined by 10% (-0.6 per 1000 (95% CI -2.5 to 0.8)). There were no changes in TB programme outcomes, potentially due to data challenges. INTERPRETATION: CHW programmes can be successfully expanded to more comprehensively address health needs in a population, although programmes should be carefully tailored to CHW and health system capacity.