UK Prescribing Safety Assessment (PSA): The development, implementation and outcomes of a national online prescribing assessment.

AIMS: The United Kingdom (UK) Prescribing Safety Assessment (PSA) is a 2-h online assessment of basic competence to prescribe and supervise the use of medicines. It has been undertaken by students and doctors in UK medical and foundation schools for the past decade. This study describes the academic characteristics and performance of the assessment; longitudinal performance of candidates and schools; stakeholder feedback; and surrogate markers of prescribing safety in UK healthcare practice. METHODS: We reviewed the performance data generated by over 70 000 medical students and 3700 foundation doctors who have participated in the PSA since its inception in 2013. These data were supplemented by Likert scale and free text feedback from candidates and a variety of stakeholder groups. Further data on medication incidents, collected by national reporting systems and the regulatory body, are reported, with permission. RESULTS: We demonstrate the feasibility, high quality and reliability of an online prescribing assessment, uniquely providing a measure of prescribing competence against a national standard. Over 90% of candidates pass the PSA on their first attempt, while a minority are identified for further training and assessment. The pass rate shows some variation between different institutions and between undergraduate and foundation cohorts. Most responders to a national survey agreed that the PSA is a useful instrument for assessing prescribing competence, and an independent review has recommended adding the PSA to the Medical Licensing Assessment. Surrogate markers suggest there has been improvement in prescribing safety in practice, temporally associated with the introduction of the PSA but other factors could be influential too. CONCLUSIONS: The PSA is a practical and cost-effective way of delivering a reliable national assessment of prescribing competence that has educational impact and is supported by the majority of stakeholders. There is a need to develop national systems to identify and report prescribing errors and the harm they cause, enabling the impact of educational interventions to be measured.

Candidates undertaking (invigilated) assessment online show no differences in performance compared to those undertaking assessment offline.

BACKGROUND: Medical education has historically relied on high stakes knowledge tests sat in examination centres with invigilators monitoring academic malpractice. The COVID-19 pandemic has made such examination formats impossible, and medical educators have explored the use of online assessments as a potential replacement. This shift has in turn led to fears that the change in format or academic malpractice might lead to considerably higher attainment scores on online assessment with no underlying improvement in student competence. METHOD: Here, we present an analysis of 8092 sittings of the Prescribing Safety Assessment (PSA), an assessment designed to test the prescribing skills of final year medical students in the UK. In-person assessments for the PSA were cancelled partway through the academic year 2020, with 6048 sittings delivered in an offline, traditionally invigilated format, and then 2044 sittings delivered in an online, webcam invigilated format. RESULTS: A comparison (able to detect very small effects) showed no attainment gap between online (M = 0.762, SD = 0.34) and offline (M = 0.761, SD = 0.34) performance. CONCLUSIONS: The finding suggests that the transition to online assessment does not affect student performance. The findings should increase confidence in the use of online testing in high-stakes assessment.

Prescribing Safety Assessment 2016: Delivery of a national prescribing assessment to 7343 UK final-year medical students.

AIMS: Newly graduated doctors write a large proportion of prescriptions in UK hospitals but recent studies have shown that they frequently make prescribing errors. The prescribing safety assessment (PSA) has been developed as an assessment of competence in relation to prescribing and supervising the use of medicines. This report describes the delivery of the PSA to all UK final-year medical students in 2016 (PSA2016). METHODS: The PSA is a 2-hour online assessment comprising eight sections which cover various aspects of prescribing defined within the outcomes of undergraduate education identified by the UK General Medical Council. Students sat one of four PSA 'papers', which had been standard-set using a modified Angoff process. RESULTS: A total of 7343 final-year medical students in all 31 UK medical schools sat the PSA. The overall pass rate was 95% with the pass rates for the individual papers ranging from 93 to 97%. The PSA was re-sat by 261 students who had failed and 80% of those candidates passed. The internal consistency (Cronbach's alpha) of the four papers ranged from 0.74 to 0.77 (standard error of measurement 4.13-4.24%). There was a statistically significant variation in performance between medical school cohorts (F = 32.6, P < 0.001) and a strongly positive correlation in performance for individual schools between PSA2015 and PSA2016 (r = 0.79, 95% CI 0.61-0.90; P < 0.01). CONCLUSIONS: PSA2016 demonstrated the feasibility of delivering a standardized national prescribing assessment online. The vast majority of UK final-year medical students were able to meet a prespecified standard of prescribing competence.

Assessing prescribing competence.

Prescribing of medicines is the key clinical activity in the working life of most doctors. In recent years, a broad consensus regarding the necessary competencies has been achieved. Each of these is a complex mix of knowledge, judgement and skills. Surveys of those on the threshold of their medical careers have revealed widespread lack of confidence in writing prescriptions. A valid and reliable assessment of prescribing competence, separate from an overall assessment of medical knowledge and skill, would have many benefits for clinical governance and patient safety, and would provide a measure of the success of training programmes in therapeutics. Delivering such an assessment presents many challenges, not least of which are the difficulty in identifying a surrogate marker for competent prescribing in clinical practice and the challenge of ensuring that competence assessed in a controlled environment predicts performance in clinical practice. This review makes the case for an on-line OSCE as the most valid form of assessment and sets out the requirements for its development, scope, composition and delivery. It describes an on-going attempt to develop a national assessment of prescribing skills towards the end of undergraduate medical training in the UK.

Exploring the utility of the Prescribing Safety Assessment in pharmacy education in England: experiences of pre-registration trainees and undergraduate (MPharm) pharmacy students.

OBJECTIVES: (i) To provide a preliminary indication of the performance of pharmacy undergraduate students and pre-registration pharmacy trainees in the Prescribing Safety Assessment (PSA). (ii) To determine the feasibility of administering and delivering the PSA in schools of pharmacy. (iii) To examine the potential relevance of the PSA and associated training materials to pharmacy education. (iv) To assess the attitudes of the cohort towards the PSA and their readiness to prescribe. METHODS: Four schools of pharmacy in England recruited final year undergraduate pharmacy students and pre-registration pharmacy trainees undertaking training with both hospital and community pharmacy employers in their locality to undertake the PSA. Performance data and feedback from candidates were obtained. KEY FINDINGS: Pre-registration pharmacy trainees in community (n = 27) and hospital (n = 209) settings mean average scores were 86.3% and 85.3%, respectively. There was a significant performance differential between undergraduate pharmacy students (n = 397) and those in pre-registration training, with the mean average score for undergraduate students being 73.0% (t test P < 0.05). Candidates felt their current course did prepare them for the PSA, some highlighted that additional curriculum content would be needed should this become a compulsory high-stakes assessment for pharmacy trainees. The majority of candidates felt that this assessment was useful and applicable to their training. CONCLUSIONS: The PSA process and associated learning tools could be introduced to pre-registration pharmacy education to support trainees in their development towards future prescribing roles.