Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100.

Why use SJTs? Traditionally, selection into medical education professions has focused primarily upon academic ability alone. This approach has been questioned more recently, as although academic attainment predicts performance early in training, research shows it has less predictive power for demonstrating competence in postgraduate clinical practice. Such evidence, coupled with an increasing focus on individuals working in healthcare roles displaying the core values of compassionate care, benevolence and respect, illustrates that individuals should be selected on attributes other than academic ability alone. Moreover, there are mounting calls to widen access to medicine, to ensure that selection methods do not unfairly disadvantage individuals from specific groups (e.g. regarding ethnicity or socio-economic status), so that the future workforce adequately represents society as a whole. These drivers necessitate a method of assessment that allows individuals to be selected on important non-academic attributes that are desirable in healthcare professionals, in a fair, reliable and valid way. What are SJTs? Situational judgement tests (SJTs) are tests used to assess individuals' reactions to a number of hypothetical role-relevant scenarios, which reflect situations candidates are likely to encounter in the target role. These scenarios are based on a detailed analysis of the role and should be developed in collaboration with subject matter experts, in order to accurately assess the key attributes that are associated with competent performance. From a theoretical perspective, SJTs are believed to measure prosocial Implicit Trait Policies (ITPs), which are shaped by socialisation processes that teach the utility of expressing certain traits in different settings such as agreeable expressions (e.g. helping others in need), or disagreeable actions (e.g. advancing ones own interest at others, expense). Are SJTs reliable, valid and fair? Several studies, including good quality meta-analytic and longitudinal research, consistently show that SJTs used in many different occupational groups are reliable and valid. Although there is over 40 years of research evidence available on SJTs, it is only within the past 10 years that SJTs have been used for recruitment into medicine. Specifically, evidence consistently shows that SJTs used in medical selection have good reliability, and predict performance across a range of medical professions, including performance in general practice, in early years (foundation training as a junior doctor) and for medical school admissions. In addition, SJTs have been found to have significant added value (incremental validity) over and above other selection methods such as knowledge tests, measures of cognitive ability, personality tests and application forms. Regarding differential attainment, generally SJTs have been found to have lower adverse impact compared to other selection methods, such as cognitive ability tests. SJTs have the benefit of being appropriate both for use in selection where candidates are novices (i.e. have no prior role experience or knowledge such as in medical school admissions) as well as settings where candidates have substantial job knowledge and specific experience (as in postgraduate recruitment for more senior roles). An SJT specification (e.g. scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT. Once designed however, SJTs are cost efficient to administer to large numbers of candidates compared to other tests of non-academic attributes (e.g. personal statements, structured interviews), as they are standardised and can be computer-delivered and machine-marked.

Situational judgment tests reliably measure professional attributes important for clinical practice.

Over the course of more than 40 years, international research has consistently shown situational judgment tests (SJTs) to be a reliable and valid selection method for assessing a range of professional attributes. However, SJTs still represent a relatively new selection method within the medical profession, and as such it is to be expected that applicant reactions will vary. In this Expert Opinion piece, we respond to Najim et al's article "The situational judgement test: a student's worst nightmare" by highlighting three key clarifications. We outline that 1) the UK Foundation Programme's SJT deliberately measures only a subset (five) of the nine professional attributes important for the role of Foundation Trainee doctor, 2) these attributes are measured in addition to academic attainment, and 3) the SJT represents a cost-effective approach to selection rather than attempting to interview approximately 8,000 candidates each year, which would be logistically impossible. We present these points to inform future research and encourage debate, and conclude that the SJT is an appropriate and fair measurement method to be used as one part of selection to the UK Foundation Programme.

A new competency model for general practice: implications for selection, training, and careers.

BACKGROUND: Recent structural and policy changes in the UK health service have significantly changed the job responsibilities for the GP role. AIM: To replicate a previous job analysis study to examine the relevance of current competency domains and selection criteria for doctors entering training. DESIGN AND METHOD: A multisource, multimethod approach comprising three phases: (1) stakeholder consultation (n = 205) using interviews, focus groups and behavioural observation of practising GPs; (2) a validation questionnaire based on results from phase 1 (n = 1082); followed by (3) an expert panel (n = 6) to review and confirm the final competency domains. RESULTS: Eleven competency domains were identified, which extends previous research findings. A new domain was identified called Leading for Continuing Improvement. Results show that, Empathy and Perspective Taking, Communication Skills, Clinical Knowledge and Expertise, and Professional Integrity are currently rated the most important domains. Results indicate a significant increase in ratings of importance for each domain in the future (P<0.001), except for Communication Skills and Empathy and Perspective Taking, which consistently remain high. CONCLUSION: The breadth of competencies required for GPs has increased significantly. GPs are now required to resolve competing tensions to be effective in their role, such as maintaining a patient focus while overseeing commissioning, with a potential ethical conflict between these aspects. Selection criteria remain largely unchanged but with increased priority in some domains (for example, Effective Teamworking). However, there is an urgent need to review the training provision arrangements to reflect the greater breadth of competencies now required.