Estimating and comparing the reliability of a suite of workplace-based assessments: an obstetrics and gynaecology setting.

This paper reports on a study that compares estimates of the reliability of a suite of workplace based assessment forms as employed to formatively assess the progress of trainee obstetricians and gynaecologists. The use of such forms of assessment is growing nationally and internationally in many specialties, but there is little research evidence on comparisons by procedure/competency and form-type across an entire specialty. Generalisability theory combined with a multilevel modelling approach is used to estimate variance components, G-coefficients and standard errors of measurement across 13 procedures and three form-types (mini-CEX, OSATS and CbD). The main finding is that there are wide variations in the estimates of reliability across the forms, and that therefore the guidance on assessment within the specialty does not always allow for enough forms per trainee to ensure that the levels of reliability of the process is adequate. There is, however, little evidence that reliability varies systematically by form-type. Methodologically, the problems of accurately estimating reliability in these contexts through the calculation of variance components and, crucially, their associated standard errors are considered. The importance of the use of appropriate methods in such calculations is emphasised, and the unavoidable limitations of research in naturalistic settings are discussed.

Patient involvement in teaching and assessing intimate examination skills: a systematic review.

OBJECTIVES: There are recognised difficulties in teaching and assessing intimate examination skills that relate to the sensitive nature of the various examinations and the anxiety faced by novice learners. This systematic review provides a summary of the evidence for the involvement of real patients (RPs) and simulated patients (SPs) in the training of health care professionals in intimate examination skills. METHODS: For the review, 'intimate examinations' included pelvic, breast, testicular and rectal examinations. Major databases were searched from the start of the database to December 2008. The synthesis of findings is integrated by narrative structured to address the main research questions, which sought to establish: the objectives of programmes involving RPs and SPs as teachers of intimate examination skills; reasons why SPs have been involved in this training; the evidence for the effectiveness of such training programmes; the evidence for measures of anxiety in students learning how to perform intimate examinations; how well issues of sexuality are addressed in the literature; any reported negative effects of involvement in teaching on the patients, and suggestions for practical strategies for involving patients in the teaching of intimate examination skills. RESULTS: A total of 65 articles were included in the review. Involving patients in teaching intimate examination skills offers advantages over traditional methods of teaching. Objective evidence for the effectiveness of this method is demonstrated through improved clinical performance, reduced anxiety and positive evaluation of programmes. Practical strategies for implementing such programmes are also reported. CONCLUSIONS: There is evidence of a short-term positive impact of patient involvement in the teaching and assessment of intimate examination skills; however, evidence of longer-term impact is still limited. The influences of sexuality and anxiety related to such examinations are explored to some extent, but the psychological impact on learners and patients is not well addressed.

Obstetric services in the UK: what we need now.

Pregnancy and childbirth should be a safe and memorable experience for both women and their partners. This article discusses the current position of maternity services, their strengths and limitations, and suggests future directions required to improve these services further.