Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment.

BACKGROUND: Fairness is a critical component of defensible assessment. Candidates should perform according to ability without influence from background characteristics such as ethnicity or sex. However, performance differs by candidate background in many assessment environments. Many potential causes of such differences exist, and examinations must be routinely analysed to ensure they do not present inappropriate progression barriers for any candidate group. By analysing the individual questions of an examination through techniques such as Differential Item Functioning (DIF), we can test whether a subset of unfair questions explains group-level differences. Such items can then be revised or removed. METHODS: We used DIF to investigate fairness for 13,694 candidates sitting a major international summative postgraduate examination in internal medicine. We compared (a) ethnically white UK graduates against ethnically non-white UK graduates and (b) male UK graduates against female UK graduates. DIF was used to test 2773 questions across 14 sittings. RESULTS: Across 2773 questions eight (0.29%) showed notable DIF after correcting for multiple comparisons: seven medium effects and one large effect. Blinded analysis of these questions by a panel of clinician assessors identified no plausible explanations for the differences. These questions were removed from the question bank and we present them here to share knowledge of questions with DIF. These questions did not significantly impact the overall performance of the cohort. Group-level differences in performance between the groups we studied in this examination cannot be explained by a subset of unfair questions. CONCLUSIONS: DIF helps explore fairness in assessment at the question level. This is especially important in high-stakes assessment where a small number of unfair questions may adversely impact the passing rates of some groups. However, very few questions exhibited notable DIF so differences in passing rates for the groups we studied cannot be explained by unfairness at the question level.

Passing MRCP (UK) PACES: a cross-sectional study examining the performance of doctors by sex and country.

BACKGROUND: There is much discussion about the sex differences that exist in medical education. Research from the United Kingdom (UK) and United States has found female doctors earn less, and are less likely to be senior authors on academic papers, but female doctors are also less likely to be sanctioned, and have been found to perform better academically and clinically. It is also known that international medical graduates tend to perform more poorly academically compared to home-trained graduates in the UK, US, and Canada. It is uncertain whether the magnitude and direction of sex differences in doctors' performance is variable by country. We explored the association between doctors' sex and their performance at a large international high-stakes clinical examination: the Membership of the Royal Colleges of Physicians (UK) Practical Assessment of Clinical Examination Skills (PACES). We examined how sex differences varied by the country in which the doctor received their primary medical qualification, the country in which they took the PACES examination, and by the country in which they are registered to practise. METHODS: Seven thousand six hundred seventy-one doctors attempted PACES between October 2010 and May 2013. We analysed sex differences in first time pass rates, controlling for ethnicity, in three groups: (i) UK medical graduates (N = 3574); (ii) non-UK medical graduates registered with the UK medical regulator, the General Medical Council (GMC), and thus likely to be working in the UK (N = 1067); and (iii) non-UK medical graduates without GMC registration and so legally unable to work or train in the UK (N = 2179). RESULTS: Female doctors were statistically significantly more likely to pass at their first attempt in all three groups, with the greatest sex effect seen in non-UK medical graduates without GMC registration (OR = 1.99; 95% CI = 1.65-2.39; P < 0.0001) and the smallest in the UK graduates (OR = 1.18; 95% CI = 1.03-1.35; P = 0.02). CONCLUSIONS: As found in a previous format of this examination and in other clinical examinations, female doctors outperformed male doctors. Further work is required to explore why sex differences were greater in non-UK graduates, especially those without GMC registration, and to consider how examination performance may relate to performance in practice.

WITHDRAWN: Electrical cardioversion for atrial fibrillation and flutter.

BACKGROUND: Atrial fibrillation increases stroke risk and adversely affects cardiovascular haemodynamics. Electrical cardioversion may, by restoring sinus rhythm, improve cardiovascular haemodynamics, reduce the risk of stroke, and obviate the need for long-term anticoagulation. OBJECTIVES: To assess the effects of electrical cardioversion of atrial fibrillation or flutter on the risk of thromboembolic events, strokes and mortality (primary outcomes), the rate of cognitive decline, quality of life, the use of anticoagulants and the risk of re-hospitalisation (secondary outcomes) in adults (>18 years). SEARCH METHODS: We searched the Cochrane CENTRAL Register of Controlled Trials (1967 to May 2004), MEDLINE (1966 to May 2004), Embase (1980 to May 2004), CINAHL (1982 to May 2004), proceedings of the American College of Cardiology (published in Journal of the American College of Cardiology 1983 to 2003), www.trialscentral.org, www.controlled-trials.com and reference lists of articles. We hand-searched the indexes of the Proceedings of the British Cardiac Society published in British Heart Journal (1980 to 1995) and in Heart (1995 to 2002); proceedings of the European Congress of Cardiology and meetings of the Joint Working Groups of the European Society of Cardiology (published in European Heart Journal 1983-2003); scientific sessions of the American Heart Association (published in Circulation 1990-2003). Personal contact was made with experts. SELECTION CRITERIA: Randomised controlled trial or controlled clinical trials of electrical cardioversion plus 'usual care' versus 'usual care' only, where 'usual care' included any combination of anticoagulants, antiplatelet drugs and drugs for 'rate control'. We excluded trials which used pharmacological cardioversion as the first intervention, and trials of new onset atrial fibrillation after cardiac surgery. There were no language restrictions. DATA COLLECTION AND ANALYSIS: For dichotomous data, odds ratios were calculated; and for continuous data, the weighted mean difference was calculated. MAIN RESULTS: We found three completed trials of electrical cardioversion (rhythm control) versus rate control, recruiting a total of 927 participants (Hot Cafe; RACE; STAF) and one ongoing trial (J-RHYTHM). There was no difference in mortality between the two strategies (OR 0.83; CI 0.48 to 1.43). There was a trend towards more strokes in the rhythm control group (OR 1.9; 95% CI 0.99 to 3.64). At follow up, three domains of quality of life (physical functioning, physical role function and vitality) were significantly better in the rhythm control group (RACE 2002; STAF 2003). AUTHORS' CONCLUSIONS: Electrical cardioversion (rhythm control) led to a non-significant increase in stroke risk but improved three domains of quality of life.

Balint group leadership: Conceptual foundations and a framework for leadership development?

This article is based on a talk given at the International Balint Federation leadership congress in Warsaw (September 2016). It explores the conceptual foundations of Balint group leadership and starts by emphasizing the mutuality between psychoanalysts and medical practitioners working together. The parallel process between consulting room and group, and subsequently from group back to the consulting room, is delineated as the central construct in understanding the role of the leader. Having proposed a conceptual model for thinking about leadership interventions, the article discusses developments in some contemporary approaches to leadership: coleadership, the use of role play, psychodrama, pushback, and morale. It concludes by introducing Keats' notion of negative capability as a way of thinking about creativity in Balint group leadership.

The Value of Physical Examination: A New Conceptual Framework.

The physical examination defines medical practice, yet its role is being questioned increasingly, with statistical comparisons of diagnostic accuracy often the sole metric used against newer technologies. We set out to highlight seven ways in which the physical examination has value beyond diagnostic accuracy to reaffirm its place in the core skills of a physician and guide future research, teaching, and curriculum design. We show that this more comprehensive approach to the physical examination of its "utility" beyond that of reaching a diagnosis can be beneficial to both doctor and patient.

Exploring cultural and linguistic influences on clinical communication skills: a qualitative study of International Medical Graduates.

BACKGROUND: International Medical Graduates (IMGs) are known to perform less well in many postgraduate medical examinations when compared to their UK trained counterparts. This "differential attainment" is observed in both knowledge-based and clinical skills assessments. This study explored the influence of culture and language on IMGs clinical communication skills, in particular, their ability to seek, detect and acknowledge patients' concerns in a high stakes postgraduate clinical skills examination. Hofstede's cultural dimensions framework was used to look at the impact of culture on examination performance. METHODS: This was a qualitative, interpretative study using thematic content analysis of video-recorded doctor-simulated patient consultations of candidates sitting the MRCP(UK) PACES examination, at a single examination centre in November 2012. The research utilised Hofstede's cultural dimension theory, a framework for comparing cultural factors amongst different nations, to help understand the reasons for failure. RESULTS: Five key themes accounted for the majority of communication failures in station 2, "history taking" and station 4, "communication skills and ethics" of the MRCP(UK) PACES examination. Two themes, the ability to detect clues and the ability to address concerns, related directly to the overall construct managing patients' concerns. Three other themes were found to impact the whole consultation. These were building relationships, providing structure and explanation and planning. CONCLUSION: Hofstede's cultural dimensions may help to contextualise some of these observations. In some cultures doctor and patient roles are relatively inflexible: the doctor may convey less information to the patient (higher power distance societies) and give less attention to building rapport (high uncertainty avoidance societies.) This may explain why cues and concerns presented by patients were overlooked in this setting. Understanding cultural differences through Hofstede's cultural dimensions theory can inform the preparation of candidates for high stakes bedside clinical skills examinations and for professional practice.

Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP(UK) PACES and nPACES examinations.

BACKGROUND: Bias of clinical examiners against some types of candidate, based on characteristics such as sex or ethnicity, would represent a threat to the validity of an examination, since sex or ethnicity are 'construct-irrelevant' characteristics. In this paper we report a novel method for assessing sex and ethnic bias in over 2000 examiners who had taken part in the PACES and nPACES (new PACES) examinations of the MRCP(UK). METHOD: PACES and nPACES are clinical skills examinations that have two examiners at each station who mark candidates independently. Differences between examiners cannot be due to differences in performance of a candidate because that is the same for the two examiners, and hence may result from bias or unreliability on the part of the examiners. By comparing each examiner against a 'basket' of all of their co-examiners, it is possible to identify examiners whose behaviour is anomalous. The method assessed hawkishness-doveishness, sex bias, ethnic bias and, as a control condition to assess the statistical method, 'even-number bias' (i.e. treating candidates with odd and even exam numbers differently). Significance levels were Bonferroni corrected because of the large number of examiners being considered. RESULTS: The results of 26 diets of PACES and six diets of nPACES were examined statistically to assess the extent of hawkishness, as well as sex bias and ethnicity bias in individual examiners. The control (odd-number) condition suggested that about 5% of examiners were significant at an (uncorrected) 5% level, and that the method therefore worked as expected. As in a previous study (BMC Medical Education, 2006, 6:42), some examiners were hawkish or doveish relative to their peers. No examiners showed significant sex bias, and only a single examiner showed evidence consistent with ethnic bias. A re-analysis of the data considering only one examiner per station, as would be the case for many clinical examinations, showed that analysis with a single examiner runs a serious risk of false positive identifications probably due to differences in case-mix and content-specificity. CONCLUSIONS: In examinations where there are two independent examiners at a station, our method can assess the extent of bias against candidates with particular characteristics. The method would be far less sensitive in examinations with only a single examiner per station as examiner variance would be confounded with candidate performance variance. The method however works well when there is more than one examiner at a station and in the case of the current MRCP(UK) clinical examination, nPACES, found possible sex bias in no examiners and possible ethnic bias in only one.

Camouflaging the permanent maxillary canine to substitute an absent lateral incisor - part 2: challenges and solutions.

The developmental absence of permanent maxillary lateral incisors is a common developmental anomaly and presents clinical challenges. The maxillary lateral incisor is the second most common developmentally absent tooth after the mandibular second premolar, therefore can present frequently in practice. Patients with hypodontia of the maxillary lateral incisor present clinical challenges in relation to function and aesthetics. In part one of this two-part series, we focused on the assessment, diagnosis and management. In this part, we will discuss some of the frequently encountered challenges in canine camouflage cases and possible solutions.

Camouflaging the permanent maxillary canine to substitute an absent lateral incisor - part 1: assessment and management.

There remains debate as to whether, when there is a missing maxillary lateral incisor, space should be orthodontically opened to allow for prosthetic replacement or closed to camouflage the canines to substitute the absent lateral incisor by recontouring and simple restorative procedures. Each case presents its own features which will help decision making. Treatment planning these cases is multifactorial in nature and the decision ultimately depends on factors such as the malocclusion; inter-arch and intra-arch relationship; and canine properties (size, colour and shape). The ideal treatment is the most conservative one that satisfies the individual's aesthetic and functional requirements. Often, but not always, it is therefore space closure with camouflaging of the canines as lateral incisors. Multidisciplinary planning and delivery of care is essential and 'mandatory' in the delivery of optimal outcomes. The first part of this two-part series will discuss the principles of assessment, diagnosis and management. The second part will discuss the challenges encountered and possible solutions.

Changing PACES: developments to the examination in 2009.

As a consequence of change in medical, educational and regulatory practice, MRCP(UK) successfully modified the international PACES examination in 2009. This brief paper explains the rationale for change and summarises the development and implementation process.

Effect of alkyl chain length on the antioxidant activity of alkylresorcinol homologues in bulk oils and oil-in-water emulsions.

The antioxidant cut-off theory details the importance of fine-tuning antioxidant hydrophobicity to optimize antioxidant effectiveness for a given food system; however, previous research has utilized synthetic antioxidant homologues which fail to align with the food industry's demand for natural ingredients. Alkylresorcinols represent a natural homologous series of phenolipid antioxidants. The antioxidant activities of individual alkylresorcinol homologues were investigated in bulk oils and oil-in-water emulsions. In oils, antioxidant activity decreased as alkyl chain length increased and there was no effect on rate of loss. In emulsions, optimum antioxidant activity was observed at intermediate alkyl chain length (C21:0) and longer homologues were lost more rapidly. Radical scavenging capacity decreased as alkyl chain length increased but alkylresorcinols were unable to chelate iron. This suggests that intrinsic properties (e.g. radical scavenging capacity) are responsible for the antioxidant activity of alkylresorcinols in oils while physicochemical phenomena (e.g. partitioning) drive antioxidant activity of alkylresorcinols in emulsions.

Antioxidant activity of a winterized, acetonic rye bran extract containing alkylresorcinols in oil-in-water emulsions.

Naturally derived antioxidants are in high demand as the food industry strives to meet consumer preferences for non-synthetic additives. Alkylresorcinols (ARs) represent a novel class of natural antioxidants that can be derived from a natural waste stream (bran) and have the potential to inhibit lipid peroxidation given their phenolic structure. The antioxidant activity of rye bran extract containing ARs was investigated in an oil-in-water emulsion and was found to inhibit lipid oxidation reactions. The concentration of ARs in the continuous phase of emulsions was measured to understand partitioning behavior, as this is known to impact antioxidant activity. It was found that a majority of the ARs were associated with the lipid phase and those in the continuous phase were associated with surfactant micelles, perhaps inhibiting their interaction with water-soluble pro-oxidants. These results show that a rye bran extract containing ARs can function as a radical scavenging antioxidant in lipid dispersions.

Reinvigorating the clinical examination for the 21st century.

At its most fundamental level, the clinical encounter between a patient and their doctor seeks to solve a mystery. Clinicians uncover clues through the history, physical examination, and ancillary tests to arrive at a diagnosis and develop a management plan. Despite advances in technology, the majority of clinical diagnoses are still reached through the history and physical examination without the use of laboratory and imaging tests. However, in the modern American hospital, clinicians spend as little as 12% of their time in direct contact with patients and their families. This has led to a decline in clinical examination skills and contributes to diagnostic error. There is a growing movement to return clinicians and trainees back to the bedside. In 2017, we formed the Society of Bedside Medicine to encourage innovation, education, and research on the role of the clinical encounter in 21st century medicine. Over the last 3 years, we have embraced the following 6 strategies to reinvigorate the practice of the clinical examination: 1) be present with the patient; 2) practice an evidence­based approach to the physical exam; 3) create opportunities for intentional practice of the physical exam; 4) recognize the power of the physical examination beyond diagnosis; 5) use point­of­care technology to aid in diagnosis and reinforce skills; and 6) seek and provide specific feedback on physical examination skills. By employing these strategies in both teaching and practice, clinicians can maximize the value of time spent with patients and renew the importance of the clinical examination in 21st century practice.