In-course and career outcomes predicted by medical school selection procedures based on personal qualities.

This paper reports a 40-year follow-up of 57 graduates from the initial intake to an Australian medical school who were selected on the basis of either academic criteria alone or desirable personal qualities as assessed by non-cognitive tests and an interview (with a third small group satisfying both criteria). Both students and teaching staff have remained blind to the basis for selection until the present day. Analysis of their under- and post-graduate careers indicates that 'academic' entrants were more likely to complete an intercalated BMedSci degree and to become specialists, while 'personal qualities' entrants were more likely to graduate with honours, become GPs, and win higher degrees after graduation. However, gender more significantly predicted these outcomes, with female results similar to 'personal qualities' entrants and males' similar to 'academic.' The results are interpreted with reference to the aims and structure of the Newcastle medicine curriculum.

Why is it so hard to consider personal qualities when selecting medical students?

Having 'good' doctors is important to everybody. How to select medical students better has been discussed repeatedly for more than seventy years, implying that prevailing methods could be improved. A significant body of research exists about selection methods and their application in medicine. Yet most medical schools world-wide continue to use prior academic performance and cognitive ability as their major criteria for selection, with minor or no consideration of personal qualities and interpersonal skills (possibly assuming they will develop naturally during training and practice). We describe the main methods available for assessing personal qualities of applicants to medical school and have attempted to identify some reasons and systemic disincentives working against their adoption.

Reliability of personality and values tests: The effects of "high stakes" selection conditions, and of four years in medical school.

This study investigated if scores on tests of personal qualities are affected by whether they will determine selection decisions ("high stakes") or not; and whether they are stable for individuals and groups across a four-year medical course. Two tests, one assessing values and one assessing components of personality, were administered either at the same time as a medical university entrance exam (first cohort; N = 216), or after entry was confirmed (second cohort; N = 142). Both cohorts took the tests again after four years of medical school. Analysis of variance was used to compare group mean scores and interactions, and correlation coefficients to measure temporal reliability. The high stakes cohort initially presented themselves in a significantly more positive light on the personality test. After four years of medical school scores on both tests changed significantly, towards more communitarian values and less empathic attitudes. Thus, personality scores were affected by both the conditions under which the initial tests were conducted and by the passage of time, but values only by the passage of time. Before and after scores were significantly correlated.

Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study.

BACKGROUND: Over the past 70 years, there has been a recurring debate in the literature and in the popular press about how best to select medical students. This implies that we are still not getting it right: either some students are unsuited to medicine or the graduating doctors are considered unsatisfactory, or both. AIM: To determine whether particular variables at the point of selection might distinguish those more likely to become satisfactory professional doctors, by following a complete intake cohort of students throughout medical school and analysing all the data used for the students' selection, their performance on a range of other potential selection tests, academic and clinical assessments throughout their studies, and records of professional behaviour covering the entire five years of the course. METHODS: A longitudinal database captured the following anonymised information for every student (n = 146) admitted in 2007 to the Hull York Medical School (HYMS) in the UK: demographic data (age, sex, citizenship); performance in each component of the selection procedure; performance in some other possible selection instruments (cognitive and non-cognitive psychometric tests); professional behaviour in tutorials and in other clinical settings; academic performance, clinical and communication skills at summative assessments throughout; professional behaviour lapses monitored routinely as part of the fitness-to-practise procedures. Correlations were sought between predictor variables and criterion variables chosen to demonstrate the full range of course outcomes from failure to complete the course to graduation with honours, and to reveal clinical and professional strengths and weaknesses. RESULTS: Student demography was found to be an important predictor of outcomes, with females, younger students and British citizens performing better overall. The selection variable "HYMS academic score", based on prior academic performance, was a significant predictor of components of Year 4 written and Year 5 clinical examinations. Some cognitive subtest scores from the UK Clinical Aptitude Test (UKCAT) and the UKCAT total score were also significant predictors of the same components, and a unique predictor of the Year 5 written examination. A number of the non-cognitive tests were significant independent predictors of Years 4 and 5 clinical performance, and of lapses in professional behaviour. First- and second-year tutor ratings were significant predictors of all outcomes, both desirable and undesirable. Performance in Years 1 and 2 written exams did not predict performance in Year 4 but did generally predict Year 5 written and clinical performance. CONCLUSIONS: Measures of a range of relevant selection attributes and personal qualities can predict intermediate and end of course achievements in academic, clinical and professional behaviour domains. In this study HYMS academic score, some UKCAT subtest scores and the total UKCAT score, and some non-cognitive tests completed at the outset of studies, together predicted outcomes most comprehensively. Tutor evaluation of students early in the course also identified the more and less successful students in the three domains of academic, clinical and professional performance. These results may be helpful in informing the future development of selection tools.

Trial use of the Personal Qualities Assessment (PQA) in the entrance examination of a Japanese medical university: similarities to the results in western countries.

BACKGROUND: The Personal Qualities Assessment (PQA), developed by the University of Newcastle, Australia to assess the aptitude of future medical professionals, has been used in Western countries. PURPOSES: The objective was to investigate whether the PQA is appropriate for Japanese medical school applicants. METHODS: Two of the PQA tests, Libertarian-Dual-Communitarian moral orientations (Mojac) and Narcissism, Aloofness, Confidence, and Empathy (NACE), were translated into Japanese, and administered at the Tokyo Women's Medical University entrance examinations from 2007 to 2009. RESULTS: The distributions of the applicants' Mojac and NACE scores were close to the normal distribution, and the mean scores did not exhibit a large difference from those in Western countries. The only significant difference was that the mean score of the NACE test was slightly lower than the Western norm. CONCLUSIONS: The translated PQA tests may be appropriate for use with Japanese applicants, though further research considering cultural differences is required.

Comparing the cognitive, personality and moral characteristics of high school and graduate medical entrants to the Tehran University of Medical Sciences in Iran.

BACKGROUND: Tehran University of Medical Sciences has two streams of medical student admission: an established high school entry (HSE) route and an experimental graduate entry (GE) route. AIM: To compare the cognitive skills, personality traits and moral characteristics of HSE and GE students admitted to this university. METHODS: The personal qualities assessment tool (PQA; www.pqa.net.au ) was translated from English to Persian and then back-translated. Afterwards 35 individuals from the GE and 109 individuals from the 2007 to 2008 HSE completed the test. The results were compared by t-test and Chi-square. RESULTS: The HSE students showed significantly higher ability in the cognitive skills tests (p < 0.001). They were also more libertarian (p = 0.022), but had lower ability to confront stress and unpleasant events (p < 0.001), and had lower self-awareness and self-control (p < 0.001). CONCLUSION: On the basis of their personal qualities, the GE students had more self-control and strength when coping with stress than the HSE students, but the latter had superior cognitive abilities. Hence it may be useful to include cognitive tests in GE students' entry exam and include tests of personal qualities to exclude those with unsuitable characteristics.

Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study.

BACKGROUND: Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests' predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course. METHODS: This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students' first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students' interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product-moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance. RESULTS: Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year's objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year's examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures. CONCLUSIONS: This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.

Using personality as a predictor of diet induced weight loss and weight management.

BACKGROUND: A major challenge for successful weight management is tailoring weight loss programs to individual needs. The aim of this study was to investigate whether personality traits could be used to match individuals to a compatible weight loss program that would maximize weight loss. METHOD: Two different weight loss trials were conducted, both with a weight loss greater than 5% the measure of success. Fifty-four individuals, BMI 30-40 kg/m2, either followed a slow, healthy eating weight loss diet (HEWLD) of 5000-6000 kJ/day for 12 weeks (n = 22), or a fast, very low energy diet (VLED) of 3000 kJ/day for 4 weeks (n = 32). Anthropometric measurements were recorded at baseline, at the end of the weight loss period and, for VLED, at the end of 10 weeks of weight maintenance. Personality traits were measured at baseline using the Tangney Self Control Scale plus 3 of the scales from the Five Factor Model - Neuroticism, Conscientiousness and Extraversion. RESULTS: The percentage weight loss was significantly greater in VLED (-7.38%) compared to HEWLD (-4.11%), (p < 0.001). Weight loss in HEWLD was positively correlated with Anxiety, a facet of Neuroticism. Weight loss in VLED was positively correlated with Neuroticism (r = 0.5, p < 0.01), and negatively correlated with Dutifulness and Discipline, facets of Conscientiousness, (p < 0.05 for both). No link was observed between weight loss and the personality trait, Self Control, in either HEWLD or VLED. CONCLUSION: The personality factor, Neuroticism, was linked to successful weight loss (that is ≥ 5%) with a particular weight loss treatment, suggesting that there is a potential to use measures of personality to identify appropriate weight loss/management strategies for individuals. TRIAL REGISTRATION: Australia and New Zealand Clinical Trials Register (ANZCTR): ACTRN12611000716965.

Predictive validity of the personal qualities assessment for selection of medical students in Scotland.

BACKGROUND: The Personal Qualities Assessment (PQA) was developed to enhance medical student selection by measuring a range of non-cognitive attributes in the applicants to medical school. Applicants to the five Scottish medical schools were invited to pilot the test in 2001 and 2002. AIMS: To evaluate the predictive validity of PQA for selecting medical students. METHODS: A longitudinal cohort study was conducted in which PQA scores were compared with senior year medical school performance. RESULTS: Consent to access performance markers was obtained from 626 students (61.6% of 1017 entrants in 2002-2003). Linkable Foundation Year (4th) rankings were available for 411 (66%) students and objective structured clinical examination (OSCE) rankings for 335 (54%) of those consenting. Both samples were representative of the original cohort. No significant correlations were detected between separate elements of the PQA assessment and student performance. However, using the algorithm advocated by Powis et al. those defined as 'non-extreme' (<±1.5 SD from the cohort mean scores; SD, standard deviation) character types on the involved-detached and on the libertarian-communitarian moral orientation scales were ranked higher in OSCEs (average of 7.5% or 25 out of 335, p = 0.049). CONCLUSIONS: This study was limited by high attrition and basic outcome markers which are insensitive to relevant non-cognitive characteristics. However, it is the largest currently available study of predictive validity for the PQA assessment. There was one finding of significance: that those students who were identified by PQA as 'not extreme' on the two personal characteristics scales performed better in an OSCE measure of professionalism. Futures studies are required since psychometric testing for both cognitive and non-cognitive attributes are increasingly used in admission process and these should include more and better measures of professionalism against which to correlate non-cognitive traits.

A comprehensive model for the selection of medical students.

BACKGROUND: Medical schools have a need to select their students from an excess of applicants. Selection procedures have evolved piecemeal: Academic thresholds have risen, written tests have been incorporated and interview protocols are developed. AIM: To develop and offer for critical review and, ultimately, present for adoption by medical schools, an evidence-based and defensible model for medical student selection. METHODS: We have described here a comprehensive model for selecting medical students which is grounded on the theoretical and empirical selection and assessment literature, and has been shaped by our own research and experience. RESULTS: The model includes the following selection criteria: Informed self-selection, academic achievement, general cognitive ability (GCA) and aspects of personality and interpersonal skills. A psychometrically robust procedure by which cognitive and non-cognitive test scores can be used to make selection decisions is described. Using de-identified data (n = 1000) from actual selection procedures, we demonstrate how the model and the procedure can be used in practice. CONCLUSION: The model presented is based on a currently best-practice approach and uses measures and methods that maximise the probability of making accurate, fair and defensible selection decisions.

Graduate entry to medicine: widening psychological diversity.

BACKGROUND: At Nottingham University more than 95% of entrants to the traditional 5-year medical course are school leavers. Since 2003 we have admitted graduate entrants (GEM) to a shortened (4-year) course to 'widen access to students from more disadvantaged backgrounds'. We have recently shown that the GEM course widens academic and socio-demographic diversity of the medical student population. This study explored whether GEM students also bring psychological diversity and whether this could be beneficial. METHODS: We studied: a) 217 and 96 applicants to the Nottingham 5- and 4-year courses respectively, applying in the 2002-3 UCAS cycle, and, b) 246 school leavers starting the 5-year course and 39 graduate entrants to the 4-year course in October 2003. The psychological profiles of the two groups of applicants and two groups of entrants were compared using their performance in the Goldberg 'Big 5' Personality test, the Personal Qualities Assessment (PQA; measuring interpersonal traits and interpersonal values), and the Lovibond and Lovibond measure of depression, anxiety and stress. For the comparison of the Entrants we excluded the 33 school leavers and seven graduates who took the tests as Applicants.Statistical analyses were undertaken using SPSS software (version 16.0). RESULTS: Graduate applicants compared to school leaver applicants were significantly more conscientious, more confident, more self controlled, more communitarian in moral orientation and less anxious. Only one of these differences was preserved in the entrants with graduates being less anxious. However, the graduate entrants were significantly less empathetic and conscientious than the school leavers. CONCLUSION: This study has shown that school leaver and graduate entrants to medical school differ in some psychological characteristics. However, if confirmed in other studies and if they were manifest in the extreme, not all the traits brought by graduates would be desirable for someone aiming for a medical career.

Using personal qualities assessment to measure the moral orientation and personal qualities of medical students in a non-Western culture.

How to select candidates with appropriate personal qualities for medical school is an important issue. This study examined the psychometric properties and group differences of the Personal Qualities Assessment (PQA) to test the feasibility of using it as a tool to assess the medical school applicants in a non-Western culture. Seven hundred forty-six medical students in Taiwan completed two psychometric measures: Mojac to assess moral orientation and NACE to assess four aspects of interpersonal relationships. Thirty-one students completed the tests twice to establish test-retest reliability. A subsample of 127 students also completed a measure of the "Big Five" personality traits to examine the construct validity of these scales. Both Mojac and NACE had acceptable internal consistency and test-retest reliability. Conceptually, coherent and significant relationships were observed between test components and between the NACE and Big Five. NACE but not Mojac varied significantly between different sociodemographic groups. Both tests demonstrated acceptable psychometric properties. However, the predictive validity of PQA requires future studies.

Selection of medical students according to their moral orientation.

INTRODUCTION: Consideration has been given to the use of tests of moral reasoning in the selection procedure for medical students. We argue that moral orientation, rather than moral reasoning, might be more efficacious in minimising the likelihood of inappropriate ethical behaviour in medicine. A conceptualisation and measure of moral orientation are presented, together with findings from 11 samples of medical school applicants and students. AIM: To provide empirical evidence for the reliability and validity of a measure of moral orientation and to explore gender, age, cultural and educational influences on moral orientation. METHODS: A questionnaire designed to measure a libertarian-dual-communitarian dimension of moral orientation was completed by 7864 medical school applicants and students in Australia, Israel, Fiji, New Zealand, Scotland and England and by 84 Australian psychology students between 1997 and 2001. RESULTS: Older respondents produced marginally higher (more communitarian) moral orientation scores, as did women compared to men. Minor but significant (P <0.05) cultural differences were found. The Israeli samples produced higher mean moral orientation scores, while the Australian psychology student sample produced a lower (more libertarian) mean score relative to all other samples. No significant change in moral orientation score was observed after 1 year in a sample of Australian medical school students (n=59), although some differences observed between 5 cohorts of Australian medical students (Years 1-5; n=234) did reach significance. Moral orientation scores were found to be significantly correlated with a number of personality measures, providing evidence of construct validity. In all samples moral orientation significantly predicted the moral decisions made in response to the hypothetical dilemmas embedded in the measurement instrument. Discussion The results provide support for the conceptualisation of a libertarian-dual-communitarian dimension of moral orientation and demonstrate the psychometric properties of the measurement instrument. A number of questions concerning the use of such tests in selection procedures are considered.