Power calculations for cluster randomized trials (CRTs) with right-truncated Poisson-distributed outcomes: a motivating example from a malaria vector control trial.

BACKGROUND: Cluster randomized trials (CRTs) are increasingly used to study the efficacy of interventions targeted at the population level. Formulae exist to calculate sample sizes for CRTs, but they assume that the domain of the outcomes being considered covers the full range of values of the considered distribution. This assumption is frequently incorrect in epidemiological trials in which counts of infection episodes are right-truncated due to practical constraints on the number of times a person can be tested. METHODS: Motivated by a malaria vector control trial with right-truncated Poisson-distributed outcomes, we investigated the effect of right-truncation on power using Monte Carlo simulations. RESULTS: The results demonstrate that the adverse impact of right-truncation is directly proportional to the magnitude of the event rate, λ, with calculations of power being overestimated in instances where right-truncation was not accounted for. The severity of the adverse impact of right-truncation on power was more pronounced when the number of clusters was ≤30 but decreased the further the right-truncation point was from zero. CONCLUSIONS: Potential right-truncation should always be accounted for in the calculation of sample size requirements at the study design stage.

What is the effect of secondary (high) schooling on subsequent medical school performance? A national, UK-based, cohort study.

OBJECTIVES: University academic achievement may be inversely related to the performance of the secondary (high) school an entrant attended. Indeed, some medical schools already offer 'grade discounts' to applicants from less well-performing schools. However, evidence to guide such policies is lacking. In this study, we analyse a national dataset in order to understand the relationship between the two main predictors of medical school admission in the UK (prior educational attainment (PEA) and performance on the United Kingdom Clinical Aptitude Test (UKCAT)) and subsequent undergraduate knowledge and skills-related outcomes analysed separately. METHODS: The study was based on national selection data and linked medical school outcomes for knowledge and skills-based tests during the first five years of medical school. UKCAT scores and PEA grades were available for 2107 students enrolled at 18 medical schools. Models were developed to investigate the potential mediating role played by a student's previous secondary school's performance. Multilevel models were created to explore the influence of students' secondary schools on undergraduate achievement in medical school. RESULTS: The ability of the UKCAT scores to predict undergraduate academic performance was significantly mediated by PEA in all five years of medical school. Undergraduate achievement was inversely related to secondary school-level performance. This effect waned over time and was less marked for skills, compared with undergraduate knowledge-based outcomes. Thus, the predictive value of secondary school grades was generally dependent on the secondary school in which they were obtained. CONCLUSIONS: The UKCAT scores added some value, above and beyond secondary school achievement, in predicting undergraduate performance, especially in the later years of study. Importantly, the findings suggest that the academic entry criteria should be relaxed for candidates applying from the least well performing secondary schools. In the UK, this would translate into a decrease of approximately one to two A-level grades.

Predictors of fitness to practise declarations in UK medical undergraduates.

BACKGROUND: Misconduct during medical school predicts subsequent fitness to practise (FtP) events in doctors, but relatively little is known about which factors are associated with such issues during undergraduate education. This study exploits the newly created UK medical education database (UKMED), with the aim of identifying predictors of conduct or health-related issues that could potentially impair FtP. The findings would have implications for policies related to both the selection and support of medical students. METHODS: Data were available for 14,379 students obtaining provisional registration with the General Medical Council who started medical school in 2007 and 2008. FtP declarations made by students were available, as were various educational and demographic predictor variables, including self-report 'personality measures' for students who participated in UK Clinical Aptitude Test (UKCAT) pilot studies. Univariable and multivariable logistic regression models were developed to evaluate the predictors of FtP declarations. RESULTS: Significant univariable predictors (p < 0.05) for conduct-related declarations included male gender, white ethnicity and a non-professional parental background. Male gender (OR 3.07) and higher 'self-esteem' (OR 1.45) were independently associated with an increased risk of a conduct issue. Female gender, a non-professional background, and lower self-reported 'confidence' were, among others, associated with increased odds of a health-related declaration. Only 'confidence' was a significant independent predictor of a health declaration (OR 0.69). Female gender, higher UKCAT score, a non-professional background and lower 'confidence' scores were significant predictors of reported depression, and the latter two variables were independent predictors of declared depression. CONCLUSIONS: White ethnicity and UK nationality were associated with increased odds of both conduct and health-related declarations, as were certain personality traits. Students from non-professional backgrounds may be at increased risk of depression and therefore could benefit from targeted support. The small effect sizes observed for the 'personality measures' suggest they would offer little potential benefit for selection, over and above those measures already in use.

Predicting fitness to practise events in international medical graduates who registered as UK doctors via the Professional and Linguistic Assessments Board (PLAB) system: a national cohort study.

BACKGROUND: International medical graduates working in the UK are more likely to be censured in relation to fitness to practise compared to home graduates. Performance on the General Medical Council's (GMC's) Professional and Linguistic Assessments Board (PLAB) tests and English fluency have previously been shown to predict later educational performance in this group of doctors. It is unknown whether the PLAB system is also a valid predictor of unprofessional behaviour and malpractice. The findings would have implications for regulatory policy. METHODS: This was an observational study linking data relating to fitness to practise events (referral or censure), PLAB performance, demographic variables and English language competence, as evaluated via the International English Language Test System (IELTS). Data from 27,330 international medical graduates registered with the GMC were analysed, including 210 doctors who had been sanctioned in relation to at least one fitness to practise issue. The main outcome was risk of eventual censure (including a warning). RESULTS: The significant univariable educational predictors of eventual censure (versus no censures or referrals) were lower PLAB part 1 (hazard ratio [HR], 0.99; 95% confidence interval, 0.98 to 1.00) and part 2 scores (HR, 0.94; 0.91 to 0.97) at first sitting, multiple attempts at both parts of the PLAB, lower IELTS reading (HR, 0.79; 0.65 to 0.94) and listening scores (HR, 0.76; 0.62 to 0.93) and higher IELTS speaking scores (HR, 1.28; 1.04 to 1.57). Multiple resits at either part of the PLAB and higher IELTS speaking score (HR, 1.49; 1.20 to 1.84) were also independent predictors of censure. We estimated that the proposed limit of four attempts at both parts of the PLAB would reduce the risk in this entire group by only approximately two censures per 5 years in this group of doctors. CONCLUSIONS: Making the PLAB, or any replacement assessment, more stringent and raising the required standards of English reading and listening may result in fewer fitness to practice events in international medical graduates. However, the number of PLAB resits permitted would have to be further capped to meaningfully impact the risk of sanctions in this group of doctors.

Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study.

BACKGROUND: The UK Clinical Aptitude Test (UKCAT) has been shown to have a modest but statistically significant ability to predict aspects of academic performance throughout medical school. Previously, this ability has been shown to be incremental to conventional measures of educational performance for the first year of medical school. This study evaluates whether this predictive ability extends throughout the whole of undergraduate medical study and explores the potential impact of using the test as a selection screening tool. METHODS: This was an observational prospective study, linking UKCAT scores, prior educational attainment and sociodemographic variables with subsequent academic outcomes during the 5 years of UK medical undergraduate training. The participants were 6812 entrants to UK medical schools in 2007-8 using the UKCAT. The main outcome was academic performance at each year of medical school. A receiver operating characteristic (ROC) curve analysis was also conducted, treating the UKCAT as a screening test for a negative academic outcome (failing at least 1 year at first attempt). RESULTS: All four of the UKCAT scale scores significantly predicted performance in theory- and skills-based exams. After adjustment for prior educational achievement, the UKCAT scale scores remained significantly predictive for most years. Findings from the ROC analysis suggested that, if used as a sole screening test, with the mean applicant UKCAT score as the cut-off, the test could be used to reject candidates at high risk of failing at least 1 year at first attempt. However, the 'number needed to reject' value would be high (at 1.18), with roughly one candidate who would have been likely to pass all years at first sitting being rejected for every higher risk candidate potentially declined entry on this basis. CONCLUSIONS: The UKCAT scores demonstrate a statistically significant but modest degree of incremental predictive validity throughout undergraduate training. Whilst the UKCAT could be considered a fairly crude screening tool for future academic performance, it may offer added value when used in conjunction with other selection measures. Future work should focus on the optimum role of such tests within the selection process and the prediction of post-graduate performance.