Inter-rater reliability in clinical assessments: do examiner pairings influence candidate ratings?

BACKGROUND: The reliability of clinical assessments is known to vary considerably with inter-rater reliability a key contributor. Many of the mechanisms that contribute to inter-rater reliability however remain largely unexplained and unclear. While research in other fields suggests personality of raters can impact ratings, studies looking at personality factors in clinical assessments are few. Many schools use the approach of pairing examiners in clinical assessments and asking them to come to an agreed score. Little is known however, about what occurs when these paired examiners interact to generate a score. Could personality factors have an impact? METHODS: A fully-crossed design was employed with each participant examiner observing and scoring. A quasi-experimental research design used candidate's observed scores in a mock clinical assessment as the dependent variable. The independent variables were examiner numbers, demographics and personality with data collected by questionnaire. A purposeful sample of doctors who examine in the Final Medical examination at our institution was recruited. RESULTS: Variability between scores given by examiner pairs (N = 6) was less than the variability with individual examiners (N = 12). 75% of examiners (N = 9) scored below average for neuroticism and 75% also scored high or very high for extroversion. Two-thirds scored high or very high for conscientiousness. The higher an examiner's personality score for extroversion, the lower the amount of change in his/her score when paired up with a co-examiner; reflecting possibly a more dominant role in the process of reaching a consensus score. CONCLUSIONS: The reliability of clinical assessments using paired examiners is comparable to assessments with single examiners. Personality factors, such as extroversion, may influence the magnitude of change in score an individual examiner agrees to when paired up with another examiner. Further studies on personality factors and examiner behaviour are needed to test associations and determine if personality testing has a role in reducing examiner variability.

A "HOLTER" for Parkinson's disease: Validation of the ability to detect on-off states using the REMPARK system.

The treatment of Parkinson's disease (PD) with levodopa is very effective. However, over time, motor complications (MCs) appear, restricting the patient from leading a normal life. One of the most disabling MCs is ON-OFF fluctuations. Gathering accurate information about the clinical status of the patient is essential for planning treatment and assessing its effect. Systems such as the REMPARK system, capable of accurately and reliably monitoring ON-OFF fluctuations, are of great interest. OBJECTIVE: To analyze the ability of the REMPARK System to detect ON-OFF fluctuations. METHODS: Forty-one patients with moderate to severe idiopathic PD were recruited according to the UK Parkinson's Disease Society Brain Bank criteria. Patients with motor fluctuations, freezing of gait and/or dyskinesia and who were able to walk unassisted in the OFF phase, were included in the study. Patients wore the REMPARK System for 3days and completed a diary of their motor state once every hour. RESULTS: The record obtained by the REMPARK System, compared with patient-completed diaries, demonstrated 97% sensitivity in detecting OFF states and 88% specificity (i.e., accuracy in detecting ON states). CONCLUSION: The REMPARK System detects an accurate evaluation of ON-OFF fluctuations in PD; this technology paves the way for an optimisation of the symptomatic control of PD motor symptoms as well as an accurate assessment of medication efficacy.

PRRT2 gene mutations: from paroxysmal dyskinesia to episodic ataxia and hemiplegic migraine.

OBJECTIVE: The proline-rich transmembrane protein (PRRT2) gene was recently identified using exome sequencing as the cause of autosomal dominant paroxysmal kinesigenic dyskinesia (PKD) with or without infantile convulsions (IC) (PKD/IC syndrome). Episodic neurologic disorders, such as epilepsy, migraine, and paroxysmal movement disorders, often coexist and are thought to have a shared channel-related etiology. To investigate further the frequency, spectrum, and phenotype of PRRT2 mutations, we analyzed this gene in 3 large series of episodic neurologic disorders with PKD/IC, episodic ataxia (EA), and hemiplegic migraine (HM). METHODS: The PRRT2 gene was sequenced in 58 family probands/sporadic individuals with PKD/IC, 182 with EA, 128 with HM, and 475 UK and 96 Asian controls. RESULTS: PRRT2 genetic mutations were identified in 28 out of 58 individuals with PKD/IC (48%), 1/182 individuals with EA, and 1/128 individuals with HM. A number of loss-of-function and coding missense mutations were identified; the most common mutation found was the p.R217Pfs*8 insertion. Males were more frequently affected than females (ratio 52:32). There was a high proportion of PRRT2 mutations found in families and sporadic cases with PKD associated with migraine or HM (10 out of 28). One family had EA with HM and another large family had typical HM alone. CONCLUSIONS: This work expands the phenotype of mutations in the PRRT2 gene to include the frequent occurrence of migraine and HM with PKD/IC, and the association of mutations with EA and HM and with familial HM alone. We have also extended the PRRT2 mutation type and frequency in PKD and other episodic neurologic disorders.