Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions.

INTRODUCTION: Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. METHODS: We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students' performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. RESULTS: We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. DISCUSSION: Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

DNA methylation and gene expression changes in monozygotic twins discordant for psoriasis: identification of epigenetically dysregulated genes.

Monozygotic (MZ) twins do not show complete concordance for many complex diseases; for example, discordance rates for autoimmune diseases are 20%-80%. MZ discordance indicates a role for epigenetic or environmental factors in disease. We used MZ twins discordant for psoriasis to search for genome-wide differences in DNA methylation and gene expression in CD4(+) and CD8(+) cells using Illumina's HumanMethylation27 and HT-12 expression assays, respectively. Analysis of these data revealed no differentially methylated or expressed genes between co-twins when analyzed separately, although we observed a substantial amount of small differences. However, combined analysis of DNA methylation and gene expression identified genes where differences in DNA methylation between unaffected and affected twins were correlated with differences in gene expression. Several of the top-ranked genes according to significance of the correlation in CD4(+) cells are known to be associated with psoriasis. Further, gene ontology (GO) analysis revealed enrichment of biological processes associated with the immune response and clustering of genes in a biological pathway comprising cytokines and chemokines. These data suggest that DNA methylation is involved in an epigenetic dysregulation of biological pathways involved in the pathogenesis of psoriasis. This is the first study based on data from MZ twins discordant for psoriasis to detect epigenetic alterations that potentially contribute to development of the disease.

X chromosome inactivation in cervical cancer patients.

Development of cervical carcinomas is strongly associated with presence of human papilloma virus (HPV). Recently we found that young patients with breast cancer had a higher frequency of skewed X inactivation in peripheral blood cells, indicating an effect of X-linked genes on breast cancer development. In this study, we investigated the frequency of skewed X-inactivation pattern in blood and tumor biopsies from patients with cervical cancer. No difference in the frequency of skewed X inactivation in blood was found between 142 patients and 437 age-matched controls. Elderly females have a higher frequency of skewed X inactivation in blood cells than younger females. An age effect was confirmed in this study for blood cells in both patients and controls. A tendency to an age effect was also found in the tumor biopsies. The correlation between X inactivation in blood and biopsies was 0.39 (P<0.001), showing that the X inactivation in biopsies to some degree reflects skewing in blood. Furthermore, of eight patients with a skewing of > or =75% in biopsies, seven patients had a skewing in the same direction in their blood cells (P=0.03). Our results indicate that if X-inactivation analysis is to be used in clonality studies of cervical cancers, it is essential to consider both the age and the X-inactivation pattern in blood cells.