Are different station formats assessing different dimensions in multiple mini-interviews? Findings from the Canadian integrated French multiple mini-interviews.

BACKGROUND: Multiple mini-interviews (MMI) are used to assess non-academic attributes for selection in medicine and other healthcare professions. It remains unclear if different MMI station formats (discussions, role-plays, collaboration) assess different dimensions. METHODS: Based on station formats of the 2018 and 2019 Integrated French MMI (IFMMI), which comprised five discussions, three role-plays and two collaboration stations, the authors performed confirmatory factor analysis (CFA) using the lavaan 0.6-5 R package and compared a one-factor solution to a three-factor solution for scores of the 2018 (n = 1438) and 2019 (n = 1440) cohorts of the IFMMI across three medical schools in Quebec, Canada. RESULTS: The three-factor solution was retained, with discussions, role-plays and collaboration stations all loading adequately with their scores. Furthermore, all three factors had moderate-to-high covariance (range 0.44 to 0.64). The model fit was also excellent with a Comparative fit index (CFI) of 0.983 (good if > 0.9), a Tucker Lewis index of 0.976 (good if > 0.95), a Standardized Root Mean Square Residual of 0.021 (good if < .08) and a Root Mean Square Error of 0.023 (good if < 0.08) for 2018 and similar results for 2019. In comparison, the single factor solution presented a lower fit (CFI = 0.819, TLI = 0.767, SRMR = 0.049 and RMSEA = 0.070). CONCLUSIONS: The IFMMI assessed three dimensions that were related to stations formats, a finding that was consistent across two cohorts. This suggests that different station formats may be assessing different skills, and has implications for the choice of appropriate reliability metrics and the interpretation of scores. Further studies should try to characterize the underlying constructs associated with each station format and look for differential predictive validity according to these formats.

Snijders's correction of Infit and Outfit indexes with estimated ability level: an analysis with the Rasch model.

The Infit mean square W and the Outfit mean square U are commonly used person fit indexes under Rasch measurement. However, they suffer from two major weaknesses. First, their asymptotic distribution is usually derived by assuming that the true ability levels are known. Second, such distributions are even not clearly stated for indexes U and W. Both issues can seriously affect the selection of an appropriate cut-score for person fit identification. Snijders (2001) proposed a general approach to correct some person fit indexes when specific ability estimators are used. The purpose of this paper is to adapt this approach to U and W indexes. First, a brief sketch of the methodology and its application to U and W is proposed. Then, the corrected indexes are compared to their classical versions through a simulation study. The suggested correction yields controlled Type I errors against both conservatism and inflation, while the power to detect specific misfitting response patterns gets significantly increased.

A Comparison of Modern and Popular Approaches to Calculating Reliability for Dichotomously Scored Items.

Recent work on reliability coefficients has largely focused on continuous items, including critiques of Cronbach's alpha. Although two new model-based reliability coefficients have been proposed for dichotomous items (Dimitrov, 2003a,b; Green & Yang, 2009a), these approaches have yet to be compared to each other or other popular estimates of reliability such as omega, alpha, and the greatest lower bound. We seek computational improvements to one of these model-based reliability coefficients and, in addition, conduct initial Monte Carlo simulations to compare coefficients using dichotomous data. Our results suggest that such improvements to the model-based approach are warranted, while model-based approaches were generally superior.

Different measures of holistic face processing tap into distinct but partially overlapping mechanisms.

Holistic processing, which includes the integration of facial features and analysis of their relations to one another, is a hallmark of what makes faces 'special'. Various experimental paradigms purport to measure holistic processing but these have often produced inconsistent results. This has led researchers to question the nature and structure of the mechanism(s) underlying holistic processing. Using an individual differences approach, researchers have examined relations between various measures of holistic processing in an attempt to resolve these questions. In keeping with this, we examined relationships between four commonly used measures of holistic face processing in a large group of participants (N = 223): (1) The Face Inversion Effect, (2) the Part Whole Effect (PWE), (3) the Composite Face Effect, and (4) the Configural Featural Detection Task (CFDT). Several novel methodological and analytical elements were introduced, including the use of factor analysis and the inclusion of control conditions to confirm the face specificity of all of the effects measured. The four indexes of holistic processing derived from each measure loaded onto two factors, one encompassing the PWE and the CFDT, and one encompassing the CE. The 16 conditions tested across the four tasks loaded onto four factors, each factor corresponding to a different measure. These results, together with those of other studies, suggest that holistic processing is a multifaceted construct and that different measures tap into distinct but partially overlapping elements of it.

The Mini-Med School and Its Impact on Future Health Care Professionals' Attitudes toward Indigenous People.

Mini-Med Schools (MiMS) are an opportunity for health sciences and social work undergraduates to discuss health-related topics with Innu and Atikamekw youth in Canada. More than 500 undergraduates and 1,000 students have taken part in the project since its beginning in 2011. This study aims to assess the impact of both 1) MiMS's predeparture training and 2) the MiMS themselves on undergraduates' prejudices toward Indigenous peoples. Satisfaction of the undergraduates taking part in the activity was also assessed. Seventy-eight undergraduates were recruited and completed the Old-fashioned and Modern Prejudiced Attitudes Toward Aboriginals Scales (O-PATAS and M-PATAS) at baseline, after the pre-departure training, and after the MiMS. They also completed satisfaction surveys. This study shows a reduction of prejudices after participating to a MiMS, but no effect of a pre-departure training. The activities were overall appreciated by undergraduates and most of them would like to take part again in the MiMS.

Screening for Cognitive Dysfunction Using the Rowland Universal Dementia Assessment Scale in Adults With Sickle Cell Disease.

Importance: Adults with sickle cell disease (SCD) disproportionally experience early cognitive decline; however, guidance on the optimal screening strategy for cognitive dysfunction is lacking, and several available tools are biased by language, educational level, socioeconomic status, and race/ethnicity. The Rowland Universal Dementia Assessment Scale (RUDAS) was specifically designed for cognitive screening in multicultural populations. Objective: To ascertain the prevalence of suspected dementia in adults with SCD using the RUDAS, and to identify whether age, sex, educational level, several biological variables, and SCD complications were associated with RUDAS scores. Design, Setting, and Participants: This multicenter, bilingual, cross-sectional study was conducted in 2 SCD comprehensive care centers in Canada (Centre Hospitalier de l'Université Montréal in Montréal and University Health Network in Toronto). Participants were adults aged 18 years or older and were enrolled in the study between July 1, 2018, and July 30, 2019. All outpatients were eligible and offered study participation, unless they had an acute medical condition that required inpatient care or they were unable to follow study instructions. Interventions: The RUDAS was administered by trained personnel in either French or English, according to the patient's language preference. A questionnaire on social determinants of health was also administered, and participants underwent screening for anxiety and depression. Main Outcomes and Measures: Proportion of participants with RUDAS scores that were suggestive of dementia and the RUDAS score. Any score lower than 23 points was suggestive of dementia, a score between 23 and 27 points indicated a possible association with mild neurocognitive disorder, and a score higher than 27 points was normal. Results: A total of 252 adult patients with SCD were included (136 women [54.0%]; mean [range] age, 34.8 [18-75] years). Overall, 29 patients (11.5%) had RUDAS scores that were suggestive of dementia, and this proportion increased with age (15 [8.7%] in the 18-39 years age group, 10 [14.5%] in the 40-59 years age group, and 4 [36.4%] in the ≥60 years age group). The RUDAS scores were not associated with sex, language, SCD genotype, and SCD complications. The highest level of education was significantly associated with the RUDAS score; however, the association was small (η2 = 0.02; 95% CI, 0.00-0.07; P = .02). In a multivariable analysis, lower glomerular filtration rate (r = 0.40; 95% CI, 0.29-0.50; P < .001) and increasing age (r = -0.37; 95% CI, -0.47 to -0.26; P < .001), but not SCD genotype or disease severity, were associated with lower RUDAS scores. Conclusions and Relevance: This study found that using the RUDAS revealed a high prevalence of suspected dementia in adult patients with SCD that was associated with worsening kidney function and age. Cognition should be screened in all adult patients with SCD, regardless of age, disease severity, and SCD genotype; further validation of the RUDAS is ongoing.

A general framework and an R package for the detection of dichotomous differential item functioning.

Differential item functioning (DIF) is an important issue of interest in psychometrics and educational measurement. Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. Starting from a framework for classifying DIF detection methods and from a comparative overview of the most traditional methods, an R package for nine methods, called difR, is presented. The commands and options are briefly described, and the package is illustrated through the analysis of a data set on verbal aggression.

Cognitive diagnostic assessment via Bayesian evaluation of informative diagnostic hypotheses.

There exist diverse approaches that can be used for cognitive diagnostic assessment, such as mastery testing, constrained latent class analysis, rule space methodology, diagnostic cognitive modeling, and person-fit analysis. Each of these approaches can be used within 1 of the 4 psychometric perspectives on diagnostic testing discussed by Borsboom (2008), that is, the dimensional, diagnostic, constructivist, and causal system perspectives. Bayesian evaluation of informative diagnostic hypotheses is an alternative for each of the other approaches that is more flexible in the diagnostic hypotheses that can be evaluated, and it can be used in each of the 4 psychometric perspectives on diagnostic testing. After being formulated, informative diagnostic hypotheses are evaluated by means of the Bayes factor using only the data from the person to be diagnosed. Already, relatively small diagnostic tests render Bayes factors that provide convincing evidence in favor of 1 of the diagnostic hypotheses under consideration.