Assessing a research training programme for rural physicians.

INTRODUCTION: To assess the effect of a training programme called 6for6 (the programme) on research competency and productivity amongst rural physicians. The programme develops the research skills of six rural physicians over six weekends. Physicians learn about various research methods and writing techniques through blended learning components. METHODS: We conducted a quasi-experimental study, comparing research competency and productivity between intervention and non-equivalent control groups and over time through a repeated measures design. Generalized linear mixed model (GLMM), ANOVA, and Cochran Q tests were conducted. The intervention was provided to five groups of 6 rural physicians each between 2014 and 2019. Main outcome measures: self-assessed research competency (knowledge, attitudes and skills) and productivity (publications, grants and presentations of research-related work at conferences) were our primary and secondary outcomes, respectively. We measured the outcomes before, during and after the programme. Controls: Rural physicians who expressed interest in the programme and later enrolled. RESULTS: This study shows that, amongst its thirty participants, overall research competency was significantly different between intervention and control groups (65.7% ± 37.6% and 58.6% ± 14.4%, P < 0.05 for GLMM). The percentage of participants who were productive before, during and after the programme was 26.7%, 16.7% and 50.0%, respectively. Overall, productivity rates were significantly different between intervention and control groups (rate difference was 72.2/100 person-years, P < 0.05 for GLMM). CONCLUSION: This study suggests that the programme improves research competency and productivity for rural physicians. Rural physicians who wish to improve their research competency would benefit from participating in similar programmes.

Résumé Introduction: Évaluer l'effet d'un programme de formation intitulé 6for6 (le programme) sur les compétences en recherche et la productivité parmi les médecins des régions rurales. Le programme permet à six médecins en région rurale d'acquérir des compétences en recherche durant six fins de semaine. Les médecins apprennent diverses méthodologies de recherche et techniques de rédaction par l'entremise d'un programme d'apprentissage mixte. Méthodologie: Nous avons réalisé une étude quasi-expérimentale, qui comparait les compétences en recherche et la productivité entre des groupes non-équivalents intervention et témoin, et dans le temps, par une méthodologie à mesures répétitives. Un modèle linéaire à effets mixtes généralisé (GLMM), un modèle d'analyse de variance, et des tests Q de Cochran ont été réalisés. L'intervention a été appliquée à 5 groupes de 6 médecins en région rurale, entre 2014 et 2019 dans tous les cas. Paramètre d'évaluation: compétences en recherche évaluées par l'apprenant (connaissances, attitudes et compétences) et productivité (publications, subventions et présentation des travaux de recherche aux congrès) étaient respectivement nos paramètres d'évaluation principal et secondaire. Nous avons mesuré les paramètres avant, durant et après le programme. Les médecins en région rurale ayant manifesté de l'intérêt à l'égard du programme ont été inscrits. Résultats: L'étude montre que parmi les 30 participants, les compétences générales en recherche étaient significativement différentes entre les groupes intervention et témoin (65,7 ± 37,6% et 58,6 ± 14,4%, P < 0,05 pour le GLMM). Le pourcentage de participants qui étaient productifs avant, durant et après le programme était respectivement de 26,7, 16,7 et 50,0%. Dans l'ensemble, la productivité était significativement différente entre les groupes intervention et témoin (différence des taux: 72,2 par 100 années-personnes, P < 0,05 pour le GLMM). Conclusion: Cette étude laisse penser que le programme améliore les compétences en recherche et la productivité chez les médecins en région rurale. Les médecins en région rurale qui souhaitent améliorer leurs compétences en recherche bénéficieront de programmes semblables. Mots-clés: Compétences en recherche, programme de formation en recherche, santé en région rurale, communautés rurales.

Evaluation plan of the 6for6 research skills program for rural and remote physicians.

Overwhelming issues and barriers often prevent rural and remote physicians (RRPs) from pursuing the many socially accountable research questions they encounter on a daily basis. Although research training programs can empower RRPs to rise to these challenges, there is a lack of evidence on how they should be developed and refined. At Memorial University, a faculty development program (FDP) called 6for6 has been helping RRPs surmount their research quagmires and engage in scholarship since 2014. After an initial three-year (2014-17) pilot, we prepared a detailed plan to evaluate the 6for6 research FDP for RRPs and inform future years of delivery. Using a modified Delphi method and participatory action model a group of program team members, stakeholders and evaluation experts developed an evaluation plan including a logic model and an evaluation matrix addressing five key themes. To our knowledge, this is the first evaluation plan for a research-focused FDP targeting RRPs. While this plan was developed specifically for the 6for6 FDP, our approach to its development may be useful to any institution interested in evaluating an FDP with limited resources.

An Evaluation of Interrater Reliability Measures on Binary Tasks Using d-Prime.

Many indices of interrater agreement on binary tasks have been proposed to assess reliability, but none has escaped criticism. In a series of Monte Carlo simulations, five such indices were evaluated using d-prime, an unbiased indicator of raters' ability to distinguish between the true presence or absence of the characteristic being judged. Phi and, to a lesser extent, Kappa coefficients performed best across variations in characteristic prevalence, and raters' expertise and bias. Correlations with d-prime for Percentage Agreement, Scott's Pi, and Gwet's AC1 were markedly lower. In situations where two raters make a series of binary judgments, the findings suggest that researchers should choose Phi or Kappa to assess interrater agreement as the superiority of these indices was least influenced by variations in the decision environment and characteristics of the decision makers.