Factors predicting rural location employment intent and choice among medical students and graduates.

PURPOSE: Workforce shortage is a contributing cause of health inequality in rural Australia. There is inconclusive evidence demonstrating which factors cause doctors to choose rural practice. This study's objective is to determine predictive factors for medical students' intent to work rurally and for graduates' current rural employment location choice. METHODS: This prospective cohort study, utilized data gathered from the University of New South Wales about students and graduates who had spent one or more years in a Rural Clinical School. Participants were final year students and graduates already working in Australia. Stepwise logistic regression was used to determine predictive factors for the two outcomes. RESULTS: Predictors for student intent to work rurally are rural background (odds ratio [OR], 7.16; 95% confidence interval [CI], 2.59-19.53), choosing to study at the Rural Clinical School (OR, 8.72; 95% CI, 1.32-57.63), and perceiving rural areas as opportunistic for career advancement (OR, 1.69; 95% CI, 1.15-2.49). Predictors for graduates currently working in a rural location are Bonded Medical Program participation (OR, 6.40; 95% CI, 1.15-35.59) and personal altruism (OR, 1.91; 95% CI, 1.02-3.57). CONCLUSION: While intent is predicted by having a rural background, choosing to study at the Rural Clinical School and perception of rural areas as having positive career opportunities, a current rural workplace location among graduates is predicted by holding a bonded medical position and a desire to serve an under-resourced population. Maintaining the Bonded Medical Program and clear communication regarding training pathways may increase numbers of rural doctors.

Empirical analysis comparing the tele-objective structured clinical examination (teleOSCE) and the in-person assessment in Australia.

PURPOSE: It aimed to compare the use of the tele objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE. METHODS: This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia. RESULTS: In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference -0.277, P<0.001, effect size 0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020. CONCLUSION: The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.

Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student's competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

Insights into student assessment outcomes in rural clinical campuses.

BACKGROUND: There is an ongoing debate about the impact of studying medicine in rural vs. metropolitan campuses on student assessment outcomes. The UNSW Medicine Rural Clinical School has five main campuses; Albury-Wodonga, Coffs Harbour, Griffith, Port Macquarie and Wagga Wagga. Historical data of student assessment outcomes at these campuses raised concerns regarding potential biases in assessment undertaken, as well as the availability and quality of learning resources. The current study aims to identify the extent to which the location of examination (rural versus metropolitan) has an impact on student marks in OSCEs. METHODS: Assessment data was employed for this study from 275 medical students who sat their final examinations in Years 3 and 6 of the undergraduate Medicine program at UNSW in 2018. The data consists of matched student assessment results from the Year 3 (Y3) MCQ examination and OSCE, and from the Year 6 (Y6) MCQ, OSCE and management viva examinations. The analysis used Univariate Analysis of Variance and linear regression models to identify the impact of site of learning and site of examination on assessment outcomes. RESULTS: The results demonstrate that neither site of learning nor site of examination had any significant impact on OSCE or Management Viva assessment outcomes while potential confounders are controlled. CONCLUSION: It is suggested that some of the supposed disadvantages inherent at rural campuses are effectively mitigated by perceived advantages; more intensive interaction with patients, the general and medical communities at those sites, as well as effective e-learning resources and moderation of assessment grades.

Retest effects in medical selection interviews.

CONTEXT: Repetition of a cognitive ability test is known to increase scores, but almost no research has examined whether similar improvement occurs with repetition of interviews. Retest effects can change the rank order of candidates and reduce the test's criterion validity. Because interviews are widely used to select medical students and postgraduate trainees, and because applicants apply to multiple programmes and often reapply if unsuccessful, the potential for retest effects needs to be understood. OBJECTIVES: This study was designed to identify if retest improvements occur when candidates undertake multiple interviews and, if so, whether the effect is attributable to general interview experience or specific experience and whether repeat testing affects criterion validity. METHODS: We compared interview scores of applicants who were interviewed for one or more of three independent undergraduate medical programmes in two consecutive years and those who were interviewed in both years for the same programme. Correlations between initial and repeat interview scores and a written test of social understanding were compared. RESULTS: General experience (being interviewed by multiple programmes) did not produce improvement in subsequent interview performance. There was no evidence of method effect (having prior experience of the multiple mini-interview process). Specific experience (being interviewed by the same programme across 2 years) resulted in a significant improvement in scores for which regression to the mean did not fully account. Criterion validity did not appear to be affected. CONCLUSIONS: Unsuccessful candidates for medical school who reapply and are re-interviewed on a subsequent occasion at the same institution are likely to increase their scores. The results of this study suggest the increase is probably not attributable to improved ability.

Pilot simulation-based research for evaluation, training and assessment of holistic nursing at four intensive care unit sites.

This article was migrated. The article was marked as recommended. Background: Holistic nursing of intensive care unit (ICU) patients encompasses diverse challenges requiring regular in situ evaluation, training and assessment. Simulation has been adopted as a research and training tool in medicine; however, evidence for its use in enhancing holistic care at multi-sites is limited. Objective: This study aims to conduct a simulation-based research (SBR) at four ICU for standardized training of holistic nursing. Methods: There are stages of evaluating, training+in-training assessment and post-course assessment in this SBR. Specifically, the curriculum-mapped scenarios were developed according to the deficiency of each site after evaluating stage. At the training stage, the first simulation by team was defined as preparation step and the in-training assessment was undertaken at the second simulation. Results: From January 2017 to October 2018, sixty-four ICU nurses (16 teams, 4 teams in each site) at RCU, PICU, NICU and GYN ICU, attend 8 similar courses (2 courses at each site) over 20 months. In comparison with baseline performance, in-training assessments revealed the significant improvement of attendee's skills of holistic nursing. Attendees commented that simulation was a valuable training modality to enhance skills of holistic care including history taking, physical examination, communication and teamwork that are rarely taught among ICU nurses. Post-course workplace assessment by senior nurses revealed the high frequency of clinical application of holistic nursing by attendees. Additionally, post-course self assessment revealed a high attendee's confidence of holistic approaching in ICU. Conclusion: This pilot SBR demonstrated the feasibility of a standardized holistic care simulation program across four ICUs. In situ simulation and post-course workplace assessment affords situational learning without compromising patient safety and is an exciting and novel training of holistic nursing for ICU that could be integrated into regular intervention.

Validation of the self-assessment teamwork tool (SATT) in a cohort of nursing and medical students.

INTRODUCTION: Poor teamwork has been implicated in medical error and teamwork training has been shown to improve patient care. Simulation is an effective educational method for teamwork training. Post-simulation reflection aims to promote learning and we have previously developed a self-assessment teamwork tool (SATT) for health students to measure teamwork performance. This study aimed to evaluate the psychometric properties of a revised self-assessment teamwork tool. METHODS: The tool was tested in 257 medical and nursing students after their participation in one of several mass casualty simulations. RESULTS: Using exploratory and confirmatory factor analysis, the revised self-assessment teamwork tool was shown to have strong construct validity, high reliability, and the construct demonstrated invariance across groups (Medicine & Nursing). CONCLUSIONS: The modified SATT was shown to be a reliable and valid student self-assessment tool. The SATT is a quick and practical method of guiding students' reflection on important teamwork skills.

Development of a self-assessment teamwork tool for use by medical and nursing students.

BACKGROUND: Teamwork training is an essential component of health professional student education. A valid and reliable teamwork self-assessment tool could assist students to identify desirable teamwork behaviours with the potential to promote learning about effective teamwork. The aim of this study was to develop and evaluate a self-assessment teamwork tool for health professional students for use in the context of emergency response to a mass casualty. METHODS: The authors modified a previously published teamwork instrument designed for experienced critical care teams for use with medical and nursing students involved in mass casualty simulations. The 17-item questionnaire was administered to students immediately following the simulations. These scores were used to explore the psychometric properties of the tool, using Exploratory and Confirmatory Factor Analysis. RESULTS: 202 (128 medical and 74 nursing) students completed the self-assessment teamwork tool for students. Exploratory factor analysis revealed 2 factors (5 items - Teamwork coordination and communication; 4 items - Information sharing and support) and these were justified with confirmatory factor analysis. Internal consistency was 0.823 for Teamwork coordination and communication, and 0.812 for Information sharing and support. CONCLUSIONS: These data provide evidence to support the validity and reliability of the self-assessment teamwork tool for students This self-assessment tool could be of value to health professional students following team training activities to help them identify the attributes of effective teamwork.

Development and use of a research productivity assessment tool for clinicians in low-resource settings in the Pacific Islands: a Delphi study.

BACKGROUND: Research performance assessments have proliferated, but research indicators for use amongst clinicians in poorly resourced countries have been ill-defined. The aims of the present paper were to determine a set of indicators as determined by clinician participants from the Pacific Islands and a panel of research experts for use in the performance assessment of clinicians. METHODS: Two focus group discussions, one for nurses and one for doctors, were used to obtain the views of 28 Pacific Island clinicians of the BRRACAP Study about what the research indicators should be. A modified Delphi survey was used to obtain a consensus amongst 19 research experts, with Pacific Island research experience, as to what the indicators should be and then to rank these in terms of importance. A survey of the participants obtained data on the research tasks/actions performed 20 months after the initial research workshop. A resultant tool comprising of 21 indicators was used to assess the performance of 18 Pacific participants. RESULTS: The Pacific Island clinicians determined that research was important and that performance should be measured. They identified research indicators that could be used in their settings and ranked their importance using a points system. The panel of experts identified implementation of research findings, collaborations and actual change in practice as more important, with bibliometric measurements low down in the scale. Although only 64% of the 28 BRRACAP Study participants returned the questionnaire, 39% of those performed more than half of the 21 indicators used. Of the 18 Pacific clinicians assessed, 7 (39%) performed 10 or more tasks. CONCLUSIONS: A research performance assessment tool was developed using process and output indicators identified by Pacific clinicians and a panel of research experts. The tool, which placed emphasis on process and outputs that were not bibliometric based, proved useful in assessing the performance of Pacific clinicians working in a low-resource setting.

Exploring examiner judgement of professional competence in rater based assessment.

Exercise physiology courses have transitioned to competency based, forcing Universities to rethink assessment to ensure students are competent to practice. This study built on earlier research to explore rater cognition, capturing factors that contribute to assessor decision making about students' competency. The aims were to determine the source of variation in the examination process and document the factors impacting on examiner judgment. Examiner judgement was explored from both a quantitative and qualitative perspective. Twenty-three examiners viewed three video encounters of student performance on an OSCE. Once rated, analysis of variance was performed to determine where the variance was attributed. A semi-structured interview drew out the examiners reasoning behind their ratings. Results highlighted variability of the process of observation, judgement and rating, with each examiner viewing student performance from different lenses. However, at a global level, analysis of variance indicated that the examiner had a minimal impact on the variance, with the majority of variance explained by the student performance on task. One anomaly noted was in the assessment of technical competency, whereby the examiner had a large impact on the rating, linked to assessing according to curriculum content. The thought processes behind judgement were diverse and if the qualitative results had been used in isolation, may have led to the researchers drawing conclusions that the examined performances would have yielded widely different ratings. However, as a cohort, the examiners were able to distinguish good and poor levels of competency with the majority of student competency linked to the varying ability of the student.

Rethinking the admission criteria to nursing school.

The main objective of this study was to identify the best predictors for student achievements (Undergraduate Grade Point Average (UGPA)) in their first year in an undergraduate nursing programme. Data were acquired from the Tracking Project database which is held by the Faculty of Medical and Health Sciences at the University of Auckland. The data (n=134) included information on student demographics, final year secondary school achievements (National Certificate of Educational Achievement Grade Point Average (NCEAGPA) & NCEA Credits), university admission ranking scores, and achievements in first year in the undergraduate nursing programme (UGPA). Linear regression models were used to identify the best predictors for first year students' UGPA in the nursing programme. The regression models suggest that the best predictor for the first year GPA is the NCEAGPA (beta=.488; R(2)(for the entire model)=.53), followed by the admission ranking scores (beta=.308; R(2)=.40). Based on these findings, it is suggested that a Dual Admission Model (DAM) be utilised whereby students could be admitted either by the current university admission criteria or by an alternative model, which is purely based on the predictability of achievement within the nursing programme. Application of the DAM to other institutions/countries was discussed.