Using mobile technology in assessment of entrustable professional activities in undergraduate medical education.

BACKGROUND: The adoption of competency-based medical education requires objective assessments of a learner's capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum. APPROACH: A paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017-2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience. EVALUATION: The mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students' scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved. REFLECTION: Our preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.

Peer assessment of professionalism in undergraduate medical education.

BACKGROUND: Fostering professional behaviour has become increasingly important in medical education and non-traditional approaches to assessment of professionalism may offer a more holistic representation of students' professional behaviour development. Emerging evidence suggests peer assessment may offer potential as an alternative method of professionalism assessment. We introduced peer assessment of professionalism in pre-clerkship phases of undergraduate medical education curriculum at our institution and evaluated suitability of adopting a professional behaviour scale for longitudinal tracking of student development, and student comfort and acceptance of peer assessment. METHODS: Peer assessment was introduced using a validated professional behaviours scale. Students conducted repeated, longitudinal assessments of their peers from small-group, clinical skills learning activities. An electronic assessment system was used to collect peer assessments, collate and provide reports to students. Student opinions of peer assessment were initially surveyed before introducing the process, confirmatory analyses were conducted of the adopted scale, and students were surveyed to explore satisfaction with the peer assessment process. RESULTS: Students across all phases of the curriculum were initially supportive of anonymous peer assessment using small-group learning sessions. Peer scores showed improvement over time, however the magnitude of increase was limited by ceiling effects attributed to the adopted scale. Students agreed that the professional behaviours scale was easy to use and understand, however a majority disagreed that peer assessment improved their understanding of professionalism or was a useful learning experience. CONCLUSIONS: Peer assessment of professional behaviours does expose students to the process of assessing one's peers, however the value of such processes at early stages of medical education may not be fully recognized nor appreciated by students. Electronic means for administering peer assessment is feasible for collecting and reporting peer feedback. Improvement in peer assessed scores was observed over time, however student opinions of the educational value were mixed and indeterminate.

Evaluation of the Characteristics of a Workplace Assessment Form to Assess Entrustable Professional Activities (EPAs) in an Undergraduate Surgery Core Clerkship.

OBJECTIVE: Entrustable Professional Activities (EPAs) are explicit, directly observable tasks requiring the demonstration of specific knowledge, skills, and behaviors that learners are expected to perform without direct supervision once they have gained sufficient competence. Undergraduate level implementation of EPAs is relatively new. We examined the characteristics of a workplace assessment form (clinic card) as part of a formative programmatic assessment process of EPAs for a core undergraduate surgery rotation. DESIGN: A clinic card was introduced to assess progression towards EPA achievement in the clerkship curriculum phase. Students completing their core eight (8) week clerkship surgery rotation submitted at least 1 clinic card per week. We compiled assessment scores for the 2015 to 2016 academic year, in which EPAs were introduced, and analyzed relationships between scores and time, EPA, training site, and assessor role. We surveyed preceptors and students, and conducted a focus group with clinical discipline coordinators of all core rotations. SETTING: This study took place at the Faculty of Medicine, Memorial University in St. John's, Newfoundland, Canada. PARTICIPANTS: Third year medical students (n = 79) who completed their core eight (8) week surgery clerkship rotation during the 2015 to 2016 academic year, preceptors, and clinical discipline coordinators participated in this study. RESULTS: EPAs reflecting tasks commonly performed by students were more likely to be assessed. EPAs frequently observed during preceptor-student encounters had higher entrustment ratings. Most EPAs showed increased entrustment scores over time and no significant differences in ratings between teaching sites nor preceptors and residents. Survey and focus group feedback suggest clinic cards fostered direct observation by preceptors and promoted constructive feedback on clinical tasks. A binary rating scale (entrustable/pre-entrustable) was not educationally beneficial. CONCLUSIONS: The findings support the feasibility, utility, catalytic and educational benefits of clinic cards in assessing EPAs in a core surgery rotation in undergraduate medical education.

Rural professionals' perceptions of interprofessional continuing education in mental health.

We describe the impact of an interprofessional education programme in mental health for professionals in six rural Canadian communities. The 10-session programme, offered primarily via videoconference, focussed on eight domains of mental health practice. One hundred and twenty-five professionals, representing 15 professions, attended at least some sessions, although attendance was variable. Data were collected between September 2006 and December 2007. The programme was evaluated using a mixed methods approach. Participants reported high levels of satisfaction for all topics and all aspects of the presentations: they were most satisfied with the opportunity to interact with other professionals and least satisfied with the videoconference technology. Professionals' confidence (n = 49) with mental health interventions, issues and populations was measured pre- and post-programme. There was a significant increase in confidence for seven of the eight mental health interventions and four of the six mental health issues that had been taught in the programme. Participants reported developing a more reflective mental health practice, becoming more aware of mental health issues, integrating new knowledge and skills into their work and they expressed a desire for further mental health training. They noted that interprofessional referrals, inter-agency linkages and collaborations had increased. Conditions that appeared to underpin the programme's success included: scheduling the programme over an extended time period, a positive relationship between the facilitator and participants, experiential learning format and community co-ordinators as liaisons. Participants' dissatisfaction with the videoconference technology was mitigated by the strong connection between the facilitator and participants. One challenge was designing a curriculum that met the needs of professionals with varied expertise and work demands. The programme seemed to benefit most of those professionals who had a mental health background. This programme has the potential to be of use in rural communities where professionals often do not have access to professional development in mental health.

Factors influencing rural health care professionals' access to continuing professional education.

OBJECTIVES: The purposes of this study were to explore the perceived barriers and challenges to continuing professional education (CPE) access for Canadian health care professionals and to identify best practices for improving access to CPE. DESIGN: Key informant interviews and Web-based online surveys were conducted. PARTICIPANTS: Key informant interviews were conducted with national CPE accreditation bodies and health professional associations. An online survey was distributed to health professional education programs, as well as provincial professional associations, licensing and professional regulatory bodies. MAIN OUTCOME MEASURES: The perceived barriers and challenges to CPE access for Canadian health care professionals and best practices for improving access to CPE. RESULTS AND CONCLUSIONS: Geographic isolation and poor technological and telecommunications infrastructure were identified as key barriers to CPE delivery and access. Financial factors, such as funding to support travel or cost of attendance, were also identified as major challenges. Tele-education programming was identified as a best practice approach to improve CPE access, as were regional CPE activities and self-directed learning programs. Employer-sponsored initiatives, including staff coverage or locum support, remuneration for time off and paid travel expenses for CPE participation were also identified as best practice approaches.

A comparison of face-to-face versus remote assessment of neonatal resuscitation skills.

The neonatal resuscitation skills of 30 third-year medical students were assessed in real time by a face-to-face examiner in the same room as the student, and by a remote examiner located in a separate room using the ANAKIN system. The ANAKIN system combines an instrumented manikin simulator, computer-based assessment and high-bandwidth videoconferencing. The students were assessed while performing a neonatal resuscitation megacode using the ANAKIN system. Students were satisfied with the ANAKIN system as an assessment system and were not intimidated by its use. However, the correlation between the face-to-face and remote examiner's mean total performance assessment scores was 0.27, which was not significant (P=0.14). The results indicated variation between the examiners' performance scores in a number of key technical skill areas. The findings from this study have implications for the use of technology-mediated systems in assessing resuscitation skills. Examiner orientation is critical for individuals using such systems. These persons must be comfortable and confident in using the technology. Interface and design features of the system need to be carefully scrutinized and tested.