Exploring the usage of learning resources by medical students in the basic science stage and their effect on academic performance.

BACKGROUND: The United States Medical Licensing Examination (USMLE) step 1 is one of the two examinations written after completion of the first two years (basic science stage) of medical school to be eligible to apply for residency training in the USA. A huge number and types of study materials are available to prepare for the exam which might confuse students choosing a resource. We investigated learning resources being used by the third and fifth-semester medical students and their association with academic performance. We also compared learning resources and exam scores of high-performing and low-performing students. METHODS: Data collection was done using structured (quantitative study) and semi-structured (qualitative study) questionnaires during a face-to-face interview. This article is about the quantitative part which was designed as a correlational study. Single factor one-way analysis of variance (ANOVA), Pearson correlation coefficient test, T-test, and Fisher's exact test were used to analyze the data. RESULTS: About half of all students used three or more commercial resources dealing with the same content. A weak negative correlation was observed between the number of commercial resources and the exam scores, especially when the number of these resources was three or more (r = -0.26). The mean exam score of textbook users was statistically significantly higher than the mean score of textbook non-users (p = 0.01). The usage of textbooks was statistically significantly higher in the cohort of top performers in comparison to the rest of the students (p = 0.006). In addition to less usage of textbooks, the mean number of review books was higher in the group of weakest students (2.84 versus 3.7; p = 0.75). CONCLUSIONS: Most students did not use professional textbooks and about half used too many commercial review resources. While the former fact was significantly associated with poor academic performance, the later fact had weak negative correlation with exam score. Pedagogical interventions are urgently needed to make the right type of learning resources available by making professional textbooks more USMLE-oriented and helping the students choose the best and right number of resources for optimum academic performance. By fulfilling the observed needs of the students in this way, they might feel empowered because of self-determination which will motivate studies.

Peer assessment of professionalism in undergraduate medical education.

BACKGROUND: Fostering professional behaviour has become increasingly important in medical education and non-traditional approaches to assessment of professionalism may offer a more holistic representation of students' professional behaviour development. Emerging evidence suggests peer assessment may offer potential as an alternative method of professionalism assessment. We introduced peer assessment of professionalism in pre-clerkship phases of undergraduate medical education curriculum at our institution and evaluated suitability of adopting a professional behaviour scale for longitudinal tracking of student development, and student comfort and acceptance of peer assessment. METHODS: Peer assessment was introduced using a validated professional behaviours scale. Students conducted repeated, longitudinal assessments of their peers from small-group, clinical skills learning activities. An electronic assessment system was used to collect peer assessments, collate and provide reports to students. Student opinions of peer assessment were initially surveyed before introducing the process, confirmatory analyses were conducted of the adopted scale, and students were surveyed to explore satisfaction with the peer assessment process. RESULTS: Students across all phases of the curriculum were initially supportive of anonymous peer assessment using small-group learning sessions. Peer scores showed improvement over time, however the magnitude of increase was limited by ceiling effects attributed to the adopted scale. Students agreed that the professional behaviours scale was easy to use and understand, however a majority disagreed that peer assessment improved their understanding of professionalism or was a useful learning experience. CONCLUSIONS: Peer assessment of professional behaviours does expose students to the process of assessing one's peers, however the value of such processes at early stages of medical education may not be fully recognized nor appreciated by students. Electronic means for administering peer assessment is feasible for collecting and reporting peer feedback. Improvement in peer assessed scores was observed over time, however student opinions of the educational value were mixed and indeterminate.

A randomized controlled study of manikin simulator fidelity on neonatal resuscitation program learning outcomes.

The neonatal resuscitation program (NRP) has been developed to educate physicians and other health care providers about newborn resuscitation and has been shown to improve neonatal resuscitation skills. Simulation-based training is recommended as an effective modality for instructing neonatal resuscitation and both low and high-fidelity manikin simulators are used. There is limited research that has compared the effect of low and high-fidelity manikin simulators for NRP learning outcomes, and more specifically on teamwork performance and confidence. The purpose of this study was to examine the effect of using low versus high-fidelity manikin simulators in NRP instruction. A randomized posttest-only control group study design was conducted. Third year undergraduate medical students participated in NRP instruction and were assigned to an experimental group (high-fidelity manikin simulator) or control group (low-fidelity manikin simulator). Integrated skills station (megacode) performance, participant satisfaction, confidence and teamwork behaviour scores were compared between the study groups. Participants in the high-fidelity manikin simulator instructional group reported significantly higher total scores in overall satisfaction (p = 0.001) and confidence (p = 0.001). There were no significant differences in teamwork behaviour scores, as observed by two independent raters, nor differences on mandatory integrated skills station performance items at the p < 0.05 level. Medical students' reported greater satisfaction and confidence with high-fidelity manikin simulators, but did not demonstrate overall significantly improved teamwork or integrated skills station performance. Low and high-fidelity manikin simulators facilitate similar levels of objectively measured NRP outcomes for integrated skills station and teamwork performance.

Making medical student course evaluations meaningful: implementation of an intensive course review protocol.

BACKGROUND: Ongoing course evaluation is a key component of quality improvement in higher education. The complexities associated with delivering high quality medical education programs involving multiple lecturers can make course and instructor evaluation challenging. We describe the implementation and evaluation of an "intensive course review protocol" in an undergraduate medical program METHODS: We examined pre-clerkship courses from 2006 to 2011 - prior to and following protocol implementation. Our non-parametric analysis included Mann-Whitney U tests to compare the 2006/07 and 2010/11 academic years. RESULTS: We included 30 courses in our analysis. In the 2006/07 academic year, 13/30 courses (43.3 %) did not meet the minimum benchmark and were put under intensive review. By 2010/11, only 3/30 courses (10.0 %) were still below the minimum benchmark. Compared to 2006/07, courses ratings in the 2010/11 year were significantly higher (p = 0.004). However, during the study period mean response rates fell from 76.5 % in 2006/07 to 49.7 % in 2010/11. CONCLUSION: These results suggest an intensive course review protocol can have a significant impact on pre-clerkship course ratings in an undergraduate medical program. Reductions in survey response rates represent an ongoing challenge in the interpretation of student feedback.

Interprofessional mental health training in rural primary care: findings from a mixed methods study.

The benefits of interprofessional care in providing mental health services have been widely recognized, particularly in rural communities where access to health services is limited. There continues to be a need for more continuing interprofessional education in mental health intervention in rural areas. There have been few reports of rural programs in which mental health content has been combined with training in collaborative practice. The current study used a sequential mixed-method and quasi-experimental design to evaluate the impact of an interprofessional, intersectoral education program designed to enhance collaborative mental health capacity in six rural sites. Quantitative results reveal a significant increase in positive attitudes toward interprofessional mental health care teams and self-reported increases in knowledge and understanding about collaborative mental health care delivery. The analysis of qualitative data collected following completion of the program, reinforced the value of teaching mental health content within the context of collaborative practice and revealed practice changes, including more interprofessional and intersectoral collaboration. This study suggests that imbedding explicit training in collaborative care in content focused continuing professional education for more complex and chronic health issues may increase the likelihood that professionals will work together to effectively meet client needs.

The use of information and communications technologies in the delivery of interprofessional education: A review of evaluation outcome levels.

Interprofessional education (IPE) in health and human services educational and clinical settings has proliferated internationally. The use of information and communication technologies (ICTs) in the facilitation of interprofessional learning is also growing, yet reviews of the effectiveness of ICTs in the delivery of pre- and/or post-licensure IPE have been limited. The current study's purpose was to review the evaluation outcomes of IPE initiatives delivered using ICTs. Relevant electronic databases and journals from 1996 to 2013 were searched. Studies which evaluated the effectiveness of an IPE intervention using ICTs were included and analyzed using the Barr et al. modified Kirkpatrick educational outcomes typology. Fifty-five studies were identified and a majority reported evaluation findings at the level 1 (reaction/satisfaction). Analysis revealed that learners react favorably to the use of ICTs in the delivery of IPE, and ICT-mediated IPE can lead to positive attitudinal and knowledge change. A majority of the studies reported positive evaluation outcomes at the learner satisfaction level, with the use of web-based learning modalities. The limited number of studies at other levels of the outcomes typology and deficiencies in study designs indicate the need for more rigorous evaluation of outcomes in ICT-mediated IPE.

Toward a common understanding: supporting and promoting education scholarship for medical school faculty.

CONTEXT: Education scholarship (ES) is integral to the transformation of medical education. Faculty members who engage in ES need encouragement and recognition of this work. Beginning with the definition of ES as 'an umbrella term which can encompass both research and innovation in health professions education', and which as such represents an activity that is separate and distinct from teaching and leadership, the purpose of our study was to explore how promotion policies and processes are used in Canadian medical schools to support and promote ES. METHODS: We conducted an analysis of the promotion policies of 17 Canadian medical schools and interviews with a key informant at each institution. We drew on an interpretive approach to policy analysis to analyse the data and to understand explicit messages about how ES was represented and supported. RESULTS: Of the 17 schools' promotion documents, only nine contained specific reference to ES. There was wide variation in focus and level of detail. All key informants indicated that ES is recognised and considered for academic promotion. Barriers to the support and recognition of ES included a lack of understanding of ES and its relationship to teaching and leadership. This was manifest in the variability in promotion policies and processes, support systems, and career planning and pathways for ES. CONCLUSIONS: This lack of clarity may make it challenging for medical school faculty members to make sense of how they might successfully align ES within an academic career. There is a need therefore to better articulate ES in promotion policies and support systems. Creating a common understanding of ES, developing guidelines to assess the impact of all forms of ES, developing an informed leadership and system of mentors, and creating explicit role descriptions and guidelines are identified as potential strategies to ensure that ES is appropriately valued.

Reliability of the interprofessional collaborator assessment rubric (ICAR) in multi source feedback (MSF) with post-graduate medical residents.

BACKGROUND: Increased attention on collaboration and teamwork competency development in medical education has raised the need for valid and reliable approaches to the assessment of collaboration competencies in post-graduate medical education. The purpose of this study was to evaluate the reliability of a modified Interprofessional Collaborator Assessment Rubric (ICAR) in a multi-source feedback (MSF) process for assessing post-graduate medical residents' collaborator competencies. METHODS: Post-graduate medical residents (n = 16) received ICAR assessments from three different rater groups (physicians, nurses and allied health professionals) over a four-week rotation. Internal consistency, inter-rater reliability, inter-group differences and relationship between rater characteristics and ICAR scores were analyzed using Cronbach's alpha, one-way and two-way repeated measures ANOVA, and logistic regression. RESULTS: Missing data decreased from 13.1% using daily assessments to 8.8% utilizing an MSF process, p = .032. High internal consistency measures were demonstrated for overall ICAR scores (α = .981) and individual assessment domains within the ICAR (α = .881 to .963). There were no significant differences between scores of physician, nurse, and allied health raters on collaborator competencies (F2,5 = 1.225, p = .297, η2 = .016). Rater gender was the only significant factor influencing scores with female raters scoring residents significantly lower than male raters (6.12 v. 6.82; F1,5 = 7.184, p = .008, η 2 = .045). CONCLUSION: The study findings suggest that the use of the modified ICAR in a MSF assessment process could be a feasible and reliable assessment approach to providing formative feedback to post-graduate medical residents on collaborator competencies.

Effects of Simulation Fidelity on Health Care Providers on Team Training-A Systematic Review.

ABSTRACT: This systematic review, following PRISMA standards, aimed to assess the effectiveness of higher versus lower fidelity simulation on health care providers engaged in team training. A comprehensive search from January 1, 2011 to January 24, 2023 identified 1390 studies of which 14 randomized (n = 1530) and 5 case controlled (n = 257) studies met the inclusion criteria. The certainty of evidence was very low due to a high risk of bias and inconsistency. Heterogeneity prevented any metaanalysis. Limited evidence showed benefit for confidence, technical skills, and nontechnical skills. No significant difference was found in knowledge outcomes and teamwork abilities between lower and higher fidelity simulation. Participants reported higher satisfaction but also higher stress with higher fidelity materials. Both higher and lower fidelity simulation can be beneficial for team training, with higher fidelity simulation preferred by participants if resources allow. Standardizing definitions and outcomes, as well as conducting robust cost-comparative analyses, are important for future research.

A Phenomenological Study of Postgraduate Medical Trainees' Incidental Learning Experiences and Psychological Well-Being During the COVID-19 Pandemic.

BACKGROUND:  During the COVID-19 pandemic, postgraduate medical trainees contributed significantly to the healthcare workforce, as multiple vulnerabilities in the healthcare system and medical training were expounded. The burden of training, learning, and working at this time introduced unique psychological and emotional stressors within a context of generalized volatility and radically different ways to work and learn. This study explored postgraduate trainees' experiences with coping, managing, and learning during the COVID-19 pandemic. METHODOLOGY:  Using a phenomenological approach, semistructured interviews were conducted with an intradisciplinary sample (n = 8) of postgraduate trainees in Newfoundland and Labrador, Canada, between May and October 2022. Five researchers performed inductive and deductive thematic analysis to develop a coding structure and identify common themes. RESULTS:  The COVID-19 pandemic prompted the use of restrictive public health measures and an unprecedented shift from in-person to virtual learning. This affected trainees' exposure to normalized learning experiences, training rotations, and opportunities to learn from peers and staff. Certainly, trainees reported that virtual learning improved their educational experiences in unique ways, increased engagement and attendance, and enabled regular meetings and learning when in-person options were unavailable. Trainees also reported enhanced self-directed learning skills, greater ownership of and leadership in their education, and increased confidence and experience with virtual care. Some also reported a perceived increase in elements of emotional intelligence (e.g., self-awareness, empathy, and compassion). CONCLUSIONS:  Trainees reported a variety of incidental learning experiences from working and training during COVID-19. Although some experiences were challenging, there was a perception that such experiences led to new learnings that were beneficial to one's professional development and future career, as well as implications for future training provided to trainees. While there was a reported shift in the culture surrounding postgraduate trainees' health and safety, respondents also described the need for additional support for postgraduate trainees' well-being during a pandemic.

Training for virtual care: What do the experts think?

Introduction: Virtual care has expanded during COVID-19 and enabled continued access to healthcare services. As with the introduction of any new technology in healthcare delivery, the preparation of healthcare providers for adopting and using such systems is imperative. The purpose of this qualitative study was to explore experts' ascribed opinions on healthcare providers' continuing professional development (CPD) needs in virtual care. Methods: Semistructured interviews were conducted with a purposive sample of key informants representing Canadian provincial and national organizations with expertise in virtual care delivery. Results: Three main areas of knowledge, skills, and abilities that would be most helpful for healthcare providers in preparing to adopt and use virtual care were identified. The use of technology necessitates knowledge of how to integrate technology and virtual care in the practice workflow. This includes knowing how to use the technology and the privacy and security of the technology. Providers need to be able to adapt their clinical skills to virtual care and build rapport through good communication with patients. Virtual care is not appropriate for all visits, therefore providers need to understand when an in-person visit is necessary with respect to the nature of the appointment, as well as contextual factors for individual patients. Finally, providers need to adapt their examination skills to virtual care. Discussion: Beyond the COVID-19 pandemic, virtual care will have a continuing role in enhancing continuity of care through access that is more convenient. Key informants identified barriers and challenges in adopting and using virtual care effectively, fundamental knowledge, skills and/or abilities required, and important topics and/or educational experiences to guide CPD program development on virtual care for healthcare providers.

Patient Experiences With Virtual Care During the COVID-19 Pandemic: Phenomenological Focus Group Study.

BACKGROUND: Virtual care has expanded during the COVID-19 pandemic and enabled greater access and continuity of care for many patients. From a patient-oriented research perspective, understanding the patient experience with virtual care appointments is an important first step in identifying ways to better support patient use and satisfaction. OBJECTIVE: The purpose of this qualitative study was (1) to explore patients' experiences and perspectives with the adoption and use of virtual care during COVID-19 in Newfoundland and Labrador, Canada, and (2) identify the education and informational needs of patients to inform future strategies for supporting patient use of virtual care. METHODS: Using a phenomenological approach, we conducted a focus group interview with a purposive sample of patient representatives representing a cross-section of the population of the province of Newfoundland and Labrador. Five patient representatives were recruited from the Newfoundland and Labrador Support Patient Advisory Council and participated in the focus group. The focus group was conducted in February 2022 via videoconferencing technology. Using thematic analysis, we identified several recurrent themes that described respondents' experiences with the use of virtual care during COVID-19, as well as their perceptions of education and informational needs to support more effective patient use of virtual care. RESULTS: Respondents felt that virtual care is a beneficial addition to the health care system, enabling greater convenience and access to health care services. Key barriers and challenges in adopting and using virtual care appear to primarily arise from patients' lack of knowledge, understanding, and familiarity with respect to virtual care. Cost, technological access, connectivity, and low digital literacy were challenges for some patients, particularly in rural communities and among older patient population. Patient education and support were critical and needed to be inclusive, easy to understand, and include information regarding privacy, security, consent, and the technology itself. The types of patient education experiences regarded as most helpful included peer support and knowledge sharing among patients themselves. CONCLUSIONS: Beyond the COVID-19 pandemic, virtual care will have a continuing role in enhancing the continuity of care for patients through more convenient access. The education and informational needs of patients are important considerations in promoting the adoption and use of virtual care. Key education and informational needs and strategies were identified to enable and empower patients with the knowledge, digital literacy skills, and support to effectively use virtual care.

Use of Extended Reality in Medical Education: An Integrative Review.

Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-022-01698-4.

An approach to integrating interprofessional education in collaborative mental health care.

OBJECTIVE: This article describes an evaluation of a curriculum approach to integrating interprofessional education (IPE) in collaborative mental health practice across the pre- to post-licensure continuum of medical education. METHODS: A systematic evaluation of IPE activities was conducted, utilizing a combination of evaluation study designs, including: pretest-posttest control group; one-group pre-test-post-test; and one-shot case study. Participant satisfaction, attitudes toward teamwork, and self-reported teamwork abilities were key evaluative outcome measures. RESULTS: IPE in collaborative mental health practice was well received at both the pre- and post-licensure levels. Satisfaction scores were very high, and students, trainees, and practitioners welcomed the opportunity to learn about collaboration in the context of mental health. Medical student satisfaction increased significantly with the introduction of standardized patients (SPs) as an interprofessional learning method. Medical students and faculty reported that experiential learning in practice-based settings is a key component of effective approaches to IPE implementation. At a post-licensure level, practitioners reported significant improvement in attitudes toward interprofessional collaboration in mental health care after participation in IPE. CONCLUSION: IPE in collaborative mental health is feasible, and mental health settings offer practical and useful learning experiences for students, trainees, and practitioners in interprofessional collaboration.

E-learning use in the review of neonatal resuscitation program in physicians: a scoping review.

OBJECTIVE: To determine if e-learning interventions are efficient to review Neonatal Resuscitation Program (NRP) and to prevent performance deterioration in neonatal resuscitation of already-certified healthcare professionals. STUDY DESIGN: In this scoping review, we searched for manuscripts published until June 2020 in five databases. We included all studies on e-learning use for NRP review in already-certified healthcare providers. RESULTS: Among 593 abstracts retrieved, 38 full-text articles were assessed for eligibility. Five studies were included. Four studies evaluated the effectiveness of e-learning interventions immediately or months after their completion by providers. These interventions did not consistently enhance their NRP knowledge and their performance. One study showed that a growth mindset can influence positively neonatal resuscitation performance after an e-learning simulation. CONCLUSION: There is not enough evidence to conclude that e-learning interventions can prevent neonatal resuscitation knowledge and performance decay in already-certified providers. More research is needed on the use of e-learning simulation-based scenarios to improve NRP retention.

Virtual care and COVID-19: A survey study of adoption, satisfaction and continuing education preferences of healthcare providers in Newfoundland and Labrador, Canada.

Introduction: Virtual care has expanded during COVID-19 and enabled continued access to healthcare services. For many healthcare providers, the adoption of virtual care has been a new experience in the provision of healthcare services. The purpose of this survey study was to explore healthcare providers' experiences with virtual care during COVID-19. Methods: A web-based survey-questionnaire was developed by applying Rogers' theory of diffusion of innovation and distributed to healthcare providers (physicians, nurses and allied health professionals) in Newfoundland and Labrador, Canada to explore virtual care experiences, satisfaction and continuing professional development (CPD) needs. Analyses included descriptive statistics and thematic analysis of survey responses. Results: Fifty-one percent of respondents (n = 432) indicated they were currently offering virtual care and a majority (68.9%) reported it has improved their work experience. Telephone appointments were preferred over videoconferencing by respondents, with key challenges including the inability to conduct a physical exam, patients' cell phone services being unreliable and patients knowing how to use videoconferencing. Majority of respondents (57.5%) reported quality of care by telephone was lower than in-person, whereas quality of care by videoconferencing was equivalent to in-person. Main benefits of virtual care included increased patient access, ability to work from home, and reduction in no-show appointments. Key supports for adopting virtual care included in-house organizational supports (e.g., technical support staff), local colleague support, and technology training. Important topics for virtual care CPD included complying with regulatory standards/rules, understanding privacy or ethical boundaries, and developing competency and digital professionalism while engaging in virtual care. Discussion: Beyond the COVID-19 pandemic, virtual care will have a continuing role in enhancing continuity of care through access that is more convenient. Survey findings reveal a number of opportunities for supporting healthcare providers in use of virtual care, including CPD, guidelines and resources to support adaptation to virtual care provision (e.g., virtual examinations/assessments), as well as patient educational support.

Fostering "Reflection-On-Practice" Through a Multisource Feedback and Peer Coaching Pilot Program.

INTRODUCTION: Reflective practice involves thinking about one's practice and often involves using data to effect such reflection. Multisource feedback (MSF) involves evaluation by peers, patients, and coworkers. Coaching has been identified as a key aspect of MSF with peer coaching involving two or more colleagues working together to reflect on current practices and share ideas. We introduced a pilot MSF and peer coaching program with a goal to evaluate its effect on fostering reflective practice. METHODS: Physician participants completed a 360-degree assessment of their practices, followed by peer coaching sessions. Peer coaches were oriented to an evidence-based theory-driven feedback model (R2C2) to support coaching skills development. A mixed-methods evaluation study was undertaken, including pre to post surveys of readiness for self-directed learning, a postevaluation survey of participant satisfaction, and semistructured participant interviews. RESULTS: Thirty four (N = 34) participants completed the 360-degree assessment, and 22 participants took part in two coaching meetings. Respondents reported significant improvement to aspects of their readiness for self-directed learning (P <.05), including knowing about learning strategies to achieve key learning goals, knowing about resources to support one's own learning, and being able to evaluate one's learning outcomes. Overall, respondents felt empowered to "reflect" on their practices, affirm what they were doing well, and, for some, identify opportunities for further and ongoing professional development. DISCUSSION: MSF and peer coaching emerged as key elements in enabling reflective practice by facilitating reflection on one's practice and conversations with one's peers to affirm strengths and opportunities for strengthening practice through self-directed professional development.

Training family medicine residents to care for children: What is the best approach?

Problem addressed There is a lack of consensus around the optimal way to train family medicine residents to care for children. Objective of program Evaluation of an ambulatory versus an inpatient pediatrics rotation for family medicine residents. Program description A 4-week pediatrics rotation for second-year family medicine residents was introduced involving half-day ambulatory pediatric clinics. A nonequivalent control group evaluation study design was followed. Patient logbook entries, as well as residents' satisfaction, knowledge, and self-reported confidence outcomes were compared between family medicine residents completing the new ambulatory rotation and those completing a traditional inpatient-ambulatory pediatrics rotation. Conclusion An ambulatory rotation in pediatrics is a feasible option for facilitating family medicine resident learning in child health care. Residents report exposure to more patient cases that reflect a family practice office setting and the same level of knowledge and confidence as residents completing an inpatient-ambulatory rotation. Intraprofessional collaboration, flexibility in scheduling, and the support of pediatric preceptors are key factors in the organization and implementation of an ambulatory rotation.

Development and validation of the interprofessional collaborator assessment rubric (ICAR).

There have been increasing calls for a competency-based approach in interprofessional education (IPE). The purpose of this multi-site research project was to develop a validated set of interprofessional collaborator competencies and an associated competency-based assessment rubric, in both English and French languages. The first phase involved a detailed comparative analysis of peer-reviewed and grey literature using typological analysis to construct a draft list of interprofessional collaborator competency categories and statements. A two-round Delphi survey of experts was undertaken to validate these competencies. In the second phase, an assessment rubric was developed based on the validated competencies and then evaluated for utility, clarity, practicality and fairness through multi-site focus groups with students and faculty at both college and university levels. The paper outlines an approach to developing, constructing and validating a bilingual instrument for interprofessional learning and assessment. The approach was collaborative in nature, involving an interprofessional project team and respondents from across multiple health profession education programs. The Delphi survey ratings indicate a high level of agreement with the importance of the competency statements and focus group participants rated the rubric positively and felt it had value. The focus group results were also useful in pre-piloting the contextual application of the instrument across multiple health profession education programs. This rubric instrument may be used across a variety of professions and learning contexts. Future work includes evaluation of further dimensions of validity and reliability for this tool across a variety of settings.

Using mobile technology in assessment of entrustable professional activities in undergraduate medical education.

BACKGROUND: The adoption of competency-based medical education requires objective assessments of a learner's capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum. APPROACH: A paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017-2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience. EVALUATION: The mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students' scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved. REFLECTION: Our preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.