The Ottawa resident observation form for nurses (O-RON): evaluation of an assessment tool's psychometric properties in different specialties.

BACKGROUND: Workplace-based assessment (WBA) used in post-graduate medical education relies on physician supervisors' feedback. However, in a training environment where supervisors are unavailable to assess certain aspects of a resident's performance, nurses are well-positioned to do so. The Ottawa Resident Observation Form for Nurses (O-RON) was developed to capture nurses' assessment of trainee performance and results have demonstrated strong evidence for validity in Orthopedic Surgery. However, different clinical settings may impact a tool's performance. This project studied the use of the O-RON in three different specialties at the University of Ottawa. METHODS: O-RON forms were distributed on Internal Medicine, General Surgery, and Obstetrical wards at the University of Ottawa over nine months. Validity evidence related to quantitative data was collected. Exit interviews with nurse managers were performed and content was thematically analyzed. RESULTS: 179 O-RONs were completed on 30 residents. With four forms per resident, the ORON's reliability was 0.82. Global judgement response and frequency of concerns was correlated (r = 0.627, P < 0.001). CONCLUSIONS: Consistent with the original study, the findings demonstrated strong evidence for validity. However, the number of forms collected was less than expected. Exit interviews identified factors impacting form completion, which included clinical workloads and interprofessional dynamics.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Abdominal MRI on a Commercial 0.55T System: Initial Evaluation and Comparison to Higher Field Strengths.

RATIONALE AND OBJECTIVES: This study aims to assess the quality of abdominal MR images acquired on a commercial 0.55T scanner and compare these images with those acquired on conventional 1.5T/3T scanners in both healthy subjects and patients. MATERIALS AND METHODS: Fifteen healthy subjects and 52 patients underwent abdominal Magnetic Resonance Imaging at 0.55T. Images were also collected in healthy subjects at 1.5T, and comparison 1.5/3T images identified for 28 of the 52 patients. Image quality was rated by two radiologists on a 4-point Likert scale. Readers were asked whether they could answer the clinical question for patient studies. Wilcoxon signed-rank test was used to test for significant differences in image ratings and acquisition times, and inter-reader reliability was computed. RESULTS: The overall image quality of all sequences at 0.55T were rated as acceptable in healthy subjects. Sequences were modified to improve signal-to-noise ratio and reduce artifacts and deployed for clinical use; 52 patients were enrolled in this study. Radiologists were able to answer the clinical question in 52 (reader 1) and 46 (reader 2) of the patient cases. Average image quality was considered to be diagnostic (>3) for all sequences except arterial phase FS 3D T1w gradient echo (GRE) and 3D magnetic resonance cholangiopancreatography for one reader. In comparison to higher field images, significantly lower scores were given to 0.55T IP 2D GRE and arterial phase FS 3D T1w GRE, and significantly higher scores to diffusion-weighted echo planar imaging at 0.55T; other sequences were equivalent. The average scan time at 0.55T was 54 ± 10 minutes vs 36 ± 11 minutes at higher field strengths (P < .001). CONCLUSION: Diagnostic-quality abdominal MR images can be obtained on a commercial 0.55T scanner at a longer overall acquisition time compared to higher field systems, although some sequences may benefit from additional optimization.

Exploring gender influences in the quality of workplace-based assessments.

INTRODUCTION: Workplace-based assessments are an important tool for trainee feedback and as a means of reporting expert judgments of trainee competence in the workplace. However, the literature has demonstrated that gender bias can exist within these assessments. We aimed to determine whether gender differences in the quality of workplace-based assessment data exist in our residency training program. METHODS: This study was conducted at the University of Ottawa in the Department of Emergency Medicine. Four end-of-shift workplace-based assessments completed by men faculty and four completed by women faculty were randomly selected for each resident during the 2018-2019 academic year. Two blinded raters scored each workplace-based assessment using the Completed Clinical Evaluation Report Rating (CCERR), a published nine-item quantitative measure of workplace-based assessment quality. A 2 × 2 mixed measures analysis of variance (ANOVA) of resident gender and faculty gender was conducted, with mean CCERR score as the dependent variable. The ANOVA was repeated with mean workplace-based assessment rating as the dependent variable. RESULTS: A total of 363 workplace-based assessments were analyzed for 46 residents. There were no significant effects of faculty or resident gender on the quality of workplace-based assessments (p = 0.30). There was no difference in mean workplace-based assessment ratings between women and men residents (p = 0.92), and no interaction between resident and faculty gender (p = 0.62). Mean CCERR score was 25.8, SD = 4.2, indicating average quality assessments. CONCLUSIONS: We did not find faculty or resident gender differences in the quality of workplace-based assessments completed in our training program. While the literature has previously demonstrated gender bias in trainee assessments, our results are not surprising as assessment culture varies by institution and program. Our study cautions against generalizing gender bias across contexts, and offers an approach that educators can use to evaluate whether gender bias in the quality of trainee assessments exists within their program.

RéSUMé: INTRODUCTION: Les évaluations sur le lieu de travail constituent un outil important pour le retour d'information des stagiaires et comme moyen de rapporter les jugements d'experts sur les compétences des stagiaires sur le lieu de travail. Cependant, la littérature a démontré que des préjugés sexistes peuvent exister dans ces évaluations. Nous avons cherché à déterminer s'il existe des différences entre les sexes dans la qualité des données d'évaluation sur le lieu de travail dans notre programme de formation en résidence. MéTHODES: Cette étude a été menée à l'Université d'Ottawa dans le département de médecine d'urgence. Quatre évaluations en fin de poste de travail complétées par des professeurs hommes et 4 complétées par des professeurs femmes ont été sélectionnées au hasard pour chaque résident au cours de l'année universitaire 2018-2019. Deux évaluateurs en aveugle ont noté chaque évaluation sur le lieu de travail à l'aide du Completed Clinical Evaluation Report Rating (CCERR), une mesure quantitative publiée en neuf points de la qualité de l'évaluation sur le lieu de travail. Une analyse de variance (ANOVA) à mesures mixtes 2 × 2 du sexe des résidents et du sexe des enseignants a été réalisée, avec le score CCERR moyen comme variable dépendante. L'ANOVA a été répétée en prenant comme variable dépendante la note moyenne de l'évaluation sur le lieu de travail. RéSULTATS: Au total, 363 évaluations sur le lieu de travail ont été analysées pour 46 résidents. Il n'y avait aucun effet significatif du sexe du corps professoral ou du résident sur la qualité des évaluations en milieu de travail (p = 0,30). Il n'y avait pas de différence dans les évaluations moyennes sur le lieu de travail entre les femmes et les hommes résidents (p = 0,92), et pas d'interaction entre le sexe du résident et celui de la faculté (p = 0,62). Le score moyen du CCERR était de 25,8, SD = 4,2, ce qui indique des évaluations de qualité moyenne. CONCLUSIONS: Nous n'avons pas constaté de différences entre les sexes au sein du corps professoral ou des résidents en ce qui concerne la qualité des évaluations en milieu de travail effectuées dans le cadre de notre programme de formation. Bien que la littérature ait déjà démontré l'existence de préjugés sexistes dans les évaluations des stagiaires, nos résultats ne sont pas surprenants car la culture de l'évaluation varie selon les établissements et les programmes. Notre étude met en garde contre la généralisation des préjugés sexistes dans tous les contextes et propose une approche que les éducateurs peuvent utiliser pour évaluer s'il existe des préjugés sexistes dans la qualité des évaluations des stagiaires au sein de leur programme.

Cross slope gait biomechanics for individuals with and without a unilateral transtibial amputation.

BACKGROUND: This research was conducted to better understand compensatory strategies during cross-slope walking for adults with and without a unilateral transtibial amputation. METHODS: Fourteen individuals with unilateral transtibial amputation and 14 individuals with no lower limb amputation participated in this study. Motion and force data were captured while participants walked on a treadmill in a virtual reality environment for level and ± 5° cross slopes. Temporal-spatial parameters, kinematics (ankle, knee, hip, pelvis, trunk), and ground reaction forces were examined. FINDINGS: Compared to level, participants had similar step width but slightly longer steps for top-cross-slope and slightly shorter steps for bottom-cross-slope. Top-cross-slope required a more flexed limb with ankle eversion, and bottom-cross-slope required a more extended limb with ankle inversion. Participants had similar lateral pelvis and trunk motion for all walking conditions, but slightly more anterior trunk lean for top cross-slope with more anterior trunk lean observed for individuals with a lower limb amputation than without lower limb amputation. Participants with a lower limb amputation compensated for limited prosthetic ankle-foot dorsiflexion on the top-cross-slope by increasing prosthetic side hip flexion, reducing intact ankle/knee flexion, and increasing intact push-off force. INTERPRETATION: Gait adaptations during cross-slope walking were primarily in the lower extremities and were largely similar for those with and without a transtibial amputation. The information presented in this paper provides a better understanding of gait strategies adopted during cross-slope walking and can guide researchers and industry in prosthetic development.

Navigating the uncertainty of health advocacy teaching and evaluation from the trainee's perspective.

BACKGROUND: There may be no competency more shrouded in uncertainty than health advocacy (HA), raising questions about the robustness of advocacy training in postgraduate medical education. By understanding how programs currently train HA, we can identify whether trainees' learning needs are being met. METHODS: From 2017 to 2019, we reviewed curricular documents across nine direct-entry specialties at all Ontario medical schools, comparing content for the HA and communicator roles to delineate role-specific challenges. We then conducted semi-structured interviews with trainees (n = 9) and faculty (n = 6) to review findings and discuss their impact. Data were analyzed using thematic content analysis. RESULTS: Curricular documents revealed vague objectives and ill-defined modes of assessment for both intrinsic roles. This uncertainty was perceived as more problematic for HA, in part because HA seemed both undervalued in, and disconnected from, clinical learning. Trainees felt that the onus was on them to figure out how to develop and demonstrate HA competence, causing many to turn their learning attention elsewhere. DISCUSSION: Lack of curricular focus seems to create the perception that advocacy isn't valuable, deterring trainees-even those keen to become competent advocates-from developing HA skills. Such ambivalence may have troubling downstream effects for both patient care and trainees' professional development.

eConsult Specialist Quality of Response (eSQUARE): A novel tool to measure specialist correspondence via electronic consultation.

High-quality correspondence between healthcare providers is critical for effective patient care. We developed an assessment tool to measure the quality of specialist correspondence to primary care providers (PCPs) via electronic consultation (eConsult), where specialists provide advice without specialist-patient interactions. We incorporated fourteen previously described features of high-quality eConsult correspondence into an assessment tool named the eConsult Specialist Quality of Response (eSQUARE). Six PCPs and two specialists applied the 10-item eSQUARE tool to 30 eConsults of varying quality as informed by PCP survey data. Content, response process, and internal structure validity evidence was gathered. Psychometric properties were calculated using descriptive statistics and generalizability analyses. Mean total score for low-quality eConsults (M = 24 ± 5.6) was significantly lower than moderate-quality eConsults (M = 38 ± 4.7; p<0.001) which was significantly lower than high-quality eConsults (M = 46 ± 3.0; p = 0.002). Reliability measures were high, including generalizability coefficient (0.96), inter-item (≥0.55) and item-total correlations (≥0.68). A decision study demonstrated that a single rater was adequate to achieve a reliability measure of ≥0.70. This study demonstrates initial validity evidence including multiple reliability measures for the eSQUARE. A single rater is adequate to achieve reliability measures for formative feedback. Future studies can apply the eSQUARE when planning educational initiatives aiming to improve specialist-to-PCP correspondence via eConsult.

Improving MR Image Quality in Patients with Metallic Implants.

The number of implanted devices such as orthopedic hardware and cardiac implantable devices continues to increase with an increase in the age of the patient population, as well as an increase in the number of indications for specific devices. Many patients with these devices have or will develop clinical conditions that are best depicted at MRI. However, implanted devices containing paramagnetic or ferromagnetic substances can cause significant artifact, which could limit the diagnostic capability of this modality. Performing imaging with MRI when an implant is present may be challenging, and there are numerous techniques the radiologist and technologist can use to help minimize artifacts related to implants. First, knowledge of the presence of an implant before patient arrival is critical to ensure safety of the patient when the device is subjected to a strong magnetic field. Once safety is ensured, the examination should be performed with the MRI system that is expected to provide the best image quality. The selection of the MRI system includes multiple considerations such as the effects of field strength and availability of specific sequences, which can reduce metal artifact. Appropriate patient positioning, attention to MRI parameters (including bandwidth, voxel size, and echo), and appropriate selection of sequences (those with less metal artifact and advanced metal reduction sequences) are critical to improve image quality. Patients with implants can be successfully imaged with MRI with appropriate planning and understanding of how to minimize artifacts. This improves image quality and the diagnostic confidence of the radiologist. ©RSNA, 2021.

Exploring the use of rating scales with entrustment anchors in workplace-based assessment.

PURPOSE: Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. METHOD: A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes. RESULTS: Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not. CONCLUSIONS: By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.

The Ottawa Resident Observation Form for Nurses (O-RON): Assessment of Resident Performance through the Eyes of the Nurses.

OBJECTIVE: Most work-place based assessment relies on physician supervisors making observations of residents. Many areas of performance are not directly observed by physicians but rather by other healthcare professionals, most often nurses. Assessment of resident performance by nurses is captured with multi-source feedback tools. However, these tools combine the assessments of nurses with other healthcare professionals and so their perspective can be lost. A novel tool was developed and implemented to assess resident performance on a hospital ward from the perspective of the nurses. DESIGN: Through a nominal group technique, nurses identified dimensions of performance that are reflective of high-quality physician performance on a hospital ward. These were included as items in the Ottawa Resident Observation Form for Nurses (O-RON). The O-RON was voluntarily completed during an 11-month period. Validity evidence related to quantitative and qualitative data was collected. SETTING: The Orthopedic Surgery Residency Program at the University of Ottawa. PARTICIPANTS: 49 nurses on the Orthopedic Surgery wards at The Ottawa Hospital (tertiary care). RESULTS: The O-RON has 15 items rated on a 3-point frequency scale, one global judgment yes/no question regarding whether they would want the resident on their team and a space for comments. 1079 O-RONs were completed on 38 residents. There was an association between the response to the global judgment question and the frequency of concerns (p < 0.01). With 8 forms per resident, the reliability of the O-RON was 0.80. Open-ended responses referred to aspects of interpersonal skills, responsiveness, dependability, communication skills, and knowledge. CONCLUSIONS: The O-RON demonstrates promise as a work-place based assessment tool to provide residents and training programs with feedback on aspects of their performance on a hospital ward through the eyes of the nurses. It appears to be easy to use, has solid evidence for validity and can provide reliable data with a small number of completed forms.

Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument.

Competency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace "units of assessment". Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents' performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents' performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident's performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

Comparing the Ottawa Emergency Department Shift Observation Tool (O-EDShOT) to the traditional daily encounter card: measuring the quality of documented assessments.

OBJECTIVES: The Ottawa Emergency Department Shift Observation Tool (O-EDShOT) is a workplace-based assessment designed to assess a trainee's performance across an entire shift. It was developed in response to validity concerns with traditional end-of-shift workplace-based assessments, such as the daily encounter card. The O-EDShOT previously demonstrated strong psychometric characteristics; however, it remains unknown whether the O-EDShOT facilitates measurable improvements in the quality of documented assessments compared to daily encounter cards. METHODS: Three randomly selected daily encounter cards and three O-EDShOTs completed by 24 faculty were scored by two raters using the Completed Clinical Evaluation Report Rating (CCERR), a previously published 9-item quantitative measure of the quality of a completed workplace-based assessment. Automated-CCERR (A-CCERR) scores, which do not require raters, were also calculated. Paired sample t tests were conducted to compare the quality of assessments between O-EDShOTs and DECs as measured by the CCERR and A-CCERR. RESULTS: CCERR scores were significantly higher for O-EDShOTs (mean(SD) = 25.6(2.6)) compared to daily encounter cards (21.5(3.9); t(23) = 5.2, p < 0.001, d = 1.1). A-CCERR scores were also significantly higher for O-EDShOTs (mean(SD) = 18.5(1.6)) than for daily encounter cards (15.5(1.2); t(24) = 8.4, p < 0.001). CCERR items 1, 4 and 9 were rated significantly higher for O-EDShOTs compared to daily encounter cards. CONCLUSIONS: The O-EDShOT yields higher quality documented assessments when compared to the traditional end-of-shift daily encounter card. Our results provide additional validity evidence for the O-EDShOT as an assessment tool for capturing trainee on-shift performance that can be used as a stimulus for actionable feedback and as a source for high-quality workplace-based assessment data to inform decisions about emergency medicine trainee progress and promotion.

RéSUMé: OBJECTIFS: L'outil d'observation des quarts de travail des services d'urgence d'Ottawa (O-EDShOT) est une évaluation en milieu de travail conçue pour évaluer la performance d'un stagiaire pendant tout un quart de travail. Il a été développé en réponse à des problèmes de validité avec les évaluations traditionnelles en milieu de travail de fin de quart de travail, comme la fiche de rencontre quotidienne (DEC). Le O-EDShOT avait préalablement démontré de fortes caractéristiques psychométriques; cependant, on ignore toujours si l'O-EDShOT facilite des améliorations mesurables de la qualité des évaluations documentées par rapport aux fiches de rencontre quotidiennes. MéTHODES: Trois fiches de rencontre quotidiennes sélectionnées au hasard et trois O-EDShOT complétés par 24 membres du corps professoral ont été marqués par deux évaluateurs à l'aide de Completed Clinical Evaluation Report Rating (CCERR), une mesure quantitative en 9 points publiée précédemment de la qualité d'une évaluation en milieu de travail réalisée. Les scores du CCERR automatisé (A-CCERR), qui ne nécessitent pas d'évaluateur, ont également été calculés. Des tests t d'échantillons appariés ont été effectués pour comparer la qualité des évaluations entre les O-EDShOT et les DEC, telle que mesurée par le CCERR et l'A-CCERR. RéSULTATS: Les scores CCERR étaient significativement plus élevés pour les O-EDShOT (moyenne (ET) = 25,6 (2,6)) par rapport aux fiches de rencontre quotidiennes (21,5 (3,9) ; t (23) = 5,2, p < 0,001, d = 1,1). Les scores A-CCERR étaient également significativement plus élevés pour les O-EDShOT (moyenne (ET) = 18,5 (1,6)) que pour les fiches de rencontre quotidiennes (15,5 (1,2) ; t (24) = 8,4, p < 0,001). Les points 1, 4 et 9 du CCERR ont été notés beaucoup plus haut pour les O-EDShOT que pour les fiches de rencontre quotidiennes. CONCLUSIONS: L'O-EDShOT produit des évaluations documentées de meilleure qualité par rapport à la traditionnelle fiche de rencontre quotidienne de fin de quart de travail. Nos résultats fournissent des preuves de validité supplémentaires pour l'O-EDShOT en tant qu'outil d'évaluation pour saisir les performances des stagiaires en poste qui peuvent être utilisés comme stimulus pour une rétroaction exploitable, et comme source de données d'évaluation de haute qualité en milieu de travail pour notifier les décisions sur la progression et promotion des stagiaires en médecine d'urgence.

Maintaining stable transtibial amputee gait on level and simulated uneven conditions in a virtual environment.

PURPOSE: This research compares gait strategies to maintain stable gait over a variety of non-level walking conditions for individuals with a transtibial amputation and able-bodied individuals. METHODS: Twelve people with unilateral transtibial amputation and twelve able-bodied individuals walked on a self-paced treadmill in a park-like virtual environment with level and continuous perturbation conditions. Walking stability was quantified by margin-of-stability, step parameters (walking speed, temporal and spatial parameters, and foot clearance), and gait variability (standard deviations for margin-of-stability, step parameters, and root-mean-square of trunk acceleration). RESULTS AND CONCLUSIONS: For non-level conditions, able-bodied and transtibial groups had greater root-mean-square of trunk acceleration and walked with a cautious and variable step strategy by changing speed, step width, foot clearance, margin-of-stability, and increasing step variability. Overall, able-bodied and transtibial amputee participants adopted similar strategies to maintain stable gait over non-level conditions, but the amputee group was more variable than the able-bodied group. These results demonstrated the importance of measuring gait variability, including trunk acceleration and step variability measures, when quantitatively assessing mobility for individuals with a transtibial amputation. Implications for rehabilitation Able-bodied and transtibial amputee groups adapted gait biomechanics for simulated uneven conditions. Adaptations for non-level conditions included increasing step width, margin-of stability, minimum foot clearance, and varying speed. Gait was also more variable for non-level conditions, with greater variability for transtibial amputee participants compared to able-bodied participants. These results highlight the importance of measuring variability when performing comprehensive walking assessment, particularly for active individuals who achieve maximal performance on standard assessments yet report functional limitations in daily living.

Identifying priorities and developing strategies for building capacity in amputation research in Canada.

BACKGROUND: Compared to other patient population groups, the field of amputation research in Canada lacks cohesion largely due to limited funding sources, lack of connection among research scientists, and loose ties among geographically dispersed healthcare centres, research institutes and advocacy groups. As a result, advances in clinical care are hampered and ultimately negatively influence outcomes of persons living with limb loss. OBJECTIVE: To stimulate a national strategy on advancing amputation research in Canada, a consensus-workshop was organized with an expert panel of stakeholders to identify key research priorities and potential strategies to build researcher and funding capacity in the field. METHODS: A modified Delphi approach was used to gain consensus on identifying and selecting an initial set of priorities for building research capacity in the field of amputation. This included an anonymous pre-meeting survey (N = 31 respondents) followed by an in-person consensus-workshop meeting that hosted 38 stakeholders (researchers, physiatrists, surgeons, prosthetists, occupational and physical therapists, community advocates, and people with limb loss). RESULTS: The top three identified research priorities were: (1) developing a national dataset; (2) obtaining health economic data to illustrate the burden of amputation to the healthcare system and to patients; and (3) improving strategies related to outcome measurement in patients with limb loss (e.g. identifying, validating, and/or developing outcome measures). Strategies for moving these priorities into action were also developed. CONCLUSIONS: The consensus-workshop provided an initial roadmap for limb loss research in Canada, and the event served as an important catalyst for stakeholders to initiate collaborations for moving identified priorities into action. Given the increasing number of people undergoing an amputation, there needs to be a stronger Canadian collaborative approach to generate the necessary research to enhance evidence-based clinical care and policy decision-making.IMPLICATIONS FOR REHABILITATIONLimb loss is a growing concern across North America, with lower-extremity amputations occurring due to complications arising from diabetes being a major cause.To advance knowledge about limb loss and to improve clinical care for this population, stronger connections are needed across the continuum of care (acute, rehabilitation, community) and across sectors (clinical, advocacy, industry and research).There are new surgical techniques, technologies, and rehabilitation approaches being explored to improve the health, mobility and community participation of people with limb loss, but further research evidence is needed to demonstrate efficacy and to better integrate them into standard clinical care.

Attaining full professor: Women's and men's experiences in medical education.

INTRODUCTION: The underrepresentation of women among senior faculty members in medical education is a longstanding problem. The purpose of this international qualitative investigation was to explore women and men's experiences of attaining full professorship and to investigate why women remain underrepresented among the senior faculty ranks. METHODS: Conducted within a social constructionist orientation, our qualitative study employed narrative analysis. Two female and two male participants working in medical education were recruited from five nations: Australia, Canada, the Netherlands, United Kingdom and United States. All participants held an MD or PhD. During telephone interviews, participants narrated the story of their careers. The five faculty members on the research team were also interviewed. Their narratives were included in analysis, rendering their experiences equal to those of the participants. RESULTS: A total of 24 full professors working in medical education were interviewed (n = 15 females and n = 9 males). While some aspects were present across all narratives (ie personal events, career milestones and facilitating and/or impeding factors), participants' experience of those aspects differed by gender. Men did not narrate fatherhood as a role navigated professionally, but women narrated motherhood as intimately connected to their professional roles. Both men and women narrated career success in terms of hard work and overcoming obstacles; however, male participants described promotion as inevitable, whereas women narrated promotion as a tenuous navigation of social structures towards uncertain outcomes. Female and male participants encountered facilitators and inhibitors throughout their careers but described acting on those experiences differently within the cultural contexts they faced. DISCUSSION: Our data suggest that female and male participants had different experiences of the work involved in achieving full professor status. Understanding these gendered experiences and their impact on career progression is an important advancement for better understanding what leads to the underrepresentation of women among senior faculty members in medical education.

The Ottawa Emergency Department Shift Observation Tool (O-EDShOT): A New Tool for Assessing Resident Competence in the Emergency Department.

OBJECTIVES: The outcome of emergency medicine (EM) training is to produce physicians who can competently run an emergency department (ED) shift. However, there are few tools with supporting validity evidence specifically designed to assess multiple key competencies across an entire shift. The investigators developed and gathered validity evidence for a novel entrustment-based tool to assess a resident's ability to safely run an ED shift. METHODS: Through a nominal group technique, local and national stakeholders identified dimensions of performance that are reflective of a competent ED physician and are required to safely manage an ED shift. These were included as items in the Ottawa Emergency Department Shift Observation Tool (O-EDShOT), and each item was scored using an entrustment-based rating scale. The tool was implemented in 2018 at the University of Ottawa Department of Emergency Medicine, and quantitative data and qualitative feedback were collected over 6 months. RESULTS: A total of 1,141 forms were completed by 78 physicians for 45 residents. An analysis of variance demonstrated an effect of training level with statistically significant increases in mean O-EDShOT scores with each subsequent postgraduate year (p < 0.001). Scores did not vary by ED treatment area. Residents rated as able to safely run the shift had significantly higher mean ± SD scores (4.8 ± 0.3) than those rated as not able (3.8 ± 0.6; p < 0.001). Faculty and residents reported that the tool was feasible to use and facilitated actionable feedback aimed at progression toward independent practice. CONCLUSIONS: The O-EDShOT successfully discriminated between trainees of different levels regardless of ED treatment area. Multiple sources of validity evidence support the O-EDShOT as a tool to assess a resident's ability to safely run an ED shift. It can serve as a stimulus for daily observation and feedback making it practical to use within an EM residency program.

Relationship between ratings of performance in the simulated and workplace environments among emergency medicine residents.

OBJECTIVES: The Emergency Medicine (EM) Specialty Committee of the Royal College of Physicians and Surgeons of Canada (RCPSC) specifies that resuscitation entrustable professional activities (EPAs) can be assessed in the workplace and simulated environments. However, limited validity evidence for these assessments in either setting exists. We sought to determine if EPA ratings improve over time and whether an association exists between ratings in the workplace v. simulation environment. METHODS: All Foundations EPA1 (F1) assessments were collected for first-year residents (n = 9) in our program during the 2018-2019 academic year. This EPA focuses on initiating and assisting in the resuscitation of critically ill patients. EPA ratings obtained in the workplace and simulation environments were compared using Lin's concordance correlation coefficient (CCC). To determine whether ratings in the two environments differed as residents progressed through training, a within-subjects analysis of variance was conducted with training environment and month as independent variables. RESULTS: We collected 104 workplace and 36 simulation assessments. No correlation was observed between mean EPA ratings in the two environments (CCC(8) = -0.01; p = 0.93). Ratings in both settings improved significantly over time (F(2,16) = 18.8; p < 0.001; η2 = 0.70), from 2.9 ± 1.2 in months 1-4 to 3.5 ± 0.2 in months 9-12. Workplace ratings (3.4 ± 0.1) were consistently higher than simulation ratings (2.9 ± 0.2) (F(2,16) = 7.2; p = 0.028; η2 = 0.47). CONCLUSIONS: No correlation was observed between EPA F1 ratings in the workplace v. simulation environments. Further studies are needed to clarify the conflicting results of our study with others and build an evidence base for the validity of EPA assessments in simulated and workplace environments.

Validity of the Comprehensive High-Level Activity Mobility Predictor in a heterogeneous population with lower extremity amputations.

BACKGROUND AND OBJECTIVE: Lower extremity amputee outcome measures assess basic mobility. The Comprehensive High-Level Activity Mobility Predictor was developed to assess high-level mobility. Validity evidence was collected in military men with traumatic lower extremity amputations. This study examines its validity in a broader population. STUDY DESIGN: Cross-sectional. METHODS: Forty-five lower extremity amputees (Medicare Functional Classification Level K3 or K4) completed the 2-min walk test, Amputee Mobility Predictor with Prosthesis, and Comprehensive High-Level Activity Mobility Predictor. RESULTS: The Comprehensive High-Level Activity Mobility Predictor correlated with the Amputee Mobility Predictor with Prosthesis (r = 0.77, p < 0.01) and the 2-min walk test (r = 0.65, p < 0.01). The Comprehensive High-Level Activity Mobility Predictor differentiated between K-levels, age groups, etiology of amputation, and amputation level (p < 0.005). No ceiling effect was observed (range: 2.5-29/40). CONCLUSION: This study provides convergent and discriminative validity evidence for Comprehensive High-Level Activity Mobility Predictor use in a more heterogeneous population than previously published, suggesting that clinicians should feel confident to use it as an outcome measure for individuals with amputations who are capable of more than level-ground walking. CLINICAL RELEVANCE: Clinically, the Comprehensive High-Level Activity Mobility Predictor has validity evidence for use in a more heterogeneous population than originally demonstrated, including civilians, women, people over age 40 years, and non-traumatic etiologies. The Comprehensive High-Level Activity Mobility Predictor may be more useful than standard outcome measures for high-level mobility.