Supervisor-trainee continuity and the quality of work-based assessments.

CONTEXT: Work-based assessments (WBAs) represent an increasingly important means of reporting expert judgements of trainee competence in clinical practice. However, the quality of WBAs completed by clinical supervisors is of concern. The episodic and fragmented interaction that often occurs between supervisors and trainees has been proposed as a barrier to the completion of high-quality WBAs. OBJECTIVES: The primary purpose of this study was to determine the effect of supervisor-trainee continuity on the quality of assessments documented on daily encounter cards (DECs), a common form of WBA. The relationship between trainee performance and DEC quality was also examined. METHODS: Daily encounter cards representing three differing degrees of supervisor-trainee continuity (low, intermediate, high) were scored by two raters using the Completed Clinical Evaluation Report Rating (CCERR), a previously published nine-item quantitative measure of DEC quality. An analysis of variance (anova) was performed to compare mean CCERR scores among the three groups. Linear regression analysis was conducted to examine the relationship between resident performance and DEC quality. RESULTS: Differences in mean CCERR scores were observed between the three continuity groups (p = 0.02); however, the magnitude of the absolute differences was small (partial eta-squared = 0.03) and not educationally meaningful. Linear regression analysis demonstrated a significant inverse relationship between resident performance and CCERR score (p < 0.001, r2  = 0.18). This inverse relationship was observed in both groups representing on-service residents (p = 0.001, r2  = 0.25; p = 0.04, r2  = 0.19), but not in the Off-service group (p = 0.62, r2  = 0.05). CONCLUSIONS: Supervisor-trainee continuity did not have an educationally meaningful influence on the quality of assessments documented on DECs. However, resident performance was found to affect assessor behaviours in the On-service group, whereas DEC quality remained poor regardless of performance in the Off-service group. The findings suggest that greater attention should be given to determining ways of improving the quality of assessments reported for off-service residents, as well as for those residents demonstrating appropriate clinical competence progression.

Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment.

UNLABELLED: CONSTRUCT: The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) is a 9-item surgical evaluation tool designed to assess technical competence in surgical trainees using behavioral anchors. BACKGROUND: The initial development of the O-SCORE produced evidence for valid results. Further work is required to determine if the use of a single surgeon or an unblinded rater introduces bias. In addition, the relationship of the O-SCORE to other currently used technical assessment tools should be explored to provide validity evidence related to the relationship to other measures. We have designed this project to provide continued validity evidence for the O-SCORE related to these two issues. APPROACH: Nineteen residents and 2 staff Orthopedic Surgeons from the University of Ottawa volunteered to participate in a 2-part OSCE style station. Participants completed a written questionnaire followed by a videotaped 10-minute simulated open reduction and internal fixation of a midshaft radius fracture. Videos were rated individually by 2 blinded staff orthopedic surgeons using an Objective Structured Assessment of Technical Skills (OSATS) global rating scale, an OSATS checklist, and the O-SCORE in random order. RESULTS: O-SCORE results appeared sensitive to surgical training level even when raters were blinded. In addition, strong agreement between two independent observers using the O-SCORE suggests that the measure captures a performance easily recognized by surgical observers. Ratings on the O-SCORE also were strongly associated with global ratings on the currently most validated technical evaluation tool (OSATS). Collectively, these results suggest that the O-SCORE generates accurate, reproducible, and meaningful results when used in a randomized and blinded fashion, providing continued validity evidence for using this tool to evaluate surgical trainee competence. CONCLUSIONS: The O-SCORE was able to differentiate surgical trainee level using blinded raters providing further evidence of validity for the O-SCORE. There was strong agreement between two independent observers using the O-SCORE. Ratings on the O-SCORE also demonstrated equivalence to scores on the most validated technical evaluation tool (OSATS). These results suggest that the O-SCORE demonstrates accurate and reproducible results when used in a randomized and blinded fashion providing continued validity evidence for this tool in the evaluation of surgical competence in the trainees.

Going Paperless? Issues in Converting a Surgical Assessment Tool to an Electronic Version.

UNLABELLED: CONSTRUCT: The competence of a trainee to perform a surgical procedure was assessed using an electronic tool. BACKGROUND: "Going paperless" in healthcare has received significant attention over the past decades given the numerous potential benefits of converting to electronic health records. Not surprisingly, medical educators have also considered the potential benefits of electronic assessments for their trainees. What literature exists on the transition from paper-based to electronic-based assessments suggests a positive outcome. In contrast, work done examining the transition to and implementation of electronic health records has noted that hospitals who have implemented these systems have not gone paperless despite the benefits of doing so. APPROACH: This study sought to transition a paper-based assessment tool, the Ottawa Surgical Competency Operating Room Evaluation (which has strong evidence for validity) to an electronic version, in three surgical specialties (Orthopedic Surgery, Urology, General Surgery). However, as the project progressed, it became necessary to change the focus of the study to explore the issues of transitioning to a paperless assessment tool as we identified an extremely low participation rate. RESULTS: Over the first 3 months 440 assessment cases were logged. However, only a small portion of these cases were assessed using the electronic tool (Orthopedic Surgery = 16%, Urology = 5%, General Surgery = 0%). Participants identified several barriers in using the electronic assessment tool such as increased time compared to the paper version and technological issues related to the log-in procedure. CONCLUSIONS: Essentially, users want the tool to be as convenient as paper. This is consistent with research on electronic health records implementation but different from previous work in medical education. Thus, we believe our study highlights an important finding. Transitioning from a paper-based assessment tool to an electronic one is not necessarily a neutral process. Consideration of potential barriers and finding solutions to these barriers will be necessary in order to realize the many benefits of electronic assessments.

In-training evaluations: developing an automated screening tool to measure report quality.

OBJECTIVES: In-training evaluation (ITE) is used to assess resident competencies in clinical settings. This assessment is documented on an evaluation report (In-Training Evaluation Report [ITER]). Unfortunately, the quality of these reports can be questionable. Therefore, training programmes to improve report quality are common. The Completed Clinical Evaluation Report Rating (CCERR) was developed to assess completed report quality and has been shown to do so in a reliable manner, thus enabling the evaluation of these programmes. The CCERR is a resource-intensive instrument, which may limit its use. The purpose of this study was to create a screening measure (Proxy-CCERR) that can predict the CCERR outcome in a less resource-intensive manner. METHODS: Using multiple regression, the authors analysed a dataset of 269 ITERs to create a model that can predict the associated CCERR scores. The resulting predictive model was tested on the CCERR scores for an additional sample of 300 ITERs. RESULTS: The quality of an ITER, as measured by the CCERR, can be predicted using a model involving only three variables (R(2)  = 0.61). The predictive variables included the total number of words in the comments, the variability of the ratings and the proportion of comment boxes completed on the form. CONCLUSIONS: It is possible to model CCERR scores in a highly predictive manner. The predictive variables can be easily extracted in an automated process. Because this model is less resource-intensive than the CCERR, it makes it possible to provide feedback from ITER training programmes to large groups of supervisors and institutions, and even to create automated feedback systems using Proxy-CCERR scores.

Quality evaluation reports: Can a faculty development program make a difference?

BACKGROUND: The quality of medical student and resident clinical evaluation reports submitted by rotation supervisors is a concern. The effectiveness of faculty development (FD) interventions in changing report quality is uncertain. AIMS: This study assessed whether faculty could be trained to complete higher quality reports. METHOD: A 3-h interactive program designed to improve evaluation report quality, previously developed and tested locally, was offered at three different Canadian medical schools. To assess for a change in report quality, three reports completed by each supervisor prior to the workshop and all reports completed for 6 months following the workshop were evaluated by three blinded, independent raters using the Completed Clinical Evaluation Report Rating (CCERR): a validated scale that assesses report quality. RESULTS: A total of 22 supervisors from multiple specialties participated. The mean CCERR score for reports completed after the workshop was significantly higher (21.74 ± 4.91 versus 18.90 ± 5.00, p = 0.02). CONCLUSIONS: This study demonstrates that this FD workshop had a positive impact upon the quality of the participants' evaluation reports suggesting that faculty have the potential to be trained with regards to trainee assessment. This adds to the literature which suggests that FD is an important component in improving assessment quality.

Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument.

Competency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace "units of assessment". Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents' performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents' performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident's performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

Identifying priorities and developing strategies for building capacity in amputation research in Canada.

BACKGROUND: Compared to other patient population groups, the field of amputation research in Canada lacks cohesion largely due to limited funding sources, lack of connection among research scientists, and loose ties among geographically dispersed healthcare centres, research institutes and advocacy groups. As a result, advances in clinical care are hampered and ultimately negatively influence outcomes of persons living with limb loss. OBJECTIVE: To stimulate a national strategy on advancing amputation research in Canada, a consensus-workshop was organized with an expert panel of stakeholders to identify key research priorities and potential strategies to build researcher and funding capacity in the field. METHODS: A modified Delphi approach was used to gain consensus on identifying and selecting an initial set of priorities for building research capacity in the field of amputation. This included an anonymous pre-meeting survey (N = 31 respondents) followed by an in-person consensus-workshop meeting that hosted 38 stakeholders (researchers, physiatrists, surgeons, prosthetists, occupational and physical therapists, community advocates, and people with limb loss). RESULTS: The top three identified research priorities were: (1) developing a national dataset; (2) obtaining health economic data to illustrate the burden of amputation to the healthcare system and to patients; and (3) improving strategies related to outcome measurement in patients with limb loss (e.g. identifying, validating, and/or developing outcome measures). Strategies for moving these priorities into action were also developed. CONCLUSIONS: The consensus-workshop provided an initial roadmap for limb loss research in Canada, and the event served as an important catalyst for stakeholders to initiate collaborations for moving identified priorities into action. Given the increasing number of people undergoing an amputation, there needs to be a stronger Canadian collaborative approach to generate the necessary research to enhance evidence-based clinical care and policy decision-making.IMPLICATIONS FOR REHABILITATIONLimb loss is a growing concern across North America, with lower-extremity amputations occurring due to complications arising from diabetes being a major cause.To advance knowledge about limb loss and to improve clinical care for this population, stronger connections are needed across the continuum of care (acute, rehabilitation, community) and across sectors (clinical, advocacy, industry and research).There are new surgical techniques, technologies, and rehabilitation approaches being explored to improve the health, mobility and community participation of people with limb loss, but further research evidence is needed to demonstrate efficacy and to better integrate them into standard clinical care.

Maintaining stable transfemoral amputee gait on level, sloped and simulated uneven conditions in a virtual environment.

PURPOSE: Describe and quantify how people with transfemoral amputations (TFA) maintain stable gait over a variety of surfaces; including, downhill and uphill, top and bottom-cross-slopes, medial-lateral translations, rolling hills and simulated rocky surfaces. METHODS: Ten TFA and ten matched people without amputations (NA) walked in a virtual environment with level, sloped and simulated uneven surfaces on a self-paced treadmill. Stability was quantified using medial-lateral margin of stability (ML-MoS), step parameters, and gait variability (standard deviations for speed, temporal-spatial parameters, foot clearance and root-mean-square of medial-lateral trunk acceleration). RESULTS AND CONCLUSIONS: TFA and NA adapted to non-level conditions by changing their walking speed, step width, and foot clearance. Variability for most parameters increased across conditions, compared to level. TFA walked slower than NA with shorter, wider and longer duration steps (most differences related to speed). ML-MoS did not change compared to level; however, ML-MoS was greater on the prosthetic side than both intact side and NA limbs. Foot clearance and root-mean-square of medial-lateral trunk acceleration were greater on the prosthetic side than the intact side and NA limbs. This research provides a comprehensive analysis of the different adaptations made by people without amputations compared to people with transfemoral amputations over non-level conditions and establishes significant differences between slopes and simulated uneven surfaces for TFA. Implications for Rehabilitation Transfemoral amputation and no amputation groups adapted walking biomechanics when traversing non-level surfaces. Greatest temporal-spatial gait adaptations were walking speed, step width and foot clearance. Gait parameter variability typically increased from the level condition in both groups. Transfemoral amputation group walked slower than no amputation group with shorter, wider steps and longer duration steps. This was related to speed. Transfemoral amputation group had more trunk motion variability on the prosthetic side than no amputation group; could be related to prosthetic fit.

Assessing the quality of supervisors' completed clinical evaluation reports.

CONTEXT: Although concern has been raised about the value of clinical evaluation reports for discriminating among trainees, there have been few efforts to formalise the dimensions and qualities that distinguish effective versus less useful styles of form completion. METHODS: Using brainstorming and a modified Delphi technique, a focus group determined the key features of high-quality completed evaluation reports. These features were used to create a rating scale to evaluate the quality of completed reports. The scale was pilot-tested locally; the results were psychometrically analysed and used to modify the scale. The scale was then tested on a national level. Psychometric analysis and final modification of the scale were completed. RESULTS: Sixteen features of high-quality reports were identified and used to develop a rating scale: the Completed Clinical Evaluation Report Rating (CCERR). The reliability of the scale after a national field test with 55 raters assessing 18 in-training evaluation reports (ITERs) was 0.82. Further revisions were made; the final version of the CCERR contains nine items rated on a 5-point scale. With this version, the mean ratings of three groups of 'gold-standard' ITERs (previously judged to be of high, average and poor quality) differed significantly (P < 0.05). DISCUSSION: The CCERR is a validated scale that can be used to help train supervisors to complete and assess the quality of evaluation reports.

Gait differences between K3 and K4 persons with transfemoral amputation across level and non-level walking conditions.

BACKGROUND:: A transfemoral amputee's functional level can be classified from K-level 0 (lowest) to K-level 4 (highest). Knowledge of the biomechanical differences between K3 and K4 transfemoral amputation could help inform clinical professionals and researchers in amputee care and gait assessment. OBJECTIVES:: Explore gait differences between K3- and K4-level transfemoral amputation across different surface conditions. STUDY DESIGN:: Cross-sectional study. METHODS:: Four K3 and six K4 transfemoral amputation and 10 matched able-bodied individuals walked in a virtual environment with simulated level and non-level surfaces on a self-paced treadmill. Stability measures included medial-lateral margin of stability, step parameters, and gait variability (standard deviations for speed, temporal-spatial parameters, root-mean-square of medial-lateral trunk acceleration). RESULTS:: K3 walked slower than K4 with wider steps, greater root-mean-square of medial-lateral trunk acceleration, and greater medial-lateral margin of stability standard deviations, indicating their stability was further challenged. K3 participants had greater asymmetry in double support time and trunk acceleration root-mean-square in the medial-lateral direction, but similar asymmetry overall. K3 participants had larger differences from AB and in more parameters than K4, although K4 differed from AB in trunk acceleration root-mean-square in the medial-lateral direction, walking speed, and double support time standard deviations. CONCLUSION:: The findings improve our understanding of K3 and K4 transfemoral amputation gait on slopes and simulated uneven surfaces. CLINICAL RELEVANCE: High performing and community ambulatory transfemoral amputees cannot match the ambulatory abilities of ablebodied individuals. Understanding gait differences between these groups under conditions that challenge balance is required to develop rehabilitation protocols and prosthetic componentry targeted at improving transfemoral amputee gait and overall mobility in their chosen environment.

Feedback to Supervisors: Is Anonymity Really So Important?

PURPOSE: Research demonstrates that physicians benefit from regular feedback on their clinical supervision from their trainees. Several features of effective feedback are enabled by nonanonymous processes (i.e., open feedback). However, most resident-to-faculty feedback processes are anonymous given concerns of power differentials and possible reprisals. This exploratory study investigated resident experiences of giving faculty open feedback, advantages, and disadvantages. METHOD: Between January and August 2014, nine graduates of a Canadian Physiatry residency program that uses open resident-to-faculty feedback participated in semistructured interviews in which they described their experiences of this system. Three members of the research team analyzed transcripts for emergent themes using conventional content analysis. In June 2014, semistructured group interviews were held with six residents who were actively enrolled in the program as a member-checking activity. Themes were refined on the basis of these data. RESULTS: Advantages of the open feedback system included giving timely feedback that was acted upon (thus enhancing residents' educational experiences), and improved ability to receive feedback (thanks to observing modeled behavior). Although some disadvantages were noted, they were often speculative (e.g., "I think others might have felt …") and were described as outweighed by advantages. Participants emphasized the program's "feedback culture" as an open feedback enabler. CONCLUSIONS: The relationship between the feedback giver and recipient has been described as influencing the uptake of feedback. Findings suggest that nonanonymous practices can enable a positive relationship in resident-to-faculty feedback. The benefits of an open system for resident-to-faculty feedback can be reaped if a "feedback culture" exists.

Center of pressure and total force analyses for amputees walking with a backpack load over four surfaces.

Understanding how load carriage affects walking is important for people with a lower extremity amputation who may use different strategies to accommodate to the additional weight. Nine unilateral traumatic transtibial amputees (K4-level) walked over four surfaces (level-ground, uneven ground, incline, decline) with and without a 24.5 kg backpack. Center of pressure (COP) and total force were analyzed from F-Scan insole pressure sensor data. COP parameters were greater on the intact limb than on the prosthetic limb, which was likely a compensation for the loss of ankle control. Double support time (DST) was greater when walking with a backpack. Although longer DST is often considered a strategy to enhance stability and/or reduce loading forces, changes in DST were only moderately correlated with changes in peak force. High functioning transtibial amputees were able to accommodate to a standard backpack load and to maintain COP progression, even when walking over different surfaces.

Failure to fail: the perspectives of clinical supervisors.

BACKGROUND: Clinical supervisors often do not fail students and residents even though they have judged their performance to be unsatisfactory. This study explored the factors identified by supervisors that affect their willingness to report poor clinical performance when completing In-Training Evaluation Reports (ITERs). METHOD: Semistructured interviews with 21 clinical supervisors at the University of Ottawa were conducted and qualitatively analyzed. RESULTS: Participants identified four major areas of the evaluation process that act as barriers to reporting a trainee who has performed poorly: (1) lack of documentation, (2) lack of knowledge of what to specifically document, (3) anticipating an appeal process and (4) lack of remediation options. CONCLUSIONS: The study provides insight as to why supervisors fail to fail the poorly performing student and resident. It also offers suggestions of how to support supervisors, increasing the likelihood that they will provide a valid ITER when faced with an underachieving trainee.

Changes to transtibial amputee gait with a weighted backpack on multiple surfaces.

BACKGROUND: Modern prosthetic technology and rehabilitation practices have enabled people with lower extremity amputations to participate in almost all occupations and physical activities. Carrying backpack loads can be an essential component for many of these jobs and activities; however, amputee gait with backpack loads is poorly understood. This knowledge gap must be addressed in order to further improve an individual's quality of living through changes in rehabilitation programs and prosthesis development. METHODS: Ten male, unilateral, K4-level (ability or potential for prosthetic ambulation that exceeds basic ambulation skills, exhibiting high impact, stress, or energy levels), transtibial amputees completed ten walking trials at a self-selected pace on simulated uneven ground, ramp ascent, and ramp descent. Five trials were with a 24.5 kg backpack load and five trials without. Temporal-spatial parameters and kinematic peak values for the ankle, knee, hip, pelvis, and trunk were collected and analyzed for differences between backpack conditions. FINDINGS: Each surface had novel findings not found on the other surfaces. However differences in temporal-spatial parameters were congruent with the literature on able bodied individuals. Pelvis and trunk angular velocities decreased with the backpack. Hip flexion on both limbs increased during weight acceptance while wearing the backpack, a common adaptation seen in able-bodied individuals on level ground. INTERPRETATION: A 24.5 kg backpack load can be accommodated by transtibial amputees at the K4 functional level. Future studies on load carriage and gait training programs should include incline and descent due to the increased difficulty. Rehabilitation programs should verify hip and knee flexor strength and work to reduce intact limb reliance.

Fixed and self-paced treadmill walking for able-bodied and transtibial amputees in a multi-terrain virtual environment.

A self-paced treadmill automatically adjusts speed in real-time to match the user's walking speed, potentially enabling more natural gait than fixed-speed treadmills. This research examined walking speed changes for able-bodied and transtibial amputee populations on a self-paced treadmill in a multi-terrain virtual environment and examined gait differences between fixed and self-paced treadmill speed conditions. Twelve able-bodied (AB) individuals and 12 individuals with unilateral transtibial amputation (TT) walked in a park-like virtual environment with level, slopes, and simulated uneven terrain scenarios. Temporal-spatial and range-of-motion parameters were analyzed. Within the self-paced condition, all participants significantly varied walking speed (p<0.001) across different walking activities. Compared to level walking, participants reduced speed for uphill and hilly activities (p<0.001). TT also reduced speed downhill (p<0.001). Generally, differences in temporal-spatial and range-of-motion parameters between fixed and self-paced speed conditions were no longer significantly different with a speed covariate. However, for uphill walking, both groups decreased stride length during self-paced trials, and increased stride length during fixed-speed trials to maintain the constant speed (p<0.01). The results from this study demonstrated self-paced treadmill mode is important for virtual reality systems with multiple movement scenarios in order to elicit more natural gait across various terrain. Fixed-speed treadmills may induce gait compensations to maintain the fixed speed.

Changes to level ground transtibial amputee gait with a weighted backpack.

BACKGROUND: Many occupations and hobbies require the use of a weighted pack. To date there has been limited backpack gait studies performed on the amputee population. It is important that we address this knowledge gap in order to further improve individual's quality of living through changes in rehabilitation, and prosthesis development. METHODS: The study population was ten male, unilateral, traumatic, K4-level (ability for prosthetic ambulation with high impact, stress, or energy levels), transtibial amputees. Ten walking trials were collected on level ground; five with a 24.5 kg backpack and five without a backpack. Temporal-spatial parameters and kinematic and kinetic peak values for the ankle, knee, hip, pelvis, and trunk were collected and analyzed for significant differences. FINDINGS: Temporal-spatial parameters incurred changes that were congruent with the literature on able bodied individuals. Pelvis speeds and range of motion decreased with the pack. Knee flexion during weight acceptance increased, and was supported on the intact limb by increased eccentric knee power during weight acceptance. Hip flexion on both limbs also increased during weight acceptance while wearing the backpack. INTERPRETATION: The backpack load can be accommodated by people at a K4 functional level for level ground walking. At the prosthetic limb, greater deformation was found at the foot-ankle and further increases in pack weight and higher impact tasks (i.e., jogging) could lead to decreased performance for some prosthetic feet. Gait training programs should focus on removing any gait asymmetries and increasing the strength of both the hip and knee flexors.

Quality in-training evaluation reports--does feedback drive faculty performance?

PURPOSE: Clinical faculty often complete in-training evaluation reports (ITERs) poorly. Faculty development (FD) strategies should address this problem. An FD workshop was shown to improve ITER quality, but few physicians attend traditional FD workshops. To reach more faculty, the authors developed an "at-home" FD program offering participants various types of feedback on their ITER quality based on the workshop content. Program impact is evaluated here. METHOD: Ninety-eight participants from four medical schools, all clinical supervisors, were recruited in 2009-2010; 37 participants completed the study. These were randomized into five groups: a control group and four other groups with different feedback conditions. ITER quality was assessed by two raters using a validated tool: the completed clinical evaluation report rating (CCERR). Participants were given feedback on their ITER quality based on group assignment. Six months later, participants submitted new ITERs. These ITERs were assessed using the CCERR, and feedback was sent to participants on the basis of their group assignment. This process was repeated two more times, ending in 2012. RESULTS: CCERR scores from the participants in all feedback groups were collapsed (n=27) and compared with scores from the control group (n=10). Mean CCERR scores significantly increased over time for the feedback group but not the control group. CONCLUSIONS: The results suggest that faculty are able to improve ITER quality following a minimal "at-home" FD intervention. This also adds to the growing literature that has found success with improving the quality of trainee assessments following rater training.

The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): a tool to assess surgical competence.

PURPOSE: Most assessment of surgical trainees is based on measures of knowledge, with limited evaluation of their competence to actually perform various surgical procedures. In this study, the authors evaluated a tool they designed to assess a trainee's competence to perform an entire surgical procedure independently, regardless of procedure type or postgraduate year (PGY). METHOD: In phase 1, the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) was piloted in the University of Ottawa's Division of Orthopaedic Surgery. In phase 2, the refined 11-item tool (8 items rated on a 5-point competency scale, 1 item assessing procedural competence, 2 feedback items) was used in the Divisions of Orthopaedic Surgery and General Surgery to assess residents' performance on 11 common procedures. Quantitative and qualitative analyses were conducted. RESULTS: In phase 2, 34 orthopaedic and general surgeons assessed the performance of 37 residents in 163 procedures. ANOVA demonstrated an effect of PGY. Post hoc analysis found that total procedure scores for PGYs 1 and 2 were lower than those for PGY 3 (P<.001), and PGY 3 scores were lower than those for PGYs 4 and 5 (P<.02). Analysis of qualitative data indicated that the rating scale was practical and useful for surgeons and residents. CONCLUSIONS: This novel evaluation tool successfully discriminated between junior and senior residents and identified surgical competency across various PGY levels regardless of procedure type. Multiple sources of evidence support the O-SCORE as a valid tool for the assessment of trainee operative competency.

A qualitative study of factors influencing the decision to have an elective amputation.

BACKGROUND: Some patients with a functionally impaired lower limb choose to have an elective amputation, whereas others do not. Functional outcomes do not favor either type of treatment, making this a complex decision. The experiences of patients who have chosen elective amputation were analyzed to identify the key factors in this decision-making process. METHODS: Patients from a tertiary care amputee clinic who had chosen to undergo elective amputation of a functionally impaired lower limb participated in the present study. A qualitative research design involved the use of one-on-one semi-structured interviews, which were audio recorded and transcribed. Narrative analysis was used by three researchers to provide triangulation. Recurrent key themes and patterns were described. Personal factors in the decision-making process were identified. RESULTS: Factors that had the largest impact on the decision-making process were pain, function, and participation. Body image, self identity, and the opinions of others had little influence. Satisfaction with the surgical outcome was related to how closely the result matched the patient's expectations. Patients who were better informed prior to surgery had more realistic expectations about living with an amputation. CONCLUSIONS: The severity of pain and the desire for improved function are strong drivers for patients deciding to undergo elective amputation of a functionally impaired lower extremity. While patients do not want others' opinions, information regarding life with an amputation helps to set realistic expectations regarding outcome.

Ambulation monitoring of transtibial amputation subjects with patient activity monitor versus pedometer.

Our study aimed to compare the accuracy of step count and ambulation distance determined with the Yamax Digi-Walker SW-700 pedometer (DW) and the Ossur patient activity monitor (PAM) in 20 transtibial amputation subjects who were functioning at the K3 Medicare Functional Classification Level. Subjects completed four simulated household tasks in an apartment setup and a gymnasium walking course designed to simulate outdoor walking without the presence of environmental barriers or varied terrain. The mean step count accuracy of the DW and the PAM was equivalent for both the household activity (75.3% vs 70.6%) and the walking course (93.8% vs 94.0%). The mean distance measurement accuracy was better with the DW than with the PAM (household activity: 72.8% vs 0%, walking course: 92.5% vs 86.3%; p < 0.05). With acceptable step count accuracy, both devices are appropriate for assessing relatively continuous ambulation. The DW may be preferred for its more accurate distance measurements. Neither device is ideal for monitoring in-home ambulation.