Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Competency-based resident education-The Canadian perspective.

Canada has been a leader in competency-based medical education for some years. Postgraduate training programs are typically 5 years in duration with opportunities to pursue 2-year subspecialty training after certification in a primary specialty. The introduction of competency-based models in Canada has progressed from a single orthopedic surgery training program at the University of Toronto through the adoption of competency-based medical education in 29 training programs at a single medical school, and the implementation across all 68 disciplines overseen by the Royal College of Physicians and Surgeons of Canada. This article outlines the introduction of competency-based medical education in postgraduate medical education in Canada.

Developing the Blueprint for a General Surgery Technical Skills Certification Examination: A Validation Study.

INTRODUCTION: There is a recognized need to develop high-stakes technical skills assessments for decisions of certification and resident promotion. High-stakes examinations requires a rigorous approach in accruing validity evidence throughout the developmental process. One of the first steps in development is the creation of a blueprint which outlines the potential content of examination. The purpose of this validation study was to develop an examination blueprint for a Canadian General Surgery assessment of technical skill certifying examination. METHODS: A Delphi methodology was used to gain consensus amongst Canadian General Surgery program directors as to the content (tasks or procedures) that could be included in a certifying Canadian General Surgery examination. Consensus was defined a priori as a Cronbach's α ≥ 0.70. All procedures or tasks reaching a positive consensus (defined as ≥80% of program directors rated items as ≥4 on the 5-point Likert scale) were then included in the final examination blueprint. RESULTS: Two Delphi rounds were needed to reach consensus. Of the 17 General Surgery Program directors across the country, 14 (82.4%) and 10 (58.8%) program directors responded to the first and second round, respectively. A total of 59 items and procedures reached positive consensus and were included in the final examination blueprint. CONCLUSIONS: The present study has outlined the development of an examination blueprint for a General Surgery certifying examination using a consensus-based methodology. This validation study will serve as the foundational work from which simulated model will be developed, pilot tested and evaluated.

Overarching challenges to the implementation of competency-based medical education.

Medical education is under increasing pressure to more effectively prepare physicians to meet the needs of patients and populations. With its emphasis on individual, programmatic, and institutional outcomes, competency-based medical education (CBME) has the potential to realign medical education with this societal expectation. Implementing CBME, however, comes with significant challenges. This manuscript describes four overarching challenges that must be confronted by medical educators worldwide in the implementation of CBME: (1) the need to align all regulatory stakeholders in order to facilitate the optimization of training programs and learning environments so that they support competency-based progression; (2) the purposeful integration of efforts to redesign both medical education and the delivery of clinical care; (3) the need to establish expected outcomes for individuals, programs, training institutions, and health care systems so that performance can be measured; and (4) the need to establish a culture of mutual accountability for the achievement of these defined outcomes. In overcoming these challenges, medical educators, leaders, and policy-makers will need to seek collaborative approaches to common problems and to learn from innovators who have already successfully made the transition to CBME.

Practice does not always make perfect: need for selection curricula in modern surgical training.

BACKGROUND: It is hypothesized that not all surgical trainees are able to reach technical competence despite ongoing practice. The objectives of the study were to assess a trainees' ability to reach technical competence by assessing learning patterns of the acquisition of surgical skills. Furthermore, it aims to determine whether individuals' learning patterns were consistent across a range of open and laparoscopic tasks of variable difficulty. METHODS: Sixty-five preclinical medical students participated in a training curriculum with standardized feedback over forty repetitions of the following laparoscopic and open technical tasks: peg transfer (PT), circle cutting (CC), intracorporeal knot tie (IKT), one-handed tie, and simulated laparotomy closure. Data mining techniques were used to analyze the prospectively collected data and stratify the students into four learning clusters. Performance was compared between groups, and learning curve characteristics unique to trainees who have difficulty reaching technical competence were quantified. RESULTS: Top performers (22-35%) and high performers (32-42%) reached proficiency in all tasks. Moderate performers (25-37%) reached proficiency for all open tasks but not all laparoscopic tasks. Low performers (8-15%) failed to reach proficiency in four of five tasks including all laparoscopic tasks (PT 7.8%; CC 9.4%; IKT 15.6%). Participants in lower performance clusters demonstrated sustained performance disadvantage across tasks, with widely variable learning curves and no evidence of progression towards a plateau phase. CONCLUSIONS: Most students reached proficiency across a range of surgical tasks, but low-performing trainees failed to reach competence in laparoscopic tasks. With increasing use of laparoscopy in surgical practice, screening potential candidates to identify the lowest performers may be beneficial.

Optimizing the Selection of General Surgery Residents: A National Consensus.

BACKGROUND: Surgical programs strive to recruit trainees who will graduate as competent surgeons; however, selection processes vary between institutions. The purpose of the present study was to (1) solicit program directors' (PDs) opinions on the proportion of trainees who have difficulty achieving competence and (2) establish consensus on the desired attributes of general surgery (GS) candidates and the technical skills that would be most indicative of future performance. METHODS: Delphi consensus methodology was used. An open-ended questionnaire, followed by a closed-ended questionnaire, formulated as a 5-point Likert scale, was administered. A Cronbach α ≥ 0.8 with 80% of responses in agreement (4-agree and 5-strongly agree) determined the threshold for consensus. RESULTS: The first and second rounds were completed by 14 and 11, of a potential 17, GS PDs, respectively. PDs felt that 5% or less of trainees have difficulty reaching competence in clinical knowledge, 5% to 10% in decision-making, and 5% to 15% in technical skill by the time of completion of training. Consensus was excellent (α = 0.92). The top attributes for success in GS included work ethic and passion for surgery. Technical skills that felt to be most appropriate were open tasks (one-handed tie and subcuticular suture) and laparoscopic tasks (coordination, grasping, and cutting). CONCLUSION: PDs indicate that of the 3 domains, the largest proportion of trainees had difficulty reaching competence in technical skill. Consensus among PDs suggests that top personal attributes include work ethic and passion for surgery. Consensus of technical tasks for inclusion into selection was basic open and laparoscopic skills.

Setting Performance Standards for Technical and Nontechnical Competence in General Surgery.

OBJECTIVES: The objectives of this study were to (1) create a technical and nontechnical performance standard for the laparoscopic cholecystectomy, (2) assess the classification accuracy and (3) credibility of these standards, (4) determine a trainees' ability to meet both standards concurrently, and (5) delineate factors that predict standard acquisition. BACKGROUND: Scores on performance assessments are difficult to interpret in the absence of established standards. METHODS: Trained raters observed General Surgery residents performing laparoscopic cholecystectomies using the Objective Structured Assessment of Technical Skill (OSATS) and the Objective Structured Assessment of Non-Technical Skills (OSANTS) instruments, while as also providing a global competent/noncompetent decision for each performance. The global decision was used to divide the trainees into 2 contrasting groups and the OSATS or OSANTS scores were graphed per group to determine the performance standard. Parametric statistics were used to determine classification accuracy and concurrent standard acquisition, receiver operator characteristic (ROC) curves were used to delineate predictive factors. RESULTS: Thirty-six trainees were observed 101 times. The technical standard was an OSATS of 21.04/35.00 and the nontechnical standard an OSANTS of 22.49/35.00. Applying these standards, competent/noncompetent trainees could be discriminated in 94% of technical and 95% of nontechnical performances (P < 0.001). A 21% discordance between technically and nontechnically competent trainees was identified (P < 0.001). ROC analysis demonstrated case experience and trainee level were both able to predict achieving the standards with an area under the curve (AUC) between 0.83 and 0.96 (P < 0.001). CONCLUSIONS: The present study presents defensible standards for technical and nontechnical performance. Such standards are imperative to implementing summative assessments into surgical training.

Strategies for increasing the feasibility of performance assessments during competency-based education: Subjective and objective evaluations correlate in the operating room.

BACKGROUND: Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. METHODS: Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. RESULTS: Fifteen cases were observed. There was a moderately positive correlation (rs = .618, P = .014) for technical performance and a strong positive correlation (rs = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. CONCLUSIONS: This study provides evidence to help operationalize competency-based assessments.

International assessment practices along the continuum of surgical training.

BACKGROUND: The objectives of this study were to assemble an international perspective on (1) current, and (2) ideal technical performance assessment methods, and (3) barriers to their adoption during: selection, in-training, and certification. METHODS: A questionnaire was distributed to international educational directorates. RESULTS: Eight of 10 jurisdictions responded. Currently, aptitude tests or simulated tasks are used during selection, observational rating scales during training and nothing is used at certification. Ideally, innate ability should be determined during selection, in-training evaluation reports, and global rating scales used during training, whereas global and procedure-specific rating scales used at the time of certification. Barriers include lack of predictive evidence for use in selection, financial limitations during training, and a combination with respect to certification. CONCLUSIONS: Identifying current and ideal evaluation methods will prove beneficial to ensure the best assessments of technical performance are chosen for each training time point.

The Future of General Surgery: Evolving to Meet a Changing Practice.

PURPOSE: Similar to other countries, the practice of General Surgery in Canada has undergone significant evolution over the past 30 years without major changes to the training model. There is growing concern that current General Surgery residency training does not provide the skills required to practice the breadth of General Surgery in all Canadian communities and practice settings. PROCEDURE: Led by a national Task Force on the Future of General Surgery, this project aimed to develop recommendations on the optimal configuration of General Surgery training in Canada. A series of 4 evidence-based sub-studies and a national survey were launched to inform these recommendations. MAIN FINDINGS: Generalized findings from the multiple methods of the project speak to the complexity of the current practice of General Surgery: (1) General surgeons have very different practice patterns depending on the location of practice; (2) General Surgery training offers strong preparation for overall clinical competence; (3) Subspecialized training is a new reality for today's general surgeons; and (4) Generation of the report and recommendations for the future of General Surgery. A total of 4 key recommendations were developed to optimize General Surgery for the 21st century. CONCLUSIONS: This project demonstrated that a high variability of practice dependent on location contrasts with the principles of implementing the same objectives of training for all General Surgery graduates. The overall results of the project have prompted the Royal College to review the training requirements and consider a more "fit for purpose" training scheme, thus ensuring that General Surgery residency training programs would optimally prepare residents for a broad range of practice settings and locations across Canada.

Can We Predict Technical Aptitude?: A Systematic Review.

OBJECTIVE: To identify background characteristics and cognitive tests that may predict surgical trainees' future technical performance, and therefore be used to supplement existing surgical residency selection criteria. BACKGROUND: Assessment of technical skills is not commonly incorporated as part of the selection process for surgical trainees in North America. Emerging evidence, however, suggests that not all trainees are capable of reaching technical competence. Therefore, incorporating technical aptitude into selection processes may prove useful. METHODS: A systematic search was carried out of the MEDLINE, PsycINFO, and Embase online databases to identify all studies that assessed associations between surrogate markers of innate technical abilities in surgical trainees, and whether these abilities correlate with technical performance. The quality of each study was evaluated using the Grading of Recommendations, Assessment, Development, and Evaluation system. RESULTS: A total of 8035 records were identified. After screening by title, abstract, and full text, 52 studies were included. Very few surrogate markers were found to predict technical performance. Significant associations with technical performance were seen for 1 of 23 participant-reported surrogate markers, 2 of 25 visual spatial tests, and 2 of 19 dexterity tests. The assessment of trainee Basic Performance Resources predicted technical performance in 62% and 75% of participants. CONCLUSIONS: To date, no single test has been shown to reliably predict the technical performance of surgical trainees. Strategies that rely on assessing multiple innate abilities, their interaction, and their relationship with technical skill may ultimately be more likely to serve as reliable predictors of future surgical performance.

Assessing Technical Competence in Surgical Trainees: A Systematic Review.

OBJECTIVE: To systematically examine the literature describing the methods by which technical competence is assessed in surgical trainees. BACKGROUND: The last decade has witnessed an evolution away from time-based surgical education. In response, governing bodies worldwide have implemented competency-based education paradigms. The definition of competence, however, remains elusive, and the impact of these education initiatives in terms of assessment methods remains unclear. METHODS: A systematic review examining the methods by which technical competence is assessed was conducted by searching MEDLINE, EMBASE, PsychINFO, and the Cochrane database of systematic reviews. Abstracts of retrieved studies were reviewed and those meeting inclusion criteria were selected for full review. Data were retrieved in a systematic manner, the validity and reliability of the assessment methods was evaluated, and quality was assessed using the Grading of Recommendations Assessment, Development and Evaluation classification. RESULTS: Of the 6814 studies identified, 85 studies involving 2369 surgical residents were included in this review. The methods used to assess technical competence were categorized into 5 groups; Likert scales (37), benchmarks (31), binary outcomes (11), novel tools (4), and surrogate outcomes (2). Their validity and reliability were mostly previously established. The overall Grading of Recommendations Assessment, Development and Evaluation for randomized controlled trials was high and low for the observational studies. CONCLUSIONS: The definition of technical competence continues to be debated within the medical literature. The methods used to evaluate technical competence predominantly include instruments that were originally created to assess technical skill. Very few studies identify standard setting approaches that differentiate competent versus noncompetent performers; subsequently, this has been identified as an area with great research potential.

Surgical fellowship training in Canada: what is its current status and is improvement required?

This paper examines current issues concerning surgical fellowship training in Canada. Other than information from a few studies of fellowship training in North America, there are scant data on this subject in the literature. Little is known about the demographic characteristics of those who pursue fellowship training in Canada, what the experiences and expectations are of fellows and their supervisors with respect to the strengths and weaknesses of this level of training, or how this level of education fits in with Canadian undergraduate and postgraduate medical training. We summarize current knowledge about fellowship training in Canada as it pertains to demographic characteristics, finances, work hours, residency training, preparation for clinical and research work and satisfaction with training. Most information on surgical fellowship training comes from the United States. As such, we used information from American studies to supplement the Canadian data. Because a surgical fellowship experience in Canada may be different from that in the United States, we propose that Canadian surgical fellows and their supervisors should be surveyed to gain an understanding of such information. This knowledge could be used to improve surgical fellowship training in Canada.

The role of platelet-rich plasma in inguinal wound healing in vascular surgery patients.

The objective was to determine whether incision application of platelet-rich plasma (PRP) will decrease postoperative wound complications in vascular surgery patients. A prospective, randomized trial randomized 81 incisions in 51 patients who underwent femoral artery exposure for elective revascularization procedures or endovascular abdominal aneurysm repairs. Incidence of diabetes, chronic renal failure, prosthetic grafts, body mass index (BMI), and steroid use did not differ. Using the ASEPSIS wound classification system, we found no difference in incidence of wound infection. Wound complications occurred in 9 (23%) of 40 of PRP group and 9 (22%) of 41 of non-PRP. Severe wound complications developed in 5 (13%) PRP and 6 (5%) of non-PRP (P = NS). In multivariate analysis, there were no predictors for wound infection. Groin wound complications rates are common in this patient group. Platelet-rich plasma did not decrease the incidence of groin wound complications in our patients.

Surgeons' and residents' double-gloving practices at 2 teaching hospitals in Ontario.

BACKGROUND: Surgeons and residents are at increased risk of exposure to blood-borne pathogens owing to percutaneous injury (PI) and contamination. One method known to reduce risk is double-gloving (DG) during surgery. METHODS: All surgeons and residents affiliated with the University of Western Ontario (UWO) and McMaster University in 2005 were asked to participate in a Web-based survey. The survey asked respondents their specialty, the number of operations they participated in per week, their age and sex, the proportion of surgeries in which they double-gloved (DG in ≥75% surgeries was considered to be routine), and the average number of PIs they sustained per year and whether or not they reported them to an employee health service. RESULTS: In total, 155 of 331 (47%) eligible surgeons and residents responded; response rates for UWO and McMaster surgeons were 50% and 39%, respectively, and for UWO and McMaster residents, they were 52% and 47%, respectively. A total of 43% of surgeons and residents reported routine DG; 50% from McMaster and 36% from UWO. Using logistic regression to simultaneously adjust for participant characteristics, we confirmed that DG was more frequent at McMaster than at UWO, with an odds ratio of 3.32 (95% confidence interval 1.35-8.17). Surgeons and residents reported an average of 3.3 surgical PIs per year (2.2 among McMaster participants and 4.5 among UWO participants). Of the 77% who reported at least 1 injury/year, 67% stated that they had not reported it to an employee health service. CONCLUSION: Percutaneous injuries occur frequently during surgery, yet routine DG, an effective means of reducing risk, was carried out by less than half of the surgeons and residents participating in this study. This highlights the need for a more concerted and broad-based approach to increase the use of a measure that is effective, inexpensive and easily carried out.

Competency-based medical education: theory to practice.

Although competency-based medical education (CBME) has attracted renewed interest in recent years among educators and policy-makers in the health care professions, there is little agreement on many aspects of this paradigm. We convened a unique partnership - the International CBME Collaborators - to examine conceptual issues and current debates in CBME. We engaged in a multi-stage group process and held a consensus conference with the aim of reviewing the scholarly literature of competency-based medical education, identifying controversies in need of clarification, proposing definitions and concepts that could be useful to educators across many jurisdictions, and exploring future directions for this approach to preparing health professionals. In this paper, we describe the evolution of CBME from the outcomes movement in the 20th century to a renewed approach that, focused on accountability and curricular outcomes and organized around competencies, promotes greater learner-centredness and de-emphasizes time-based curricular design. In this paradigm, competence and related terms are redefined to emphasize their multi-dimensional, dynamic, developmental, and contextual nature. CBME therefore has significant implications for the planning of medical curricula and will have an important impact in reshaping the enterprise of medical education. We elaborate on this emerging CBME approach and its related concepts, and invite medical educators everywhere to enter into further dialogue about the promise and the potential perils of competency-based medical curricula for the 21st century.

Identifying the policy implications of competency-based education.

At their 2009 consensus conference, the International CBME Collaborators proposed a number of central tenets of CBME in order to advance the field of medical education. Although the proposed conceptualization of CBME offers several advantages and opportunities, including a greater emphasis on outcomes, a mechanism for the promotion of learner-centred curricula, and the potential to move away from time-based training and credentialing in medicine, it is also associated with several significant barriers to adoption. This paper examines the concepts of CBME through a broad educational policy lens, identifying considerations for medical education leaders, health care institutions, and policy-makers at both the meso (program, institutional) and macro (health care system, inter-jurisdictional, and international) levels. Through this analysis, it is clear that CBME is associated with a number of complex challenges and questions, and cannot be considered in isolation from the complex systems in which it functions. Much more work is needed to engage stakeholders in dialogue, to debate the issues, and to identify possible solutions.

Successful integration of hepatitis C evaluation and treatment services with methadone maintenance.

OBJECTIVES: Despite high rates of hepatitis C virus (HCV) infection, relatively few current or former injection drug users receive evaluation and treatment for HCV. Here, we demonstrate the feasibility and effectiveness of integrating HCV care and methadone maintenance treatment (MMT). We hypothesized that colocation of these services would result in improved access to and utilization of HCV care. METHODS: In this retrospective observational study, all patient charts from a single MMT clinic were reviewed 2 years after HCV care and MMT were integrated. Information obtained included screening for and counseling about HCV infection status, on-site HCV treatment and outcomes, and demographic and substance abuse data. RESULTS: Two hundred ninety-one patient charts were reviewed. Two hundred eighty-one (99%) patients were screened for HCV antibody (HCV-Ab), and 188 (65%) were positive. Forty-nine (17%) patients were HIV/HCV coinfected. Ninety-eight percent of the HCV-Ab-positive patients received HCV counseling. Hundred fifty-nine (85%) of the HCV-Ab-positive patients were eligible to receive further evaluation and treatment for HCV on site, and 125 (78%) accepted. Hundred eighteen (94%) patients were tested for chronic HCV, and 83 were determined to have chronic HCV. Twenty-five patients received liver biopsy; low-stage disease was found in 7 patients. Twenty-one patients initiated HCV treatment. Sustained viral response was achieved in 8 patients. Seventeen patients had contraindications to HCV treatment. Further workup was prevented or delayed in 45 patients for various reasons, most commonly due to personal choice (29 patients). CONCLUSIONS: This study demonstrates that current and former injection drug users can be engaged successfully in evaluation and treatment of HCV infection when these services are collocated with MMT.