Difficult airway response team: a novel quality improvement program for managing hospital-wide airway emergencies.

BACKGROUND: Difficult airway cases can quickly become emergencies, increasing the risk of life-threatening complications or death. Emergency airway management outside the operating room is particularly challenging. METHODS: We developed a quality improvement program-the Difficult Airway Response Team (DART)-to improve emergency airway management outside the operating room. DART was implemented by a team of anesthesiologists, otolaryngologists, trauma surgeons, emergency medicine physicians, and risk managers in 2005 at The Johns Hopkins Hospital in Baltimore, Maryland. The DART program had 3 core components: operations, safety, and education. The operations component focused on developing a multidisciplinary difficult airway response team, standardizing the emergency response process, and deploying difficult airway equipment carts throughout the hospital. The safety component focused on real-time monitoring of DART activations and learning from past DART events to continuously improve system-level performance. This objective entailed monitoring the paging system, reporting difficult airway events and DART activations to a Web-based registry, and using in situ simulations to identify and mitigate defects in the emergency airway management process. The educational component included development of a multispecialty difficult airway curriculum encompassing case-based lectures, simulation, and team building/communication to ensure consistency of care. Educational materials were also developed for non-DART staff and patients to inform them about the needs of patients with difficult airways and ensure continuity of care with other providers after discharge. RESULTS: Between July 2008 and June 2013, DART managed 360 adult difficult airway events comprising 8% of all code activations. Predisposing patient factors included body mass index >40, history of head and neck tumor, prior difficult intubation, cervical spine injury, airway edema, airway bleeding, and previous or current tracheostomy. Twenty-three patients (6%) required emergent surgical airways. Sixty-two patients (17%) were stabilized and transported to the operating room for definitive airway management. There were no airway management-related deaths, sentinel events, or malpractice claims in adult patients managed by DART. Five in situ simulations conducted in the first program year improved DART's teamwork, communication, and response times and increased the functionality of the difficult airway carts. Over the 5-year period, we conducted 18 airway courses, through which >200 providers were trained. CONCLUSIONS: DART is a comprehensive program for improving difficult airway management. Future studies will examine the comparative effectiveness of the DART program and evaluate how DART has impacted patient outcomes, operational efficiency, and costs of care.

Optimizing Communication in Mechanically Ventilated Patients.

PURPOSE: To describe the types of talking tracheostomy tubes available, present four case studies of critically ill patients who used a specialized tracheostomy tube to improve speech, discuss their advantages and disadvantages, propose patient selection criteria, and provide practical recommendations for medical care providers. METHODS: Retrospective chart review of patients who underwent tracheostomy in 2010. RESULTS: Of the 220 patients who received a tracheostomy in 2010, 164 (74.55%) received a percutaneous tracheostomy and 56 (25.45%) received an open tracheostomy. Among the percutaneous tracheostomy patients, speech-language pathologists were consulted on 113 patients, 74 of whom were on a ventilator. Four of these 74 patients received a talking tracheostomy tube, and all four were able to speak successfully while on the mechanical ventilator even though they were unable to tolerate cuff deflation. CONCLUSIONS: Talking tracheostomy tubes allow patients who are unable to tolerate-cuff deflation to achieve phonation. Our experience with talking tracheostomy tubes suggests that clinicians should consider their use for patients who cannot tolerate cuff deflation.

Development of Machine Learning Copilot to Assist Novices in Learning Flexible Laryngoscopy.

OBJECTIVES: Here we describe the development and pilot testing of the first artificial intelligence (AI) software "copilot" to help train novices to competently perform flexible fiberoptic laryngoscopy (FFL) on a mannikin and improve their uptake of FFL skills. METHODS: Supervised machine learning was used to develop an image classifier model, dubbed the "anatomical region classifier," responsible for predicting the location of camera in the upper aerodigestive tract and an object detection model, dubbed the "anatomical structure detector," responsible for locating and identifying key anatomical structures in images. Training data were collected by performing FFL on an AirSim Combo Bronchi X mannikin (United Kingdom, TruCorp Ltd) using an Ambu aScope 4 RhinoLaryngo Slim connected to an Ambu® aView™ 2 Advance Displaying Unit (Ballerup, Ambu A/S). Medical students were prospectively recruited to try the FFL copilot and rate its ease of use and self-rate their skills with and without the copilot. RESULTS: This model classified anatomical regions with an overall accuracy of 91.9% on the validation set and 80.1% on the test set. The model detected anatomical structures with overall mean average precision of 0.642. Through various optimizations, we were able to run the AI copilot at approximately 28 frames per second (FPS), which is imperceptible from real time and nearly matches the video frame rate of 30 FPS. Sixty-four novice medical students were recruited for feedback on the copilot. Although 90.9% strongly agreed/agreed that the AI copilot was easy to use, their self-rating of FFL skills following use of the copilot were overall equivocal to their self-rating without the copilot. CONCLUSIONS: The AI copilot tracked successful capture of diagnosable views of key anatomical structures effectively guiding users through FFL to ensure all anatomical structures are sufficiently captured. This tool has the potential to assist novices in efficiently gaining competence in FFL. LEVEL OF EVIDENCE: NA Laryngoscope, 2024.

Competency in surgical residency training: defining and raising the bar.

Competency-based surgical residency training is rapidly becoming the norm across surgical specialties. Ensuring that graduating surgeons are competent to deliver the necessary services and skills to their patients remains a seminal objective of training programs. Defining surgical competence, the measures used to assess and quantify that competence, and the criteria used to judge whether it has been achieved are critical issues. The bar that surgical residency programs have established is, and must continue to be, set very high. Definitions of competency differ across disciplines. In education, two approaches are recognized. According to the behaviorist approach, competence is assessed by precise measures of performance, generally documented by checklists. The integrated (holistic) approach defines competence as a complex combination of personal attributes. Assessments of competence also fall under two categories: the traditional scientific paradigm, emphasizing objectivity and reproducibility, and the judgment paradigm, reflecting the need to assess clinical competence in the final stages of medical training. In surgery, competence is the ability to successfully apply professional knowledge, skills, and attitudes to new situations as well as to familiar tasks. A critical step in assessing surgical competency is developing methodology for competency evaluation and certification. Matching different aspects of surgical competency with the appropriate assessment instruments is the theme of the contemporary evaluation process, with emphasis on a whole-task approach and the assessment of professional judgment. An effective assessment program will incorporate several competency elements, using multiple sources of information to assess competencies on multiple occasions, at various levels, and in different settings.

Assessment of Surgical Skills and Competency.

Evaluation of surgical skills and competency are important aspects of the medical education process. Measurable and reproducible methods of assessment with objective feedback are essential components of surgical training. Objective Structured Assessment of Technical Skills (OSATS) is widely used across the medical specialties and otolaryngology-specific tools have been developed and validated for sinus and mastoid surgery. Although assessment of surgical skills can be time-consuming and requires human and financial resources, new evaluation methods and emerging technology may alleviate these barriers while also improving data collection practices.

Balancing standardized testing with personalized training in surgery.

Students pursuing a medical career in the US are subject to standardized testing at regular intervals. These standardized tests not only quantify the milestones students have already achieved, but also define the path for future achievements. The purpose of these examinations is to help students become self-directed, lifelong learners - an essential attribute of a medical professional. However, whether preparing for these examinations actually makes students such disciplined learners needs to be examined. Especially during residency training with its limited time and unpredictable exposure, trainees must learn in the most efficient way for their learning styles, and thus develop attributes that will be helpful to them in their medical career. In this review, we propose that a personalized, learner-centered approach tailored to residents' educational needs and preferences can not only fulfill learning interests and objectives but also serve as a time-efficient and cost-effective approach for graduate medical education.

Development and Pilot Testing of an Objective Structured Clinical Examination for Obstructive Sleep Apnea.

Objective Our aim was (1) to develop an objective structured clinical examination (OSCE) for obstructive sleep apnea (OSA) and (2) to test the reliability and evaluate the feasibility of this OSCE while assessing residents' clinical skills in multiple core competencies via the standardized patient methodology. Study Design Development of assessment tool. Setting Johns Hopkins Medicine Simulation Center. Subjects and Methods Residents of the Department of Otolaryngology-Head and Neck Surgery at The Johns Hopkins University School of Medicine were invited to participate. A 2-station OSCE was developed. The first station used a standardized patient, and the encounter was videotaped for later evaluation by medical faculty not familiar with the participants being tested. The second was a computer-based station developed per a modified Delphi technique, based on feedback from otolaryngology and sleep medicine faculty involved in the care of patients with OSA. Checklists were developed for each station to standardize the evaluation of each trainee's performance. Results The OSCE that we developed is a feasible tool for assessing residents' performance and skills to diagnose and manage a patient with OSA. Internal consistency, as assessed by Krippendorff's alpha, was 0.699 for station 1 and 0.95 for station 2. Conclusion This OSCE was found to be feasible for assessment of clinical competency in OSA. Our model provides targeted assessment of multiple competencies and opportunity to improve clinical knowledge and skills.

Predicting the need for nonstandard tracheostomy tubes in critically ill patients.

PURPOSE: Few guidelines exist regarding the selection of a particular type or size of tracheostomy tube. Although nonstandard tubes can be placed over the percutaneous kit dilator, clinicians often place standard tracheostomy tubes and change to nonstandard tubes only after problems arise. This practice risks early tracheostomy tube change, possible bleeding, or loss of the airway. We sought to identify predictors of nonstandard tracheostomy tubes. MATERIALS AND METHODS: In this matched case-control study at an urban, academic, tertiary care medical center, we reviewed 1220 records of patients who received a tracheostomy. Seventy-seven patients received nonstandard tracheostomy tubes (cases), and 154 received standard tracheostomy tubes (controls). RESULTS: Sex, endotracheal tube size, severity of illness, and computed tomography scan measurement of the distance from the trachea to the skin at the level of the superior aspect of the anterior clavicle were significant predictors of nonstandard tracheostomy tubes. Specifically, trachea-to-skin distance >4.4 cm and endotracheal tube sizes ≥8.0 were associated with nonstandard tracheostomy. CONCLUSIONS: The findings suggest that clinicians should consider using nonstandard tracheostomy tubes as the first choice if the patient is male with an endotracheal tube size ≥8.0 and has a trachea-to-skin distance >4.4 cm on the computed tomography scan.

Remediation of problematic residents--A national survey.

OBJECTIVES/HYPOTHESIS: Despite careful selection processes, residency programs face the challenge of training residents who fall below minimal performance standards. Poor performance of a resident can endanger both patient safety and the reputation of the residency program. It is important, therefore, for a program to identify such residents and implement strategies for their successful remediation. The purpose of our study was to gather information on evaluation and remediation strategies employed by different otolaryngology programs. STUDY DESIGN: Cross-sectional survey. METHODS: We conducted a national survey, sending a questionnaire to the program directors of 106 otolaryngology residency programs. We collected information on demographics of the program, identification of problematic residents, and remediation strategies. RESULTS: The response rate was 74.5%, with a 2% cumulative incidence of problematic residents in otolaryngology programs during the past 10 years. The most frequently reported deficiencies of problematic residents were unprofessional behavior with colleagues/staff (38%), insufficient medical knowledge (37%), and poor clinical judgment (34%). Personal or professional stress was the most frequently identified underlying problem (70.5%). Remediation efforts included general counseling (78%), frequent feedback sessions (73%), assignment of a mentor (58%), and extra didactics (47%). These remediation efforts failed to produce improvement in 23% of the identified residents, ultimately leading to their dismissal. CONCLUSIONS: The apparent deficiencies, underlying causes, and remediation strategies vary among otolaryngology residency programs. Based on the results of this survey, we offer recommendations for the early identification of problematic residents and a standardized remediation plan. LEVEL OF EVIDENCE: NA.

Improving skills development in residency using a deliberate-practice and learner-centered model.

OBJECTIVES/HYPOTHESIS: Work-hour restrictions, increased workload, and subjective assessment of competency are major threats to the efficacy of the traditional apprenticeship model of surgical training in modern surgical practice. In response, medical educators are developing time- and resource-efficient competency-based models of surgical training. The purpose of our project was to develop, implement, and measure the outcomes of such objective and structured programs in otolaryngology. We also investigated factors affecting the learning curve, especially deliberate practice, formative feedback, and learners' autonomy. STUDY DESIGN: Prospective, longitudinal study. METHODS: To measure the surgical skills of residents, we first developed and tested objective tools for otolaryngology procedures. Based on these instruments, we identified milestones of the procedures. Training on a virtual-reality simulator was validated to shorten the learning curve. We also studied a learner-centered approach of training, factors affecting the learning curve, and barriers to a competency-based model. RESULTS: The objective tools were found to be a feasible, reliable, and valid opportunity for measuring competency in both the laboratory and operating room. With the formative assessment from these tools, residents had a remediation target to be achieved by deliberate practice. The milestones helped identify the threshold of competency, and deliberate practice on the simulator gave an opportunity for improving skills. The learner-centered approach allowed flexibility and personalized learning by shifting the responsibility of the learning process to the learners. CONCLUSION: The competency-based model of residency, based on the principles of deliberate practice and a learner-centered approach, is a feasible model of residency training that allows development of competent surgeons and hence improves patient outcomes. Despite these advantages, challenges to this model require a concerted effort to overcome and fully implement these principles of training beyond just technical skills, ultimately creating well-rounded medical professionals and leaders in the surgical field. LEVEL OF EVIDENCE: N/A. Laryngoscope, 125:S1-S14, 2015.

Identifying quality indicators of surgical training: A national survey.

OBJECTIVES/HYPOTHESIS: Evidence shows a positive association between quality of surgical training received and patient outcomes. Traditionally, improved patient outcomes are linked with increased operative volume. However, generalizing this finding to surgeons in training is unclear. In addition, reduced exposure due to work-hour restrictions calls for alternative methods to determine the quality of training. The purpose of this study was to identify the indicators of high-quality training by surveying the trainees and trainers. METHODS: A questionnaire was developed based on input from faculty and previous studies. The survey was divided into three sections asking about the indicators of quality training, methods to measure them, and interventions for improvement. The questionnaire was administered to program directors (PDs) and senior residents of otolaryngology training programs nationwide. RESULTS: The strongest indicators of quality training that were agreed upon by both residents and PDs were having faculty development as an ideal trainer while having a balanced level of supervision and independence, logbooks for exposure to volume and variety of pathology, continuous evaluation and provision of feedback. However, structured teaching, simulation-based training, and trainee exam scores failed to reach an agreement as a metric of high-quality surgical training. CONCLUSION: Measuring quality of a residency training program is imperative to produce competent surgeons and ensuring patient safety. The results of this study will help the residency programs to better train their residents and improve the quality of their teaching. LEVEL OF EVIDENCE: N/A.

Effect of distractions on operative performance and ability to multitask--a case for deliberate practice.

OBJECTIVES/HYPOTHESIS: To measure the effect of distractions on the operative performance and analyze if practice and experience are the factors that can help to overcome the distractions. STUDY DESIGN: Prospective observational study. METHODS: Ten postgraduate year (PGY) 2-6 residents and two faculty members from Johns Hopkins' otolaryngology department were recruited and asked to deepen the dissection at the sinodural angle on the Voxel-man mastoidectomy simulator. They were asked to perform the task under four conditions: 1) no distractors, 2) differentiation and counting of a specific alarm sound among different sounds played in the background while performing the surgical task, 3) simultaneous performance of simple arithmetic task of moderate difficulty, and 4) simultaneous performance of the task with both sets of distractors combined. RESULTS: Time taken for the task (P = .02) and error scores (P = .002) increased under the third and fourth conditions. The ability to multitask and response to surgical and cognitive tasks improved with increasing level of experience of the participants. CONCLUSIONS: Distractions lead to impaired dexterity and an increase in the incidence of errors. However, experience and deliberate practice can help achieve the ability to multitask without compromising the operative performance.

A learner-centered educational curriculum improves resident performance on the otolaryngology training examination.

OBJECTIVES/HYPOTHESIS: To examine the effects of a learner-centered educational curriculum for Johns Hopkins otolaryngology-head and neck surgery residents in improving their otolaryngology training examination (OTE) scores. We hypothesized that trainees who attend a learner-centered program demonstrate improved performance on OTEs. STUDY DESIGN: Prospective longitudinal study. METHODS: In September 2008, a resident-designed educational curriculum was implemented to better prepare residents for their OTE. These mandatory sessions, led by faculty members, were held for an hour every week, during in which residents were tested on their knowledge of high-yield topics. Residents were expected to be prepared on pathophysiology, diagnosis, and treatment of the selected diseases. In 2011, residents were given responsibility to lead these sessions. OTE scores from 2002 to 2012 were analyzed to see whether there was any improvement after the implementation of this curriculum. RESULTS: Clustered linear regression analysis revealed significant improvement of OTE scores with subsequent interventions. During a 3-year period (2009-2011) after the first intervention, there was a significant increase (P = .01) of 0.69 in mean national and 0.78 in mean group stanine scoring. Similarly, after the second intervention in 2011, a further increment of 1.36 in mean national and 1.58 in mean group stanine scoring was seen (P = .001). CONCLUSIONS: Residents OTE stanines improved significantly after the implementation of a learner-centered educational curriculum. This finding suggests that trainees are able to perform better when involved in planning and implementing the educational curriculum.

Utilization of a standardized tracheostomy capping and decannulation protocol to improve patient safety.

OBJECTIVES/HYPOTHESIS: To develop and assess the feasibility of a new standardized protocol to guide tracheostomy decannulation. STUDY DESIGN: Descriptive review of quality improvement project. METHODS: A quality improvement project was conducted in the inpatient setting of a tertiary urban academic hospital. Adult patients who had received a tracheostomy and for whom the indication for tracheostomy had resolved were included. A multidisciplinary task force reviewed input from clinicians caring for tracheostomy patients and developed a protocol for screening, capping, and decannulation. The primary outcome measured was successful decannulation. RESULTS: Fifty-seven patients were screened for a capping trial over a 12-month period; 54 were capped. Six patients were lost to follow-up. Fifty patients passed the capping trial, and all 50 were decannulated successfully. When decannulation was pursued in one patient who had twice failed the screening criteria and subsequent capping trials, the patient failed decannulation and ultimately required reintubation for the management of secretions. The screening tool had high sensitivity (90%) and positive predictive value (100%) for successful decannulation. Additionally, the number of reported patient safety concerns decreased from seven in the 6 months preceding implementation of the program to one report in the 6 months after implementation. CONCLUSION: The new tracheostomy capping and decannulation protocol assisted in predicting both successful and failed decannulation. Although several patients failed certain capping criteria initially, the protocol stipulated modifications of care that enabled successful decannulation. The screening tool had high sensitivity and promoted communication, standardization of practice, and patient safety.

Assessment of improvement of trainee surgical skills in the operating room for tonsillectomy.

OBJECTIVES/HYPOTHESIS: To determine the feasibility, reliability, and validity of an objective surgical assessment tool designed to measure the development of tonsillectomy skills by resident trainees in the operating room. A tonsillectomy evaluation instrument created previously serves as a basis for this tool, which incorporates many similar concepts but was optimized to maximize feasibility and reliability with simplified anchors and inclusion of global and task-specific sections. STUDY DESIGN: Prospective longitudinal validation study. METHODS: Faculty input via modified Delphi technique was used to develop a new objective structured assessment of technical skills-based instrument for tonsillectomy. The task-specific checklist (TSC) and global rating scale (GRS) measured tonsillectomy technical skills and overall surgical performance, respectively. Twenty-one otolaryngology-head and neck residents (ranging from postgraduate year 1 to 6) were evaluated for a period of 3 years by 11 faculty members. RESULTS: Eighty-three evaluations were completed showing strong correlation between both instruments (r = 0.95, P < .001). Our tool demonstrated construct validity for both TSC and GRS, showing higher scores with increasing surgical experience. Both instruments showed high interitem reliability with Cronbach α coefficients of 0.97 for both parts. CONCLUSIONS: This assessment tool is a feasible, reliable, and valid instrument for the assessment of surgical competency in tonsillectomy. It is effective in providing structured feedback at the end of each procedure, which encourages specific, targeted development.

Achievement of competency in endoscopic sinus surgery of otolaryngology residents.

OBJECTIVES/HYPOTHESIS: The goal of our study is to identify the number of Endoscopic Sinus Surgery (ESS) cases required to obtain competency in ESS, using a previously validated assessment tool. STUDY DESIGN: Prospective observational study. METHODS: Seventeen residents from Johns Hopkins Otolaryngology-Head & Neck Surgery residency program were evaluated as they performed endoscopic sinus surgery in the operating room. Global and checklist parts of the ESS instrument were used for assessment purposes. Items on the tool were grouped into three different milestones for analysis of data (Milestone 1 = Maxillary Antrostomy + Anterior Ethmoidectomy, Milestone 2 = Posterior Ethmoidectomy + Sphenoidostomy, Milestone 3 = Frontal Sinusotomy). Residents were deemed competent if they achieved a minimum score of 3 on a 5-point Likert scale on each step of the surgery. RESULTS: A total of 73 evaluations were completed for 17 residents (Postgraduate Level 2-5) by eight evaluators between 2009 and 2011. A 60% probability of achieving competency in performance of all milestones of ESS is obtained with performing 42 ESS procedures and the probability is increased to a 100% with performance of 55 endoscopic sinus surgery procedures. On average it took residents 23 cases to become competent in performance of maxillary antrostomy and anterior ethmoidectomy. CONCLUSIONS: Our results suggest that it requires 42 ESS procedures to attain a 60% probability of competency in ESS. These results have implications for otolaryngology residency programs when developing curriculum and benchmarks for the training residents.

Otolaryngology training programs: resident and faculty perception of the mentorship experience.

OBJECTIVES/HYPOTHESIS: The purpose of this study is to compare faculty and trainees' perceptions of their mentorship experience in otolaryngology training programs in order to identify areas where improvements could yield more productive mentorship. STUDY DESIGN: Cross-sectional survey design. METHODS: Residents and faculty from three otolaryngology-head and neck surgery programs were surveyed regarding their perceptions of their mentoring relationship. Trainees were asked about the characteristics of their mentorship experience. Separately, faculty were asked to describe their mentorship relationship, available resources to provide effective mentorship, and to identify areas in which formal training would improve their mentoring skills. RESULTS: Forty-eight trainees (72%) and sixty-one faculty members (73%) completed the survey. Ninety percent of residents meet with their mentors at least twice a year. Faculty and residents (> 80%) agreed that career planning was the most commonly addressed topic in mentorship sessions. However, faculty and residents differed in their perceptions of providing mentorship in other areas including clinical judgment (P = 0.003). The majority of faculty (56%) felt that formal mentorship training would improve their mentorship skills. While 95% of mentees agreed that their mentor is accessible, only 46% of faculty believed they have enough time to dedicate to mentoring (P < 0.001). CONCLUSIONS: Mentees are generally satisfied with the mentoring they receive, while most mentors are not satisfied with the time they have to provide mentorship. Further insights into differences in faculty and trainee perceptions may improve the mentorship experience.

Determinants of resident competence in mastoidectomy: role of interest and deliberate practice.

OBJECTIVES/HYPOTHESIS: This study explores the influence of selected factors on achievement of competency in mastoid surgery. STUDY DESIGN: A prospective study of surgical performance and a retrospective survey of learner and training factors. METHODS: The longitudinal performance of 15 residents was evaluated using the mastoidectomy task-based checklist (TBC). The influence of surgical experience, resident interest, and training factors (course attendance, voluntary use of simulation laboratory) was also examined and compared for the acquisition of distinct levels of technical skill difficulty (cortical mastoidectomy vs. facial recess). RESULTS: Ninety-six observations made during the first otology rotation were analyzed. Cortical mastoidectomy tasks showed positive associations with cumulated case numbers (OR 1.13, CI 1.04-1.23, P = .003) and interest in otology (OR 3.86, CI 1.21-12.27, P = .022). Facial recess tasks showed a larger positive association with interest in otology (OR 10.38, CI 2.25-47.94, P = .003), and negative association with extra time spent in laboratory practice (OR .05, CI 0.011-0.23, P = .000). CONCLUSIONS: Learning trajectory for cortical mastoidectomy and facial recess may be influenced by different factors. Interest in otology, in particular, may have a moderating effect on the acquisition of more complex skills in mastoid surgery. A negative association between self-directed laboratory simulation and performance may reflect the impact of assessment-guided feedback in promoting deliberate practice. Further investigations are suggested to explore the interactions between individual trainee differences, educational models, and learning outcomes.

Comprehensive assessment of thyroidectomy skills development: a pilot project.

OBJECTIVES/HYPOTHESIS: To test the validity, reliability, and feasibility of an evaluation tool designed to measure the development of trainees' surgical skills in the operating room for thyroid surgery. STUDY DESIGN: Prospective validation study. METHODS: A modified Delphi technique was employed to develop a new Objective Structured Assessment of Technical Skills-based instrument for thyroid surgery. During a 1-year period, 16 otolaryngology-head and neck surgery residents (ranging from postgraduate year 2 to 6) and one endocrine surgery fellow were evaluated by one faculty member obtaining a total of 94 evaluations. Performance was rated using a task-based checklist (TBC) and a global rating scale (GRS). The TBC measured trainees' thyroidectomy technical skills, and the GRS assessed their overall surgical performance. RESULTS: Based on four clinical levels (junior, intermediate, senior, and surgical fellow) our tool demonstrated construct validity for both components of the assessment instrument, specifically for the TBC showing a mean difference of 0.9 (95% confidence interval: 0.5-1.3, P<.001) between the contiguous clinical levels senior versus intermediate. Cronbach α, a measure of internal consistency, was 0.96 for both components of the instrument. The correlation between the TBC and GRS was also high within trainee (r=0.62, n=94, P<.001) and across trainees (r=0.96, n=17, P<.001). CONCLUSIONS: Our tool proved to be a valid, reliable, and feasible instrument for assessing competency in thyroid surgery. It is effective in providing timely formative feedback during and upon the conclusion of the surgical procedure by identifying procedural tasks for which additional training is necessary. In addition, it enables longitudinal tracking of residents' surgical performance, thus ensuring their appropriate development.