The timely completion of objective assessment tools for evaluation of technical skills.

OBJECTIVES/HYPOTHESIS: To this date the effect of the time taken to complete an evaluation on the psychometric properties of the instrument has not been reported. The goal of our study was to assess the effect of time taken to complete an evaluation on its validity. STUDY DESIGN: Cross-sectional validation study. METHODS: The global and checklist parts of tonsillectomy, mastoidectomy, rigid bronchoscopy, and endoscopic sinus surgery were used in the operating room by the otolaryngology faculty to evaluate the surgical skills of the residents. We categorized evaluations into two groups depending on the time taken to complete an evaluation (group A ≤ 6 days, group B >6 days). Construct validity was calculated for both groups by comparing the mean global and checklist scores of the residents across advancing postgraduate year levels. RESULTS: A total of 468 evaluations, consisting of global and checklist parts, were completed for 29 residents by 32 evaluators. Mean number of days taken to complete an evaluation was 7.7 days. For all the evaluations completed within a 6-day time period, the construct validity was significant for both global and checklist parts of the four instruments. In cases of the evaluations completed after 6 days, the construct validity was significant for the tonsillectomy instrument only. CONCLUSIONS: Our results indicate that the time taken to complete an evaluation has a significant effect on the construct validity of the objective instrument. In the future, efforts should be focused on faculty development to ensure timely completion of the evaluation for a more valid assessment process.

Comprehensive assessment of thyroidectomy skills development: a pilot project.

OBJECTIVES/HYPOTHESIS: To test the validity, reliability, and feasibility of an evaluation tool designed to measure the development of trainees' surgical skills in the operating room for thyroid surgery. STUDY DESIGN: Prospective validation study. METHODS: A modified Delphi technique was employed to develop a new Objective Structured Assessment of Technical Skills-based instrument for thyroid surgery. During a 1-year period, 16 otolaryngology-head and neck surgery residents (ranging from postgraduate year 2 to 6) and one endocrine surgery fellow were evaluated by one faculty member obtaining a total of 94 evaluations. Performance was rated using a task-based checklist (TBC) and a global rating scale (GRS). The TBC measured trainees' thyroidectomy technical skills, and the GRS assessed their overall surgical performance. RESULTS: Based on four clinical levels (junior, intermediate, senior, and surgical fellow) our tool demonstrated construct validity for both components of the assessment instrument, specifically for the TBC showing a mean difference of 0.9 (95% confidence interval: 0.5-1.3, P<.001) between the contiguous clinical levels senior versus intermediate. Cronbach α, a measure of internal consistency, was 0.96 for both components of the instrument. The correlation between the TBC and GRS was also high within trainee (r=0.62, n=94, P<.001) and across trainees (r=0.96, n=17, P<.001). CONCLUSIONS: Our tool proved to be a valid, reliable, and feasible instrument for assessing competency in thyroid surgery. It is effective in providing timely formative feedback during and upon the conclusion of the surgical procedure by identifying procedural tasks for which additional training is necessary. In addition, it enables longitudinal tracking of residents' surgical performance, thus ensuring their appropriate development.

Learning styles in otolaryngology fellowships.

OBJECTIVES/HYPOTHESIS: Previous studies have identified a predominant learning style in trainees from different specialties, more recently in otolaryngology residents. The purpose of our study was to determine a predominant learning style within otolaryngology fellowships and to identify any differences between otolaryngology fellows and residents. STUDY DESIGN: We conducted a survey of otolaryngology fellows at 25 otolaryngology fellowship programs accredited by the Accreditation Council for Graduate Medical Education. METHODS: We emailed Kolb's Learning Style Index version 3.1 to 16 pediatric otolaryngology (PO) and 24 otology/neurotology (ON) fellows. This index is a widely used 12-item questionnaire. The participants answered each item in the questionnaire as it applied to their preferred learning style: accommodating, converging, diverging, or assimilating. Results were then analyzed and compared between each subspecialty and the previously reported preferred styles of otolaryngology residents. RESULTS: Ten PO and 20 ON fellows completed the survey, with an overall response rate of 75%. PO and ON fellows (60% of each group) preferred a learning style that was "balanced" across all four styles. For ON fellows, 35% preferred converging and 5% preferred accommodating styles. For PO fellows, converging and accommodating styles accounted for 20% each. CONCLUSIONS: It was previously reported that 74.4% of otolaryngology residents prefer either converging or accommodating styles. We believe that the fellowship training environment calls for fellows to use more than one learning style to become proficient physicians, hence the trend toward potentially developing a balanced style when at this level.

In-training assessment and predictors of competency in endoscopic sinus surgery.

OBJECTIVES: To evaluate the reliability and validity of the assessment tool for endoscopic sinus surgery when used in the operating room for in-training assessment of operative competency; to identify the tasks that may serve as the best indicators for overall surgical performance. STUDY DESIGN: Cross-sectional validation study. METHODS: We implemented the global and checklist parts of the endoscopic sinus surgery (ESS) assessment tool to evaluate the surgical skills of 13 Otolaryngology-Head & Neck Surgery residents (PGY 1-5) in the operating room over a period of 15 months. Rhinology faculty scored residents' performance of every step of ESS at the end of each procedure using a previously validated tool. Construct validity was calculated by comparing scores across training levels (using analysis of variance [ANOVA]). Regression analysis was performed to identify tasks on the ESS checklist that most strongly correlated to the overall surgical performance. RESULTS: Construct validity was demonstrated with senior residents performing better than junior residents. Average checklist and global scores improved with the number of days on rhinology rotation. "Identification of uncinate and boundaries" was found to be the strongest predictor of overall surgical performance. CONCLUSIONS: The results indicate that this evaluation tool is a reliable and valid instrument for the assessment of surgical competency in the operating room. It can be used to identify weak areas of performance for which additional training may be required early in the rotation/training.

Learning curve for competency in flexible laryngoscopy.

OBJECTIVES/HYPOTHESIS: The purpose of our study was to identify the number of attempts required to attain competency in performing flexible laryngoscopy. STUDY DESIGN: Cross-sectional prospective study. METHODS: Fifteen medical students were recruited to perform flexible laryngoscopy on a mannequin. Each participant was given unlimited time and attempts to perform the procedure until considered competent by the evaluator for two consecutive attempts. Three evaluators used a flexible laryngoscopy checklist to score performance on each step of the procedure. Time required to perform the procedure was recorded, as well as number of times the scope hit the mucosa. The criteria for attaining competence were achieving a minimum score of 3 out of 5 on all the items of the checklist and being deemed competent by the evaluator. RESULTS: A total of 105 flexible laryngoscopies were performed by 15 medical students. A mean of six attempts (range, 2-17) were necessary for a medical student to become competent in performing flexible laryngoscopy. An 80% probability of becoming competent was achieved with the 14th attempt. An inverse relationship was noted between the number of times the scope hit the mucosa and the probability of being competent. The time taken to perform the procedure decreased with increasing number of attempts. CONCLUSIONS: Our results suggest that it takes six attempts on average for a novice to become competent in performing flexible laryngoscopy. This finding has implications for residency programs because it indicates the learning curve can be overcome in the laboratory rather than with patients. Laryngoscope, 2010.

Barriers to the implementation of competency-based education and assessment: a survey of otolaryngology program directors.

OBJECTIVES/HYPOTHESIS: To identify the barriers faced by otolaryngology program directors as they implement competency-based education and assessment and to identify preferred approaches to meet these challenges as suggested by program directors. STUDY DESIGN: A national survey of otolaryngology-head and neck surgery program directors. METHODS: We developed a 20-item questionnaire that was distributed to 102 otolaryngology program directors through SurveyMonkey. Nonrespondents were reminded by follow-up email and phone calls. Results were analyzed by descriptive statistical analysis. RESULTS: A total of 88 (86%) program directors responded to the survey. There was a marked discrepancy between the income received and time spent performing the duties of the program director. Program director workload was recognized as the most important barrier to the implementation of competency-based education. Creating a practical clearinghouse of existing and emerging assessment tools was given the highest rating among the approaches to meet the challenges faced by program directors. CONCLUSIONS: Program directors in otolaryngology do not have sufficient financial support, protected time, and personnel to fulfill their administrative and educational responsibilities. They should be provided with additional institutional assistance to help them achieve the goals of the Accreditation Council for Graduate Medical Education outcome project.

Learning styles in two otolaryngology residency programs.

OBJECTIVES/HYPOTHESIS: Kolb portrays four learning styles depending on how an individual grasps or transforms experience: accommodating, assimilating, diverging, and converging. Past studies in surgery, medicine, and anesthesia identified the predominant learning style in each of these specialties. The prevalence of different learning styles and existence of a predominant style, if any, has not been reported for otolaryngology residency programs. The purpose of our study was to determine if otolaryngology residents have a preferred learning style that is different from the predominant learning styles reported for other specialties. STUDY DESIGN: We conducted a survey of the otolaryngology-head and neck surgery residents at two residency programs. METHODS: Kolb's Learning Style Index (LSI) version 3.1 was administered to 46 residents from Johns Hopkins University and Kansas University Otolaryngology-Head and Neck Surgery programs. LSI is a widely used 12-item questionnaire, with each item followed by four options. The subjects graded the options depending on how the options applied to them. RESULTS: Forty-three otolaryngology residents completed the survey, with a response rate of 93.47%. The predominant learning style was converging (55.81%) followed by accommodating (18.61%), accounting for the learning styles of 74.42% of the total population. There were only 13.95% assimilating and 6.98% diverging learning styles. Two residents (4.65%) had their preference balanced across four learning styles. CONCLUSIONS: The predominant learning styles in otolaryngology were converging and accommodating, accounting for three fourths of the population. It would be desirable to modify our curriculum in a way that will optimize and facilitate learning.

Competency in surgical residency training: defining and raising the bar.

Competency-based surgical residency training is rapidly becoming the norm across surgical specialties. Ensuring that graduating surgeons are competent to deliver the necessary services and skills to their patients remains a seminal objective of training programs. Defining surgical competence, the measures used to assess and quantify that competence, and the criteria used to judge whether it has been achieved are critical issues. The bar that surgical residency programs have established is, and must continue to be, set very high. Definitions of competency differ across disciplines. In education, two approaches are recognized. According to the behaviorist approach, competence is assessed by precise measures of performance, generally documented by checklists. The integrated (holistic) approach defines competence as a complex combination of personal attributes. Assessments of competence also fall under two categories: the traditional scientific paradigm, emphasizing objectivity and reproducibility, and the judgment paradigm, reflecting the need to assess clinical competence in the final stages of medical training. In surgery, competence is the ability to successfully apply professional knowledge, skills, and attitudes to new situations as well as to familiar tasks. A critical step in assessing surgical competency is developing methodology for competency evaluation and certification. Matching different aspects of surgical competency with the appropriate assessment instruments is the theme of the contemporary evaluation process, with emphasis on a whole-task approach and the assessment of professional judgment. An effective assessment program will incorporate several competency elements, using multiple sources of information to assess competencies on multiple occasions, at various levels, and in different settings.

The role of pre-operative CT-guided FNAB for parapharyngeal space tumors.

OBJECTIVE: To determine the role of computed tomography (CT)-guided fine needle aspiration biopsy (FNAB) in surgical planning for parapharyngeal space (PPS) tumors. DESIGN AND SETTING: Chart review of 49 consecutive patients with surgically treated PPS tumors from 1995 to 2005. RESULTS: Twenty-nine patients had CT-guided FNAB. A cytopathologic diagnosis that was the same as final pathology was rendered in 14 (48%) patients; suggestive but not conclusive in 6 (21%) patients; discordant in 3 (10%) patients; and 6 (21%) patients had a nondiagnostic result. Fourteen of 15 patients who had a final histopathologic finding of pleomorphic adenoma had a correct or highly suggestive preoperative FNAB diagnosis. The positive predictive value for CT-guided FNAB to identify benign tumors is 90%, (18 of 20) but to identify malignant PPS tumors is 75% (3 of 4). CONCLUSION: CT-guided FNAB of PPS tumors is helpful to predict the nature of the PPS tumors (especially benign), which allows the surgeon and patient to plan for treatment, accordingly.

Supracricoid laryngectomy outcomes: The Johns Hopkins experience.

OBJECTIVE: To report the oncologic and functional results from our experience in performing supracricoid laryngectomy (SCL) for selected patients with laryngeal cancer. STUDY DESIGN: Retrospective chart review. METHODS: Twenty-four consecutive patients who underwent SCL for laryngeal cancer in our institution from December 2000 to March 2006 have been reviewed. Reports of the site and extent of tumor, type of reconstruction, preoperative or postoperative radiotherapy, and the final histopathologic examination were reviewed. In addition, the reports of the preoperative examination, inpatient course, and postoperative follow-up were reviewed. RESULTS: A total of 24 patients were involved in the study; 19 had tumors involving the glottic region, and 5 patients had tumors involving both the glottic and supraglottic regions (transglottic). Ten patients had their SCL for postradiotherapy recurrence/persistence of disease. Eighteen patients underwent reconstruction through cricohyoidoepiglottopexy (CHEP), whereas six patients had cricohyoidopexy (CHP). Eleven patients had an arytenoid cartilage resected; 8 of 11 had CHEP, and 3 of 11 had CHP. All patients had a tracheostomy and percutaneous endoscopic gastrostomy (PEG) tube placement performed at the same time as the SCL. The median hospital stay period was 6 days. Twenty-three of 24 had successful tracheostomy tube decannulation, with a median time to decannulation of 37 days. The median time to removal of the PEG tube was 70 days. The complications with SCL were postoperative wound infection in two patients (SCL/CHP) and the need for completion total laryngectomy secondary to intractable aspiration in one patient with SCL/CHP. One patient with SCL/CHEP had a ruptured pexy and subsequently underwent a second reconstruction with successful tracheostomy and PEG tube removal. One of 24 patients is still PEG tube dependant, and he had postoperative radiotherapy. Fifteen patients underwent concurrent neck dissection. None of the patients had any local or regional recurrence, with a median follow-up period of 3 years. All final surgical margins were negative for tumor invasion. Three patients had postoperative radiotherapy, two patients because of nodal metastases in the excised lymph nodes and one because of perineural invasion on final histopathologic examination of the SCL specimen. There were no perioperative deaths. CONCLUSION: SCL with CHEP or CHP represents an effective technique that can be taught and effectively used to avoid a total laryngectomy while maintaining physiologic speech and swallowing in selected patients with advanced stage primary laryngeal cancer or recurrent/persistent laryngeal cancer after radiotherapy. There is a good functional recovery with acceptable morbidity and an excellent oncologic outcome when strict selection criteria are applied and a formal swallowing rehabilitation program is followed.

Importance of routine evaluation of the thyroid gland prior to open partial laryngectomy.

OBJECTIVES: To determine the incidence and presentation of synchronous thyroid lesions in a patient population undergoing open partial laryngectomy (OPL), and to determine whether routine preoperative evaluation of the thyroid gland prior to OPL is useful to identify synchronous thyroid lesions in order to reduce the need for reoperation in this complex patient population. DESIGN: Retrospective medical chart review. SETTING: Academic institution. PATIENTS: Sixty-seven consecutive patients with laryngeal tumors who had undergone OPL from 1996 to 2005. INTERVENTIONS: Charts of 67 consecutive patients with laryngeal tumors who underwent OPL in 1996 to 2005 have been reviewed for synchronous thyroid lesions. For all patients, reports of (1) complete preoperative examination findings, (2) inpatient course, (3) postoperative follow-up, and (4) postoperative final histopathologic findings were reviewed. For patients with synchronous thyroid lesions, reports of (1) thyroid evaluation and imaging and (2) preoperative (fine-needle aspiration), (3) intraoperative (frozen section), and (4) postoperative (final) histopathologic results for the thyroid lesions were reviewed. MAIN OUTCOME MEASURES: Incidence of synchronous thyroid lesions and laryngeal cancer in patients undergoing OPL. RESULTS: Eight (11.9%) of 67 (95% confidence interval, 5.3%-22.2%) patients with laryngeal tumors who underwent OPL had evidence of synchronous thyroid lesions. All 8 patients had squamous cell carcinoma of the larynx and underwent either supracricoid or supraglottic laryngectomy. In these 8 patients, synchronous thyroid lesions were incidentally detected. Four patients had papillary thyroid carcinoma, 1 had squamous metaplasia, and 3 had follicular thyroid tissue that was negative for malignancy on final pathologic examination. In 2 patients, the thyroid lesions were detected preoperatively (prior to OPL); in another 2 patients, thyroid masses were detected intraoperatively; and in 4 patients, the thyroid disease was identified postoperatively on histopathologic examination of excised cervical lymph nodes. In 2 patients, thyroidectomy was performed as a second operation after the OPL, and 1 of them had transient vocal fold paralysis for 2 months. Thyroid ultrasonography was performed in 4 patients. In 3 patients, the ultrasonography was performed after the OPL final pathologic findings indicated the presence of metastatic thyroid disease in cervical lymph nodes. Ultrasonography revealed intrathyroidal lesions in all 3 patients. CONCLUSIONS: Patients with laryngeal tumors who will be undergoing OPL might have occult synchronous thyroid lesions. Thyroid surgery in patients with previous OPL may have an increased potential for complication owing to postsurgical changes in the central neck region. Routine preoperative evaluation of the thyroid gland, especially with ultrasonography, to screen for occult synchronous thyroid lesions is recommended for all patients with laryngeal tumors who will be undergoing OPL. Eradication of any thyroid cancer detected preoperatively by fine-needle aspiration should be performed at the same time as OPL. Pros and cons of total thyroidectomy for indeterminate thyroid nodules should be discussed with this patient population.

Neck management in patients undergoing postradiotherapy salvage laryngeal surgery for recurrent/persistent laryngeal cancer.

OBJECTIVE: To determine a plan for the management of cervical lymph nodes in patients undergoing salvage laryngeal surgery (SLS) for recurrent/persistent laryngeal cancer after primary radiotherapy (RT). STUDY DESIGN: : Retrospective chart review. METHODS: Charts of 51 consecutive patients who had salvage total or supracricoid laryngectomy with or without neck dissection for recurrent/persistent laryngeal squamous cell carcinoma after primary RT from 1988 to 2005 in our institution were reviewed. No patients received concomitant or neo-adjuvant chemotherapy. Thirty-four patients underwent SLS along with unilateral or bilateral neck dissection, whereas 17 patients underwent the SLS without neck dissection. Reports of preRT and preSLS staging of the primary tumor and the neck, recorded using the TNM system, were reviewed. Reports of the final histopathologic examination for the excised laryngeal cancer and cervical lymph nodes were reviewed. RESULTS: Thirty-four patients underwent SLS with unilateral or bilateral neck dissection. The preRT staging of the primary tumor for those 34 patients showed that 32 (94%) were staged T-1 (14) and T-2 (18), whereas the preSLS staging of the primary tumor for those 34 patients showed that 29 (85%) were staged T-3 and T-4. The postSLS final histopathologic examination of the excised lymph nodes in those 34 patients demonstrated that 30 (88%) did not have any evidence of nodal metastasis. On comparing patients with and without nodal metastasis (on their postSLS final histopathology), we found that the preSLS neck staging, based on computed tomographic (CT) scanning of the neck, was significantly associated with the negative/positive postSLS status of nodal metastasis (P = .006). Of 29 patients staged preSLS as N-0, 28 (97%) patients did not have nodal metastasis on their postSLS final pathology (negative predictive value = 97%, confidence interval, 82.2-99.9). PreRT neck staging, preRT and preSLS staging of the primary tumor, along with laryngeal subsite involvement (supraglottis, glottis, subglottis) did not significantly correlate with the status of neck metastasis on final postSLS histopathology (P = .68, 0.78, 0.49, and 0.42, respectively). None of the 34 patients had any neck tumor recurrence in the postSLS follow-up period (median, 3 yr). In addition, all 17 patients who underwent SLS without neck dissection were staged N-0 both before RT as well as preSLS, and none developed neck disease in the postSLS follow-up period (median, 2.5 yr). CONCLUSION: Management of the neck in patients undergoing salvage total or supracricoid laryngectomy for laryngeal cancer recurrence/persistence after primary RT should be based on the preSLS CT staging of the neck. Patients staged N-0 preSLS are not likely to harbor occult nodal metastasis and therefore may not require elective neck dissection.

The role of positron emission tomography and computed tomography fusion in the management of early-stage and advanced-stage primary head and neck squamous cell carcinoma.

OBJECTIVE: To evaluate the role of positron emission tomography and computed tomography (PET-CT) fusion in the management of early-stage and advanced-stage primary head and neck squamous cell cancer. DESIGN: Retrospective analysis, with a blinded evaluation of clinical data and formation of a treatment plan. SETTING: Single tertiary academic medical institution. Patients Thirty-six patients with previously untreated head and neck squamous cell carcinoma who underwent staging CT or magnetic resonance imaging of the neck prior to undergoing PET-CT as part of their initial diagnostic evaluation between July 2000 and January 2005. MAIN OUTCOME MEASURES: Confirmation or alteration of the treatment plan with the addition of the PET-CT information compared with traditional clinical and radiological data alone for early-stage and advanced-stage disease. When available, histopathological results were correlated with the PET-CT findings. RESULTS: Among the 36 patients, PET-CT provided additional information that confirmed the treatment plan in 25 patients (69%) and altered the treatment plan in 11 patients (31%). Six of 11 patients in the altered-treatment group had their tumors upstaged. Four of 8 patients with early-stage tumors had their treatment plan altered, compared with 7 of 28 patients with advanced-stage disease. Among 18 patients who underwent a surgical intervention for their primary tumor, PET-CT identified the primary tumor in all 18 patients and, based on histopathological findings, correctly staged the regional nodal disease in 9 of 16 patients who had their nodal disease addressed. CONCLUSION: The use of PET-CT is important in the initial treatment planning of early-stage and advanced-stage head and neck squamous cell carcinoma.

Oncocytic neoplasms of the parotid gland: a 16-year institutional review.

BACKGROUND: Oncocytic neoplasms of the parotid gland are a rare collection of salivary gland tumors that include oncocytosis, oncocytoma, and oncocytic carcinoma. Mounting evidence has linked the presence of oncocytes to acquired mitochondrial dysfunction. OBJECTIVES: The study goals were to further delineate this poorly understood group of salivary gland tumors and to search for genetic alterations indicative of mitochondrial dysfunction. METHODS: Cases were identified by search of the surgical pathology archival files from 1984 through 2000. Corresponding medical records were reviewed. Tumor mitochondrial DNA (mtDNA) was evaluated for mtDNA mutations within the control region (C-tract). RESULTS: Twenty-one parotid oncocytic neoplasms were identified. Oncocytoma was the most frequent morphology (62%), followed by oncocytosis (28.5%) and oncocytic carcinoma (9.5%). One specimen displayed synchronous oncocytic morphologies (oncocytoma, oncocytosis, and oncocytic metaplasia). One oncocytoma specimen displayed the mtDNA C-tract alteration. CONCLUSIONS: Oncocytic neoplasia of the parotid gland is a rare form of salivary gland disease with obscure etiology. The presence of multiple oncocytic morphologies in a single specimen is suggestive of transition between forms. Although oncocytic tumorigenesis secondary to acquired mitochondrial dysfunction is a plausible mechanism, few of these tumors actually harbor mtDNA alterations within the control region.