Rescuing the Clinical Breast Examination: Advances in Classifying Technique and Assessing Physician Competency.

OBJECTIVE: Develop new performance evaluation standards for the clinical breast examination (CBE). SUMMARY BACKGROUND DATA: There are several, technical aspects of a proper CBE. Our recent work discovered a significant, linear relationship between palpation force and CBE accuracy. This article investigates the relationship between other technical aspects of the CBE and accuracy. METHODS: This performance assessment study involved data collection from physicians (n = 553) attending 3 different clinical meetings between 2013 and 2014: American Society of Breast Surgeons, American Academy of Family Physicians, and American College of Obstetricians and Gynecologists. Four, previously validated, sensor-enabled breast models were used for clinical skills assessment. Models A and B had solitary, superficial, 2 cm and 1 cm soft masses, respectively. Models C and D had solitary, deep, 2 cm hard and moderately firm masses, respectively. Finger movements (search technique) from 1137 CBE video recordings were independently classified by 2 observers. Final classifications were compared with CBE accuracy. RESULTS: Accuracy rates were model A = 99.6%, model B = 89.7%, model C = 75%, and model D = 60%. Final classification categories for search technique included rubbing movement, vertical movement, piano fingers, and other. Interrater reliability was (k = 0.79). Rubbing movement was 4 times more likely to yield an accurate assessment (odds ratio 3.81, P < 0.001) compared with vertical movement and piano fingers. Piano fingers had the highest failure rate (36.5%). Regression analysis of search pattern, search technique, palpation force, examination time, and 6 demographic variables, revealed that search technique independently and significantly affected CBE accuracy (P < 0.001). CONCLUSIONS: Our results support measurement and classification of CBE techniques and provide the foundation for a new paradigm in teaching and assessing hands-on clinical skills. The newly described piano fingers palpation technique was noted to have unusually high failure rates. Medical educators should be aware of the potential differences in effectiveness for various CBE techniques.

Decision making in trauma settings: simulation to improve diagnostic skills.

INTRODUCTION: In the setting of acute injury, a wrong, missed, or delayed diagnosis can impact survival. Clinicians rely on pattern recognition and heuristics to rapidly assess injuries, but an overreliance on these approaches can result in a diagnostic error. Simulation has been advocated as a method for practitioners to learn how to recognize the limitations of heuristics and develop better diagnostic skills. The objective of this study was to determine whether simulation could be used to provide teams the experiences in managing scenarios that require the use of heuristic as well as analytic diagnostic skills to effectively recognize and treat potentially life-threatening injuries. METHODS: Ten scenarios were developed to assess the ability of trauma teams to provide initial care to a severely injured patient. Seven standard scenarios simulated severe injuries that once diagnosed could be effectively treated using standard Advanced Trauma Life Support algorithms. Because diagnostic error occurs more commonly in complex clinical settings, 3 complex scenarios required teams to use more advanced diagnostic skills to uncover a coexisting condition and treat the patient. Teams composed of 3 to 5 practitioners were evaluated in the performance of 7 (of 10) randomly selected scenarios (5 standard, 2 complex). Expert rates scored teams using standardized checklists and global scores. RESULTS: Eighty-three surgery, emergency medicine, and anesthesia residents constituted 21 teams. Expert raters were able to reliably score the scenarios. Teams accomplished fewer checklist actions and received lower global scores on the 3 analytic scenarios (73.8% [12.3%] and 5.9 [1.6], respectively) compared with the 7 heuristic scenarios (83.2% [11.7%] and 6.6 [1.3], respectively; P < 0.05 for both). Teams led by more junior residents received higher global scores on the analytic scenarios (6.4 [1.3]) than the more senior team leaders (5.3 [1.7]). CONCLUSIONS: This preliminary study indicates that teams led by more senior residents received higher scores when managing heuristic scenarios but were less effective when managing the scenarios that require a more analytic approach. Simulation can be used to provide teams with decision-making experiences in trauma settings and could be used to improve diagnostic skills as well as study the decision-making process.

International variation in performance by clinical discipline and task on the United States medical licensing examination step 2 clinical knowledge component.

PURPOSE: To investigate country-to-country variation in performance across clinical science disciplines and tasks for examinees taking the Step 2 Clinical Knowledge (CK) component of the United States Medical Licensing Examination. METHOD: In 2012 the authors analyzed demographic characteristics, total scores, and percent-correct clinical science discipline and task scores for more than 88,500 examinees taking Step 2 CK for the first time during the 2008-2010 academic years. For each examinee and score, differences between the score and the mean performance of examinees at U.S. MD-granting medical schools were calculated, and mean differences by country of medical school were tabulated for analysis of country-to-country variation in performance by clinical discipline and task. RESULTS: Controlling for overall performance relative to U.S. examinees, results showed that international medical graduates (IMGs) performed best in Surgery and worst in Psychiatry for clinical discipline scores; for clinical tasks, IMGs performed best in Understanding Mechanisms of Disease and worst in Promoting Preventive Medicine and Health Maintenance. The pattern of results was strongest for IMGs attending schools in the Middle East and Australasia, present to a lesser degree for IMGs attending schools in Europe, and absent for IMGs attending Caribbean medical schools. CONCLUSIONS: Country-to-country differences in relative performance were present for both clinical discipline and task scores. Possible explanations include differences in learning outcomes, curriculum emphasis and clinical experience, standards of care, and culture, as well as the effects of English as a second language and relative emphasis on preparing students to take the Step 2 CK exam.

Simulation-based assessment to identify critical gaps in safe anesthesia resident performance.

BACKGROUND: Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty. METHODS: An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys. RESULTS: Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice. CONCLUSION: The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

Outcomes of cardiac surgery: associations with physician characteristics, institutional characteristics, and transfers of care.

BACKGROUND: Although there are several studies of the human and system factors that influence the outcomes of cardiac surgery, it remains difficult to draw conclusions because many do not simultaneously adjust for the characteristics of patients, physicians, and institutions. The current study explores the associations between these factors and inhospital mortality, with a particular focus on whether patients had the same operating and attending physician. METHOD AND RESULTS: This is a retrospective observational study of 114,751 hospitalizations from 2003 to 2009 in Pennsylvania that included a coronary artery bypass graft, valve surgery, or both. The study included 70 teaching and nonteaching hospitals, 289 operating physicians who were also the attending physicians for 75% of the hospitalizations, and 2654 attending physicians for the remaining hospitalizations. After adjustment, there was a 38.4% decrease (95% CI, 20.3%-56.5%) in mortality when the operating and attending physician were the same. For the operator, each procedure performed was associated with a 0.05% (95% CI, 0.04%-0.06%) decrease in mortality and each year since medical school was associated with a 0.9% (95% CI, 0.02%-1.8%) increase in mortality. For the attending, each year since medical school was associated with a 0.67% (95% CI, 0.01%-1.4%) decrease in patient mortality. CONCLUSIONS: The findings indicated that an increase in the log odds of mortality was associated with the transfer of care between an attending and operating physician. Better patient outcomes were associated with an operator with higher volume who was closer to medical school graduation and an attending physician with more clinical experience.

Acute care of pediatric patients with sickle cell disease: a simulation performance assessment.

BACKGROUND: Sickle cell disease (SCD) is a rare disorder with cardinal features including hospitalization for vaso-occlusive pain episodes, acute pulmonary injury, and increased infection rates. For physician-trainees, learning optimal SCD management is challenging because of limited exposure to life threatening complications requiring timely interventions. PROCEDURE: To create, demonstrate reliability, and validate simulation-based, acute care SCD scenarios for physician-trainees, seven scenarios were derived from SCD patient cases. For each scenario, participants had 5 minutes to complete diagnostic and treatment interventions. Participants were divided into two groups based on clinical experience: interns or residents/fellows. Two raters scored performances using diagnostic and therapeutic checklists--indicating whether specific actions were performed and a global, 1 (poor) to 9 (excellent), rating. Scenario scores were calculated by averaging rater scores on each metric. Reliability was defined through uniformity in rater scoring and consistency of participant performance over scenarios. Validity was demonstrated by the performance gradient where the more experienced trainees outperform those early in training. RESULTS: Twenty-eight pediatric residents and hematology fellows took part in the study. Reliability for assessment scores overall was moderate. Performance on all but one scenario was moderately predictive of overall performance. Senior resident/fellows performed significantly better than interns. Positive associations existed between overall performance scores (P < 0.01) and months of postgraduate training (P < 0.01). CONCLUSIONS: Mannequin-based simulation is a novel method for teaching pediatric residents SCD-specific acute care skills. The assessment provided reliable and valid measures of trainees' performance. Further studies are needed to determine simulation's utility in education and evaluation.

Evaluating the management of septic shock using patient simulation.

OBJECTIVE: Develop a scoring system that can assess the management of septic shock by individuals and teams. DESIGN: Retrospective review of videotapes of critical care house staff managing a standardized simulation of septic shock. SETTING: Academic medical center; videotapes were made in a recreated intensive care unit environment using a high-fidelity patient simulator. SUBJECTS: Residents in medicine, surgery, and anesthesiology who had participated in the intensive care unit rotation. INTERVENTIONS: The septic patient was managed by the intensive care unit team in a graded manner with interns present for the first 10 mins and more senior-level help arriving after 10 mins. The intern was graded separately for the first 10 mins, and the team was graded for the entire 35-min performance. MEASUREMENTS AND MAIN RESULTS: Both technical and nontechnical scoring systems were developed to rate the management of septic shock. Technical scores are based on guidelines and principles of managing septic shock. Team leadership, communication, contingency planning, and resource utilization were addressed by the nontechnical rating. Technical scores were calculated for both interns and teams; nontechnical scores applied only to the team. Of 16 technical checklist items, interns completed a mean of 7 with a range of 1.5-11. Team technical ratings had a mean of 9.3 with a range of 3.3-13. Nontechnical scores showed similar intergroup variability with a mean of 26 and a range of 10-35. Technical and nontechnical scores showed a modest correlation (r = .40, p = .05). Interrater reliabilities for intern and team technical scores were both r = .96 and for nontechnical scores r = .88. CONCLUSIONS: Objective measures of both knowledge-based and behavioral skills pertinent to the management of septic shock were made. Scores identified both adequate and poor levels of performance. Such assessments can be used to benchmark clinical skills of individuals and groups over time and may allow the identification of interventions that improve clinical effectiveness in sepsis management.