Assessment of Technical Skills in Axillary Lymph Node Dissection.

OBJECTIVE: A simulator to enable safe practice and assessment of ALND has been designed, and face, content and construct validity has been investigated. SUMMARY AND BACKGROUND DATA: The reduction in the number of ALNDs conducted has led to decreased resident exposure and confidence. METHODS: A cross-sectional multicenter observational study was carried out between July 2017 and August 2018. Following model development, 30 surgeons of varying experience (n = "experts," n = 11 "senior residents," and n = 10 "junior residents") were asked to perform a simulated ALND. Face and content validity questionnaires were administered immediately after ALND. All ALND procedures were retrospectively assessed by 2 attending breast surgeons, blinded to operator identity, using a video-based assessment tool, and an end product assessment tool. RESULTS: Statistically significant differences between groups were observed across all operative subphases on the axillary clearance assessment tool (P < 0.001). Significant differences between groups were observed for overall procedure quality (P < 0.05) and total number of lymph nodes harvested (P < 0.001). However, operator grade could not be distinguished across other end product variables such as axillary vein damage (P = 0.864) and long thoracic nerve injury (P = 0.094). Overall, participants indicated that the simulator has good anatomical (median score >7) and procedural realism (median score >7). CONCLUSIONS: Video-based analysis demonstrates construct validity for ALND assessment. Given reduced ALND exposure, this simulation is a useful adjunct for both technical skills training and formative Deanery or Faculty administered assessments.

Surgical Quality Assurance in COLOR III: Standardization and Competency Assessment in a Randomized Controlled Trial.

OBJECTIVE: The aim of this study was to develop an objective and reliable surgical quality assurance system (SQA) for COLOR III, an international multicenter randomized controlled trial (RCT) comparing transanal total mesorectal excision (TaTME) with laparoscopic approach for rectal cancer. BACKGROUND OF SUMMARY DATA: SQA influences outcome measures in RCTs such as lymph nodes harvest, in-hospital mortality, and locoregional cancer recurrence. However, levels of SQA are variable. METHOD: Hierarchical task analysis of TaTME was performed. A 4-round Delphi methodology was applied for standardization of TaTME steps. Semistructured interviews were conducted in round 1 to identify key steps and tasks, which were rated as mandatory, optional, or prohibited in rounds 2 to 4 using questionnaires. Competency assessment tool (CAT) was developed and its content validity was examined by expert surgeons. Twenty unedited videos were assessed to test reliability using generalizability theory. RESULTS: Eighty-three of 101 surgical tasks identified reached 70% agreement (26 mandatory, 56 optional, and 1 prohibited). An operative guide of standardized TaTME was created. CAT is matrix of 9 steps and 4 performance qualities: exposure, execution, adverse event, and end-product. The overall G-coefficient was 0.883. Inter-rater and interitem reliability were 0.883 and 0.986. To enter COLOR III, 2 unedited TaTME and 1 laparoscopic TME videos were submitted and assessed by 2 independent assessors using CAT. CONCLUSION: We described an iterative approach to develop an objective SQA within multicenter RCT. This approach provided standardization, the development of reliable and valid CAT, and the criteria for trial entry and monitoring surgical performance during the trial.

Simulating various levels of clinical challenge in the assessment of clinical procedure competence.

STUDY OBJECTIVE: Immersive simulation is increasingly used for competency assessment of emergency physicians. This group's concept of hybrid simulation (HS) (combining simulated patients and part-task trainers (a simulator that simulates a limited component of a clinical procedure) to create a multimodal clinical context) requires clinicians to integrate technical and nontechnical skills in a holistic clinical performance for assessment. It also offers the potential to provide authentic simulation of a given clinical procedure across multiple levels of challenge. The aims of this study are to systematically design and validate 2 patient-focused HS scenarios (each combining a simulated patient with a part-task simulator) for assessment of the management of a commonly encountered problem in an emergency department (ED) at different levels of clinical challenge, and to explore the effect of level of challenge of the HS scenario on physicians' performance. METHODS: A simple (HS1) and a complex (HS2) HS scenario (based on the management of a patient with a traumatic skin laceration within the ED) was developed according to expert opinion through cognitive task analysis. Interns and emergency medicine residents (stratified into expert and novice groups according to experience) were recruited to participate in both scenarios. Participants were randomized to perform either the HS1 or HS2 scenario first. Participants completed a questionnaire for face validity (realism of simulation) and content validity (comprehensiveness of simulation). Performances were assessed by 2 independent raters using validated rating tools modified to the needs of this study: the Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, the Objective Structured Assessment of Technical Skills-Global Rating Score, and the Direct Observation of Procedural Skills. RESULTS: Ten novice and 10 expert clinicians completed both scenarios. Mean face and content validity ratings were high for both HS1 (mean 4.4 [SD 0.52] and 4.2 [SD 0.48], respectively) and HS2 scenarios (mean 4.5 [SD 0.35] and 4.3 [SD 0.43], respectively). In HS1, no difference was found between experts' and novices' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, Objective Structured Assessment of Technical Skills-Global Rating Score, and Direct Observation of Procedural Skills ratings. Experts performed significantly better than novices in HS2 in terms of the 3 tools' ratings. Novices' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist and Direct Observation of Procedural Skills ratings were significantly worse in HS2 compared with HS1, but no difference was found with the Objective Structured Assessment of Technical Skills-Global Rating Score. No statistical difference was found in experts' Modified Objective Structured Assessment of Technical Skills-Task Specific Checklist, Objective Structured Assessment of Technical Skills-Global Rating Score, and Direct Observation of Procedural Skills ratings between HS2 and HS1 scenarios. CONCLUSION: Recreating clinical challenge is an important consideration in the design of simulation-based assessment of procedural skills of clinicians. In this study, we have demonstrated a systematic approach to developing HS scenarios, which may be able to recreate various levels of clinical challenge for purpose of assessment of procedural skills.

Cognitive ability, neighborhood deprivation, and young children's emotional and behavioral problems.

PURPOSE: To examine if cognitive ability moderates the effect of area (neighborhood) deprivation on young children's problem behavior. METHODS: Data from the first two sweeps of the Millennium Cohort Study (MCS) in the UK were used. Children were clustered in small areas in nine strata in the UK and were aged 9 months at Sweep 1 and 3 years at Sweep 2. Neighborhood deprivation was measured with the Index of Multiple Deprivation at Sweep 1. Overall and specific problem behavior was measured with the Strengths and Difficulties Questionnaire at Sweep 2. To explore moderator specificity we used three indices of ability (verbal cognitive ability, non-verbal cognitive ability, and attainment of developmental milestones). Adjustment was made for child's age and sex, and for Sweep 1 family adversity (number of adverse life events), family structure, mother's social class and psychological distress, and family socio-economic disadvantage. RESULTS: We found both support for our main hypothesis, and evidence for specificity. Neighborhood deprivation was, even after adjustment for covariates, significantly associated with children's peer problems. However, verbal and non-verbal cognitive ability moderated this association. CONCLUSIONS: Neighborhood deprivation was related to peer problems even at preschool age. Although the effect of neighborhood deprivation on externalizing problems was mediated by family poverty and parental socio-economic position and although its effect on internalizing problems was mediated by parental mental health, its effect on difficulties with peers was independent of both parental and child characteristics. Cognitive ability moderated the effect of neighborhood deprivation on preschoolers' peer relationships difficulties.

Modeling risks: effects of area deprivation, family socio-economic disadvantage and adverse life events on young children's psychopathology.

BACKGROUND: The effects of contextual risk on young children's behavior are not appropriately modeled. AIMS: To model the effects of area and family contextual risk on young children's psychopathology. METHOD: The final study sample consisted of 4,618 Millennium Cohort Study (MCS) children, who were 3 years old, clustered in lower layer super output areas in nine strata in the UK. Contextual risk was measured by socio-economic disadvantage (SED) at both area and family level, and by distal and proximal adverse life events at family level. Multivariate response multilevel models that allowed for correlated residuals at both individual and area level, and univariate multilevel models estimated the effect of contextual risk on specific and broad psychopathology measured by the Strengths and Difficulties Questionnaire. RESULTS: The area SED/broad psychopathology association remained significant after family SED was controlled, but not after maternal qualifications and family adverse life events were added to the model. Adverse life events predicted psychopathology in all models. Family SED did not predict emotional symptoms or hyperactivity after child characteristics were added to the model with the family-level controls. CONCLUSIONS: Area-level SED predicts child psychopathology via family characteristics; family-level SED predicts psychopathology largely by its impact on development; and adverse life events predict psychopathology independently of earlier adversity, SED and child characteristics, as well as maternal psychopathology, parenting and education.