Lack of standardization and faculty development in pediatric colonoscopy: A qualitative study.

A standard curriculum for pediatric colonoscopy training has neither been required nor universally implemented in North American fellowship programs. This qualitative study assessed the needs of colonoscopy training in pediatric gastroenterology to determine the standardized components of procedural teaching. Focus groups with pediatric gastroenterology attendings, fellows, procedural nurses, and interviews with advanced endoscopists, all practicing at a single institution, were conducted between March and June 2018. Data were analyzed using thematic analysis principles. Four themes emerged: (1) lack of standardization of colonoscopy performance, (2) lack of professional development of procedure teaching skills, (3) need for teaching behaviors that promote learner's performance, and (4) barriers to effective teaching and learning. A conceptual framework was created for developing a standardized "train-the-trainer" curriculum. Our needs assessment supports expansion of efforts to make this comprehensive training available to all pediatric gastroenterologists involved in procedure teaching.

Defining Practice Readiness in Pediatric Surgery Fellowship: A Qualitative Analysis.

INTRODUCTION: While competency-based training is at the forefront of educational innovation in General Surgery, Pediatric Surgery training programs should not wait for downstream changes. There is currently no consensus on what it means for a pediatric surgery fellow to be "practice-ready". In this study, we aimed to provide a framework for better defining competency and practice readiness in a way that can support the Milestones system and allow for improved assessment of pediatric surgery fellows. METHODS: For this exploratory qualitative study, we developed an interview guide with nine questions focused on how faculty recognize competency and advance autonomy among pediatric surgery fellows. Demographic information was collected using an anonymous online survey platform. We iteratively reviewed data from each interview to ensure adequate information power was achieved to answer the research question. We used inductive reasoning and thematic analysis to determine appropriate codes. Additionally, the Dreyfus model was used as a framework to guide interpretation and contextualize the responses. Through this method, we generated common themes. RESULTS: A total of 19 pediatric surgeons were interviewed. We identified four major themes from 127 codes that practicing pediatric surgeons associate with practice-readiness of a fellow: skill-based competency, the recognition and benefits of struggle, developing expertise and facilitating autonomy, and difficulties in variability of evaluation. While variability in evaluation is not typically included in the concept of practice readiness, assessment and evaluation were described by study participants as essential aspects of how practicing pediatric surgeons perceive practice readiness and competency in pediatric surgery fellows. Competency was further divided into interpersonal versus technical skills. Sub-themes within struggle included personal and professional struggle, benefits of struggle and how to identify and assist those who are struggling. Autonomy was commonly stated as variable based on the attending. CONCLUSION: Our analysis yielded several themes associated with practice readiness of pediatric surgery fellows. We aim to further refine our list of themes using the Dreyfus Model as our interpretive framework and establish consensus amongst the community of pediatric surgeons in order to define competency and key elements that make a fellow practice-ready. Further work will then focus on establishing assessment metrics and educational interventions directed at achieving such key elements.

Radiographic assessment of traction-induced esophageal growth and traction-related complications of the Foker process for treatment of long-gap esophageal atresia.

BACKGROUND: Radiographic assessment of esophageal growth in long-gap esophageal atresia while on traction and associated traction-related complications have not been described. OBJECTIVE: To demonstrate how chest radiography can estimate esophageal position while on traction and to evaluate radiography's utility in diagnosing certain traction system complications. MATERIALS AND METHODS: In this retrospective evaluation of portable chest radiographs obtained in infants with long-gap esophageal atresia who underwent the Foker process between 2014 and 2020, we assessed distances between the opposing trailing clips (esophageal gap) and the leading and trailing clips for each esophageal segment on serial radiographs. Growth during traction was estimated using longitudinal random-effects regression analysis to account for multiple chest radiograph measurements from the same child. RESULTS: Forty-three infants (25 male) had a median esophageal gap of 4.5 cm. Median traction time was 14 days. Median daily radiographic esophageal growth rate for both segments was 2.2 mm and median cumulative growth was 23.6 mm. Traction-related complications occurred in 13 (30%) children with median time of 8 days from traction initiation. Daily change >12% in leading-to trailing clip distance demonstrated 86% sensitivity and 92% specificity for indicating traction-related complications (area under the curve [AUC] 0.853). Cumulative change >30% in leading- to trailing-clip distance during traction was 85% sensitive and 85% specific for indicating traction complications (AUC 0.874). CONCLUSION: Portable chest radiograph measurements can serve as a quantitative surrogate for esophageal segment position in long-gap esophageal atresia. An increase of >12% between two sequential chest radiographs or >30% increase over the traction period in leading- to trailing-clip distance is highly associated with traction system complications.

Assessment of Operative Autonomy and Readiness for Independent Practice Among Pediatric Surgery Fellows.

PURPOSE: The purpose of this study was to evaluate the implementation of the System for Improving and Measuring Procedural Learning (SIMPL) "app" in a pediatric surgery training program. SIMPL is a novel workplace-based operative performance assessment platform which allows case by case assessment of trainees using three scales (autonomy, performance, and case complexity) and takes seconds to complete. METHODS: A pediatric surgery-specific procedural taxonomy for SIMPL was developed from the ACGME procedural library. SIMPL was piloted and implemented in a single pediatric surgery training program. A descriptive analysis of the operative assessments was undertaken. RESULTS: In the year following SIMPL implementation, 565 (39% of cases) assessments were completed by 21 faculty for 3 fellows for 148 unique procedures within a median of 8.5 h. Recorded feedback accompanied 61% of assessments. 2nd year fellows were more likely to be deemed autonomous (81% ratings) and practice ready (84%) vs. 1st year fellows (50%; p < 0.001 both), with improvements over time. CONCLUSION: Within pediatric surgery, this is the first implementation of a workplace-based operative performance assessment "app". With its ease of use, SIMPL drastically increased the volume of operative evaluations and diversified the mix of cases evaluated while SIMPL ratings demonstrated expected and graduated performance improvements over time. TYPE OF STUDY: Education. LEVEL OF EVIDENCE: N/A.

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure.

BACKGROUND: Outcomes of surgical trials hinge on surgeon selection and their underlying expertise. Assessment of expertise is paramount. We investigated whether surgeons' performance measured by the fundamentals of laparoscopic surgery (FLS) assessment program could predict their performance in a surgical trial. METHODS: As part of a prospective multi-institutional study of minimally invasive inguinal lymphadenectomy (MILND) for melanoma, surgical oncologists with no prior MILND experience underwent pre-trial FLS assessment. Surgeons completed MILND training, began enrolling patients, and submitted videos of each MILND case performed. Videos were scored with the global operative assessment of laparoscopic skills (GOALS) tool. Associations between baseline FLS scores and participant's trial performance metrics were assessed. RESULTS: Twelve surgeons enrolled patients; their median total baseline FLS score was 332 (range 275-380, max possible 500, passing >270). Participants enrolled 87 patients in the study (median 6 per surgeon, range 1-24), of which 72 (83%) videos were adequate for scoring. Baseline GOALS score was 17.1 (range 9.6-21.2, max possible score 30). Inter-rater reliability was excellent (ICC = 0.85). FLS scores correlated with improved GOALS scores (r = 0.57, p = 0.05) and with decreased operative time (r = -0.6, p = 0.02). No associations were found with the degree of patient recruitment (r = 0.02, p = 0.7), lymph node count (r = 0.01, p = 0.07), conversion rate (r = -0.06, p = 0.38) or major complications(r = -0.14, p = 0.6). CONCLUSIONS: FLS skill assessment of surgeons prior to their enrollment in a surgical trial is feasible. Although better FLS scores predicted improved operative performance and operative time, other trial outcome measures showed no difference. Our findings have implications for the documentation of laparoscopic expertise of surgeons in practice and may allow more appropriate selection of surgeons to participate in clinical trials.

Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

BACKGROUND: The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. METHODS: We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). RESULTS: From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. CONCLUSION: Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

Optimizing training cost-effectiveness of simulation-based laparoscopic inguinal hernia repairs.

BACKGROUND: Motor learning theory suggests that highly complex tasks are probably best trained under conditions of part task (PT), as opposed to whole-task (WT) training. Within PT, random practice of tasks has been shown to lead to improved skill retention and transfer. METHODS: General surgery residents were equally randomized to PT vs WT, mastery learning type, and simulation-based training of laparoscopic inguinal hernia repair. Training time and resources used to reach mastery (skill acquisition), performance at 1-month testing (skill retention), and intraoperative time and performance scores (skill transfer) were compared. RESULTS: Forty-four general surgery trainees were randomized. All residents achieved mastery benchmarks. Trainees in the PT group achieved mastery on average 17 minutes faster (60.2 ± 23.8 vs 77.1 ± 24.8 minutes, P = .02, saving 6.2 instructor hours), used fewer material resources (curricular cost savings of $2,380 or $121 per learner), and were more likely to retain mastery level performance at 1-month retention testing (59% vs 22.7% P = .03). No differences in intraoperative performance were encountered. CONCLUSIONS: For laparoscopic inguinal hernia repair, random PT simulation-based training seems to be more cost-effective, compared with WT training.

What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment.

Ongoing transformations in health professions education underscore the need for valid and reliable assessment. The current standard for assessment validation requires evidence from five sources: content, response process, internal structure, relations with other variables, and consequences. However, researchers remain uncertain regarding the types of data that contribute to each evidence source. We sought to enumerate the validity evidence sources and supporting data elements for assessments using technology-enhanced simulation. We conducted a systematic literature search including MEDLINE, ERIC, and Scopus through May 2011. We included original research that evaluated the validity of simulation-based assessment scores using two or more evidence sources. Working in duplicate, we abstracted information on the prevalence of each evidence source and the underlying data elements. Among 217 eligible studies only six (3 %) referenced the five-source framework, and 51 (24 %) made no reference to any validity framework. The most common evidence sources and data elements were: relations with other variables (94 % of studies; reported most often as variation in simulator scores across training levels), internal structure (76 %; supported by reliability data or item analysis), and content (63 %; reported as expert panels or modification of existing instruments). Evidence of response process and consequences were each present in <10 % of studies. We conclude that relations with training level appear to be overrepresented in this field, while evidence of consequences and response process are infrequently reported. Validation science will be improved as educators use established frameworks to collect and interpret evidence from the full spectrum of possible sources and elements.

Cost: the missing outcome in simulation-based medical education research: a systematic review.

BACKGROUND: The costs involved with technology-enhanced simulation remain unknown. Appraising the value of simulation-based medical education (SBME) requires complete accounting and reporting of cost. We sought to summarize the quantity and quality of studies that contain an economic analysis of SBME for the training of health professions learners. METHODS: We performed a systematic search of MEDLINE, EMBASE, CINAHL, ERIC, PsychINFO, Scopus, key journals, and previous review bibliographies through May 2011. Articles reporting original research in any language evaluating the cost of simulation, in comparison with nonstimulation instruction or another simulation intervention, for training practicing and student physicians, nurses, and other health professionals were selected. Reviewers working in duplicate evaluated study quality and abstracted information on learners, instructional design, cost elements, and outcomes. RESULTS: From a pool of 10,903 articles we identified 967 comparative studies. Of these, 59 studies (6.1%) reported any cost elements and 15 (1.6%) provided information on cost compared with another instructional approach. We identified 11 cost components reported, most often the cost of the simulator (n = 42 studies; 71%) and training materials (n = 21; 36%). Ten potential cost components were never reported. The median number of cost components reported per study was 2 (range, 1-9). Only 12 studies (20%) reported cost in the Results section; most reported it in the Discussion (n = 34; 58%). CONCLUSION: Cost reporting in SBME research is infrequent and incomplete. We propose a comprehensive model for accounting and reporting costs in SBME.

Cost-effectiveness of contralateral prophylactic mastectomy versus routine surveillance in patients with unilateral breast cancer.

PURPOSE: Contralateral prophylactic mastectomy (CPM) rates in women with unilateral breast cancer are increasing despite controversy regarding survival advantage. Current scrutiny of the medical costs led us to evaluate the cost-effectiveness of CPM versus routine surveillance as an alternative contralateral breast cancer (CBC) risk management strategy. METHODS: Using a Markov model, we simulated patients with breast cancer from mastectomy to death. Model parameters were gathered from published literature or national databases. Base-case analysis focused on patients with average-risk breast cancer, 45 years of age at treatment. Outcomes were valued in quality-adjusted life-years (QALYs). Patients' age, risk level of breast cancer, and quality of life (QOL) were varied to assess their impact on results. RESULTS: Mean costs of treatment for women age 45 years are comparable: $36,594 for the CPM and $35,182 for surveillance. CPM provides 21.22 mean QALYs compared with 20.93 for surveillance, resulting in an incremental cost-effectiveness ratio (ICER) of $4,869/QALY gained for CPM. To prevent one CBC, six CPMs would be needed. CPM is no longer cost-effective for patients older than 70 years (ICER $62,750/QALY). For BRCA-positive patients, CPM is clearly cost-effective, providing more QALYs while being less costly. In non-BRCA patients, cost-effectiveness of CPM is highly dependent on assumptions regarding QOL for CPM versus surveillance strategy. CONCLUSION: CPM is cost-effective compared with surveillance for patients with breast cancer who are younger than 70 years. Results are sensitive to BRCA-positive status and assumptions of QOL differences between CPM and surveillance patients. This highlights the importance of tailoring treatment for individual patients.

Predicting four or more metastatic axillary lymph nodes in patients with sentinel node-positive breast cancer: assessment of existent risk scores.

BACKGROUND: Patients with metastases in four or more axillary lymph nodes (≥4+ALN) represent a subset of patients with breast cancer who are at increased risk of local recurrence and who benefit from postmastectomy radiation. Risk prediction models designed to identify such patients have been published by Rivers et al., Chagpar et al., and Katz et al. We sought to evaluate and compare the performance of these models in an independent patient population. METHODS: We reviewed 454 patients with breast cancer with one to three positive sentinel lymph nodes who underwent completion axillary lymph node dissection at our institution. Each of the three published models was applied to our sample as described in the respective publications. The models' performances were analyzed with the Hosmer-Lemeshow goodness-of-fit test and with the area under the curve (AUC). Sensitivity, specificity, and false-negative percentages were calculated for clinically meaningful cutoff points of each score. RESULTS: Of 454 eligible patients, 87 (19.2%) had four or more positive axillary nodes. The Rivers, Chagpar, and Katz models demonstrated good calibration in our population based on the Hosmer-Lemeshow test (p = 0.82, p = 0.73, p = 0.71, respectively). Assessment of discriminatory ability for the models resulted in AUCs of 0.81, 0.73, and 0.81, respectively. CONCLUSIONS: The Rivers and Katz models performed well in our patient population and may be clinically useful to predict patients with ≥4+ALN. However, their clinical utility is limited by the current controversy surrounding the use of postmastectomy radiation for all node-positive patients.