RESUMEN
OBJECTIVE: To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting. BACKGROUND: To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown. METHODS: Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. RESULTS: Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2000; 7 of these 8 articles (87.5%) had below-average (<0.83) area under the receiver operating characteristic or accuracy. Overall, 29 articles (80.6%) performed internal validation only, 5 (13.8%) performed external validation, and 2 (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy. CONCLUSIONS: Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.
Asunto(s)
Inteligencia Artificial , Humanos , Curva ROCRESUMEN
OBJECTIVE: To examine the alignment between graduating surgical trainee operative performance and a prior survey of surgical program director expectations. BACKGROUND: Surgical trainee operative training is expected to prepare residents to independently perform clinically important surgical procedures. METHODS: We conducted a cross-sectional observational study of US general surgery residents' rated operative performance for Core general surgery procedures. Residents' expected performance on those procedures at the time of graduation was compared to the current list of Core general surgery procedures ranked by their importance for clinical practice, as assessed via a previous national survey of general surgery program directors. We also examined the frequency of individual procedures logged by residents over the course of their training. RESULTS: Operative performance ratings for 29,885 procedures performed by 1861 surgical residents in 54 general surgery programs were analyzed. For each Core general surgery procedure, adjusted mean probability of a graduating resident being deemed practice-ready ranged from 0.59 to 0.99 (mean 0.90, standard deviation 0.08). There was weak correlation between the readiness of trainees to independently perform a procedure at the time of graduation and that procedure's historical importance to clinical practice ( p = 0.22, 95% confidence interval 0.01-0.41, P = 0.06). Residents also continue to have limited opportunities to learn many procedures that are important for clinical practice. CONCLUSION: The operative performance of graduating general surgery residents may not be well aligned with surgical program director expectations.
Asunto(s)
Cirugía General , Internado y Residencia , Humanos , Competencia Clínica , Estudios Transversales , Motivación , Encuestas y Cuestionarios , Cirugía General/educación , Educación de Postgrado en MedicinaRESUMEN
BACKGROUND: Patients seeking second opinions are a challenge for the colorectal cancer provider because of complexity, failed therapeutic relationship with another provider, need for reassurance, and desire for exploration of treatment options. OBJECTIVE: The purpose of this study was to describe the patient and treatment characteristics of patients seeking initial and second opinions in colorectal cancer care at a multidisciplinary colorectal cancer clinic. DESIGN: This was a retrospective cohort study. SETTINGS: A prospectively collected clinical registry of a multidisciplinary colorectal cancer clinic was included. PATIENTS: The study included patients with colon or rectal cancer seen from 2012 to 2017. MAIN OUTCOME MEASURES: Data were analyzed for initial versus second opinion and demographic and clinical characteristics. RESULTS: Of 1711 patients with colorectal cancer, 1008 (58.9%) sought an initial opinion and 700 (40.9%) sought a second opinion. As compared with initial-opinion patients, second-opinion patients were more likely to have stage IV disease (OR = 1.94 (95% CI, 1.47-2.58)), recurrent disease (OR = 1.67 (95% CI, 1.13-2.46)), and be ages 40 to 49 years (OR = 1.47 (95% CI, 1.02-2.12)). Initial- and second-opinion cohorts were similar in terms of sex, race, and proportion of colon versus rectal cancer. Among second-opinion patients, 246 (35%) transitioned their care to the multidisciplinary colorectal cancer clinic. LIMITATIONS: We were unable to capture the final treatment plan for those patients who did not transfer care to the multidisciplinary colorectal cancer clinic. CONCLUSIONS: Patients seeking a second opinion represent a unique subset of patients with colorectal cancer. In general, they are younger and more likely to have stage IV or recurrent disease than patients seeking an initial opinion. Although transfer of care to a multidisciplinary colorectal cancer clinic after second opinion is lower than for initial consultations, multidisciplinary colorectal cancer clinics provide an important role for patients with complex disease characteristics and treatment needs. See Video Abstract at http://links.lww.com/DCR/B192. CARACTERíSTICAS DE LOS PACIENTES QUE BUSCAN UNA SEGUNDA OPINIóN EN CLíNICAS MULTIDISCIPLINARIAS ESPECIALIZADAS EN CáNCER COLORECTAL: Los pacientes que buscan una segunda opinión son un desafío para el médico que trata el cáncer colorrectal debido a la complejidad de la situación, a la relación terapéutica fallida con otro especialista, a la necesidad de tranquilidad y el deseo de explorar otras opciones del tratamiento.El describir las características y el tratamiento de los pacientes que buscan opiniones iniciales y secundarias en la atención del cáncer colorrectal en una clínica especializada de manera multidisciplinaria en cáncer colorrectal.Este es un estudio de cohortes retrospectivo.Registro clínico de casos obtenidos prospectivamente en una clínica especializada de manera multidisciplinaria en cáncer colorrectal.Todos aquellos pacientes con cáncer de colon o recto examinados entre 2012-2017.Se analizaron los datos obtenidos en la opinión inicial y se compararon con la segunda opinión, se revisaron tanto sus características demográficas como clínicas.De 1711 pacientes con cáncer colorrectal, 1008 (58.9%) buscaron una opinión inicial, 700 (40.9%) buscaron una segunda opinión. En comparación con los pacientes de opinión inicial, los pacientes de segunda opinión presentaron más probabilidades de tener enfermedad en estadio IV (OR 1.94, IC 95% 1.47-2.58), enfermedad recurrente (OR 1.67, IC 95% 1.13-2.46) y tener edades entre 40 y 49 (O 1.47, IC 95% 1.02-2.12). Las cohortes iniciales y de segunda opinión fueron similares en términos de género, raza y proporción del cáncer de colon versus cáncer de recto. Entre los pacientes de segunda opinión, 246 (35%) transfirieron su tratamiento hacia una clínica multidisplinaria especializada en cáncer colorrectal.No se obtuvieron los planes del tratamiento final de aquellos pacientes que no transfirieron sus cuidados hacia una la clínica especializada en cáncer colorrectal.Los pacientes que buscan una segunda opinión representan un subconjunto único de personas con cáncer colorrectal. En general, son más jóvenes y tienen más probabilidades de tener enfermedad en estadio IV o recurrente, con relación a aquellos pacientes que buscan una opinión inicial. Aunque la transferencia de los cuidados hacia una clínica multidisciplinaria especializada en cáncer colorrectal después de una segunda opinión es menor que para las consultas iniciales. Las clínicas multidisciplinarias especializadas en cáncer colorrectal juegan un papel importante con los pacientes que tienen características complejas de enfermedad y necesidades particulares en el tratamiento. Consulte Video Resumen en http://links.lww.com/DCR/B192. (Traducción-Dr Xavier Delgadillo).
Asunto(s)
Neoplasias del Colon/terapia , Transferencia de Pacientes/tendencias , Neoplasias del Recto/terapia , Derivación y Consulta/estadística & datos numéricos , Anciano , Estudios de Casos y Controles , Neoplasias del Colon/diagnóstico , Femenino , Humanos , Masculino , Persona de Mediana Edad , Estadificación de Neoplasias/estadística & datos numéricos , Evaluación de Resultado en la Atención de Salud , Grupo de Atención al Paciente/estadística & datos numéricos , Neoplasias del Recto/diagnóstico , Recurrencia , Sistema de Registros , Estudios Retrospectivos , Insuficiencia del TratamientoRESUMEN
Identification of cancer driver gene mutations is crucial for advancing cancer therapeutics. Due to the overwhelming number of passenger mutations in the human tumor genome, it is difficult to pinpoint causative driver genes. Using transposon mutagenesis in mice many laboratories have conducted forward genetic screens and identified thousands of candidate driver genes that are highly relevant to human cancer. Unfortunately, this information is difficult to access and utilize because it is scattered across multiple publications using different mouse genome builds and strength metrics. To improve access to these findings and facilitate meta-analyses, we developed the Candidate Cancer Gene Database (CCGD, http://ccgd-starrlab.oit.umn.edu/). The CCGD is a manually curated database containing a unified description of all identified candidate driver genes and the genomic location of transposon common insertion sites (CISs) from all currently published transposon-based screens. To demonstrate relevance to human cancer, we performed a modified gene set enrichment analysis using KEGG pathways and show that human cancer pathways are highly enriched in the database. We also used hierarchical clustering to identify pathways enriched in blood cancers compared to solid cancers. The CCGD is a novel resource available to scientists interested in the identification of genetic drivers of cancer.
Asunto(s)
Bases de Datos Genéticas , Genes Relacionados con las Neoplasias , Animales , Técnicas Genéticas , Humanos , Internet , Secuencias Repetitivas Esparcidas , RatonesRESUMEN
Using clustering analysis for early vital signs, unique patient phenotypes with distinct pathophysiological signatures and clinical outcomes may be revealed and support early clinical decision-making. Phenotyping using early vital signs has proven challenging, as vital signs are typically sampled sporadically. We proposed a novel, deep temporal interpolation and clustering network to simultaneously extract latent representations from irregularly sampled vital signs and derive phenotypes. Four distinct clusters were identified. Phenotype A (18%) had the greatest prevalence of comorbid disease with increased prevalence of prolonged respiratory insufficiency, acute kidney injury, sepsis, and long-term (3-year) mortality. Phenotypes B (33%) and C (31%) had a diffuse pattern of mild organ dysfunction. Phenotype B's favorable short-term clinical outcomes were tempered by the second highest rate of long-term mortality. Phenotype C had favorable clinical outcomes. Phenotype D (17%) exhibited early and persistent hypotension, high incidence of early surgery, and substantial biomarker incidence of inflammation. Despite early and severe illness, phenotype D had the second lowest long-term mortality. After comparing the sequential organ failure assessment scores, the clustering results did not simply provide a recapitulation of previous acuity assessments. This tool may impact triage decisions and have significant implications for clinical decision-support under time constraints and uncertainty.
Asunto(s)
Puntuaciones en la Disfunción de Órganos , Sepsis , Humanos , Enfermedad Aguda , Fenotipo , Biomarcadores , Análisis por ConglomeradosRESUMEN
Objective: To determine whether certain patients are vulnerable to errant triage decisions immediately after major surgery and whether there are unique sociodemographic phenotypes within overtriaged and undertriaged cohorts. Background: In a fair system, overtriage of low-acuity patients to intensive care units (ICUs) and undertriage of high-acuity patients to general wards would affect all sociodemographic subgroups equally. Methods: This multicenter, longitudinal cohort study of hospital admissions immediately after major surgery compared hospital mortality and value of care (risk-adjusted mortality/total costs) across 4 cohorts: overtriage (N = 660), risk-matched overtriage controls admitted to general wards (N = 3077), undertriage (N = 2335), and risk-matched undertriage controls admitted to ICUs (N = 4774). K-means clustering identified sociodemographic phenotypes within overtriage and undertriage cohorts. Results: Compared with controls, overtriaged admissions had a predominance of male patients (56.2% vs 43.1%, P < 0.001) and commercial insurance (6.4% vs 2.5%, P < 0.001); undertriaged admissions had a predominance of Black patients (28.4% vs 24.4%, P < 0.001) and greater socioeconomic deprivation. Overtriage was associated with increased total direct costs [$16.2K ($11.4K-$23.5K) vs $14.1K ($9.1K-$20.7K), P < 0.001] and low value of care; undertriage was associated with increased hospital mortality (1.5% vs 0.7%, P = 0.002) and hospice care (2.2% vs 0.6%, P < 0.001) and low value of care. Unique sociodemographic phenotypes within both overtriage and undertriage cohorts had similar outcomes and value of care, suggesting that triage decisions, rather than patient characteristics, drive outcomes and value of care. Conclusions: Postoperative triage decisions should ensure equality across sociodemographic groups by anchoring triage decisions to objective patient acuity assessments, circumventing cognitive shortcuts and mitigating bias.
RESUMEN
Objectives: This study tests the null hypotheses that overall sentiment and gendered words in verbal feedback and resident operative autonomy relative to performance are similar for female and male residents. Background: Female and male surgical residents may experience training differently, affecting the quality of learning and graduated autonomy. Methods: A longitudinal, observational study using a Society for Improving Medical Professional Learning collaborative dataset describing resident and attending evaluations of resident operative performance and autonomy and recordings of verbal feedback from attendings from surgical procedures performed at 54 US general surgery residency training programs from 2016 to 2021. Overall sentiment, adjectives, and gendered words in verbal feedback were quantified by natural language processing. Resident operative autonomy and performance, as evaluated by attendings, were reported on 5-point ordinal scales. Performance-adjusted autonomy was calculated as autonomy minus performance. Results: The final dataset included objective assessments and dictated feedback for 2683 surgical procedures. Sentiment scores were higher for female residents (95 [interquartile range (IQR), 4-100] vs 86 [IQR 2-100]; P < 0.001). Gendered words were present in a greater proportion of dictations for female residents (29% vs 25%; P = 0.04) due to male attendings disproportionately using male-associated words in feedback for female residents (28% vs 23%; P = 0.01). Overall, attendings reported that male residents received greater performance-adjusted autonomy compared with female residents (P < 0.001). Conclusions: Sentiment and gendered words in verbal feedback and performance-adjusted operative autonomy differed for female and male general surgery residents. These findings suggest a need to ensure that trainees are given appropriate and equitable operative autonomy and feedback.
RESUMEN
BACKGROUND: Common bile duct exploration (CBDE) is safe and effective for managing choledocholithiasis, but most US general surgeons have limited experience with CBDE and are uncomfortable performing this procedure in practice. Surgical trainee exposure to CBDE is limited, and their learning curve for achieving autonomous, practice-ready performance has not been previously described. This study tests the hypothesis that receipt of one or more prior CBDE operative performance assessments, combined with formative feedback, is associated with greater resident operative performance and autonomy. METHODS: Resident and attending assessments of resident operative performance and autonomy were obtained for 189 laparoscopic or open CBDEs performed at 28 institutions. Performance and autonomy were graded along validated ordinal scales. Cases in which the resident had one or more prior CBDE case evaluations (n = 48) were compared with cases in which the resident had no prior evaluations (n = 141). RESULTS: Compared with cases in which the resident had no prior CBDE case evaluations, cases with a prior evaluation had greater proportions of practice-ready or exceptional performance ratings according to both residents (27% vs. 11%, p = .009) and attendings (58% vs. 19%, p < .001) and had greater proportions of passive help or supervision only autonomy ratings according to both residents (17% vs. 4%, p = .009) and attendings (69% vs. 32%, p < .01). CONCLUSIONS: Residents with at least one prior CBDE evaluation and formative feedback demonstrated better operative performance and received greater autonomy than residents without prior evaluations, underscoring the propensity of feedback to help residents achieve autonomous, practice-ready performance for rare operations.
Asunto(s)
Coledocolitiasis , Internado y Residencia , Laparoscopía , Humanos , Retroalimentación Formativa , Coledocolitiasis/cirugía , Conducto Colédoco/cirugíaRESUMEN
BACKGROUND: Laparoscopic common bile duct exploration is safe and effective for managing choledocholithiasis, but laparoscopic common bile duct exploration is rarely performed, which threatens surgical trainee proficiency. This study tests the hypothesis that prior operative or simulation experience with laparoscopic common bile duct exploration is associated with greater resident operative performance and autonomy without adversely affecting patient outcomes. METHODS: This longitudinal cohort study included 33 consecutive patients undergoing laparoscopic common bile duct exploration in cases involving postgraduate years 3, 4, and 5 general surgery residents at a single institution during the implementation of a laparoscopic common bile duct exploration simulation curriculum. For each of the 33 cases, resident performance and autonomy were rated by residents and attendings, the resident's prior operative and simulation experience were recorded, and patient outcomes were ascertained from electronic health records for comparison among 3 cohorts: prior operative experience, prior simulation experience, and no prior experience. RESULTS: Operative approach was similar among cohorts. Overall morbidity was 6.1% and similar across cohorts. The operative performance scores were higher in prior experience cohorts according to both residents (3.0 [2.8-3.0] vs 2.0 [2.0-3.0]; P = .01) and attendings (3.0 [3.0-4.0]; P < .001). The autonomy scores were higher in prior experience cohorts according to both residents (2.0 [2.0-3.0] vs 2.0 [2.0-2.0]; P = .005) and attendings (2.5 [2.0-3.0] vs 2.0 [1.0-2.0]; P = .001). Prior simulation and prior operative experience had similar associations with performance and autonomy. CONCLUSION: Simulation experience with laparoscopic common bile duct exploration was associated with greater resident operative performance and autonomy, with effects that mimic prior operative experience. This illustrates the potential for simulation-based training to improve resident operative performance and autonomy for laparoscopic common bile duct exploration.
Asunto(s)
Colecistectomía Laparoscópica , Coledocolitiasis , Laparoscopía , Esfinterotomía , Humanos , Quirófanos , Estudios Longitudinales , Coledocolitiasis/cirugía , Curriculum , Conducto Colédoco/cirugíaRESUMEN
BACKGROUND: Intraoperative cholangiography may allow for earlier identification of common bile duct injury and choledocholithiasis. The role of intraoperative cholangiography in decreasing resource use related to biliary pathology remains unclear. This study tests the null hypothesis that there is no difference in resource use for patients undergoing laparoscopic cholecystectomy with versus without intraoperative cholangiography. METHODS: This retrospective, longitudinal cohort study included 3,151 patients who underwent laparoscopic cholecystectomy at 3 university hospitals. To minimize differences in baseline characteristics while maintaining adequate statistical power, propensity scores were used to match 830 patients who underwent intraoperative cholangiography at surgeon discretion and 795 patients who underwent cholecystectomy without intraoperative cholangiography. Primary outcomes were the incidence of postoperative endoscopic retrograde cholangiography, the interval between surgery and endoscopic retrograde cholangiography, and total direct costs. RESULTS: In the propensity-matched analysis, the intraoperative cholangiography and no intraoperative cholangiography cohorts had similar age, comorbidities, American Society of Anesthesiologists Sequential Organ Failure Assessment scores, and total/direct bilirubin ratios. The intraoperative cholangiography cohort had a lower postoperative endoscopic retrograde cholangiography (2.4% vs 4.3%; P = .04), a shorter interval between cholecystectomy and endoscopic retrograde cholangiography (2.5 [1.0-17.8] vs 4.5 [2.0-9.5] days; P = .04), and shorter length of stay (0.3 [0.2-1.5] vs 1.4 [0.3-3.2] days; P < .001). Patients undergoing intraoperative cholangiography had lower total direct costs ($4.0K [3.6K-5.4K] vs $8.1K [4.9K-13.0K]; P < .001). There were no differences in 30-day or 1-year mortality among the cohorts. CONCLUSION: Compared with laparoscopic cholecystectomy without intraoperative cholangiography, cholecystectomy with intraoperative cholangiography was associated with decreased resource use, which was primarily attributable to decreased incidence and the earlier timing of postoperative endoscopic retrograde cholangiography.
Asunto(s)
Colangiografía , Colecistectomía Laparoscópica , Coledocolitiasis , Coledocolitiasis/diagnóstico por imagen , Coledocolitiasis/cirugía , Colecistectomía Laparoscópica/efectos adversos , Humanos , Estudios Retrospectivos , Estudios LongitudinalesRESUMEN
OBJECTIVE: Workplace-based assessment is increasingly prevalent in surgical education, especially for assessing operative skill. With current implementations, not all observed clinical performances are assessed, in part because trainees often have discretion about when they seek assessment. As a result, these samples of observed operative performances may not be representative of the full breadth of experience of surgical trainees. Therefore, analyses of these samples may be biased. We aimed to benchmark patterns of procedures logged in the SIMPL operative performance assessment system against records of trainee experience in Accreditation Council for Graduate Medical Education (ACGME) case logs. DESIGN: We analyzed SIMPL longitudinal intraoperative performance assessments from categorical trainees in US general surgery residency programs. We compared overall patterns of how procedures are logged in SIMPL and in ACGME case logs using a Pearson correlation, and we examined differences in how individual procedures are logged in each system using Fisher's exact test. RESULTS: Total procedure frequency from the SIMPL dataset was strongly correlated with total procedure frequency from ACGME case logs (râ¯=â¯0.86, 95% CI 0.80-0.90). A subset of these procedures (10 of 116 procedures) was logged more frequently in the SIMPL dataset. These 10 procedures accounted for 56% of SIMPL observations and 30% of ACGME logged cases. Case complexity was comparable for assessments initiated by residents and faculty. CONCLUSIONS: Samples of intraoperative performance ratings gathered using the SIMPL application largely resemble ACGME case logs. There is no evidence to indicate that residents preferentially select fewer complex cases for assessment.
Asunto(s)
Cirugía General , Internado y Residencia , Acreditación , Competencia Clínica , Educación de Postgrado en Medicina/métodos , Cirugía General/educación , Lugar de TrabajoRESUMEN
BACKGROUND: Many US general surgery residents are interested in global surgery, but their competence with key procedures is unknown. METHODS: Using a registry managed by the Society for Improving Medical Professional Learning (SIMPL), we extracted longitudinal operative performance ratings data for a national cohort of US general surgery residents. Operative performance at the time of graduation was estimated via a Bayesian generalized linear mixed model. RESULTS: Operative performance ratings for 12,976 procedures performed by 1584 residents in 52 general surgery programs were analyzed. These spanned 17 of 31 (55%) procedures deemed important for global surgical practice. For these procedures, the probability of a graduating resident being deemed competent to perform a procedure was 0.95 (95% confidence interval 0.86-1.00) but was less than 0.9 for 3 observed procedures. CONCLUSION: Our results highlight gaps in the preparedness of US general surgery trainees to perform procedures deemed most important for global surgery settings.
Asunto(s)
Cirugía General , Internado y Residencia , Teorema de Bayes , Competencia Clínica , Estudios de Cohortes , Educación de Postgrado en Medicina , Cirugía General/educación , HumanosRESUMEN
Esophagectomy is a high-risk operation, regardless of technique. Minimally invasive transthoracic esophagectomy could reduce length of stay and pulmonary complications compared to traditional open approaches, but the benefits of minimally invasive transhiatal esophagectomy are unclear. We performed a retrospective review of prospectively gathered data for open transhiatal esophagectomies (THEs) and transhiatal robot-assisted minimally invasive esophagectomies (TH-RAMIEs) performed at a high-volume academic center between 2013 and 2017. Multivariate logistic regression was used to calculate adjusted odds ratios (aORs) for outcomes. 465 patients met inclusion criteria (378 THE and 87 TH-RAMIE). THE patients more likely had an ASA score of 3 + (89.1% vs 77.0%, p = 0.012), whereas TH-RAMIE patients more likely had a pathologic staging of 3+ (43.7% vs. 31.2%, p = 0.026). TH-RAMIE patients were less likely to receive epidurals (aOR 0.06, 95% confidence interval [CI] 0.03-0.14, p < 0.001), but epidural use itself was not associated with differences in outcomes. TH-RAMIE patients experienced higher rates of pulmonary complications (adjusted odds ratio [OR] 1.82, 95% CI 1.03-3.22, p = 0.040), particularly pulmonary embolus (aOR 5.20, 95% CI 1.30-20.82, p = 0.020). There were no statistically significant differences in lymph node harvest, unexpected ICU admission, length of stay, in-hospital mortality, or 30-day readmission or mortality rates. The TH-RAMIE approach had higher rates of pulmonary complications. There were no statistically significant advantages to the TH-RAMIE approach. Further investigation is needed to understand the benefits of a minimally invasive approach to the open transhiatal esophagectomy.
Asunto(s)
Neoplasias Esofágicas , Procedimientos Quirúrgicos Robotizados , Robótica , Neoplasias Esofágicas/patología , Neoplasias Esofágicas/cirugía , Esofagectomía/métodos , Humanos , Ganglios Linfáticos/cirugía , Procedimientos Quirúrgicos Mínimamente Invasivos/métodos , Complicaciones Posoperatorias/etiología , Procedimientos Quirúrgicos Robotizados/métodos , Resultado del TratamientoRESUMEN
Human pathophysiology is occasionally too complex for unaided hypothetical-deductive reasoning and the isolated application of additive or linear statistical methods. Clustering algorithms use input data patterns and distributions to form groups of similar patients or diseases that share distinct properties. Although clinicians frequently perform tasks that may be enhanced by clustering, few receive formal training and clinician-centered literature in clustering is sparse. To add value to clinical care and research, optimal clustering practices require a thorough understanding of how to process and optimize data, select features, weigh strengths and weaknesses of different clustering methods, select the optimal clustering method, and apply clustering methods to solve problems. These concepts and our suggestions for implementing them are described in this narrative review of published literature. All clustering methods share the weakness of finding potential clusters even when natural clusters do not exist, underscoring the importance of applying data-driven techniques as well as clinical and statistical expertise to clustering analyses. When applied properly, patient and disease phenotype clustering can reveal obscured associations that can help clinicians understand disease pathophysiology, predict treatment response, and identify patients for clinical trial enrollment.
RESUMEN
OBJECTIVE: Surgical trainees are subject to pressure from variety of stakeholders to secure board certification from the American Board of Surgery (ABS). To meet these expectations, trainees must pass a written qualifying exam (QE) and an oral certifying exam (CE) within 7 years of completing general surgery residency. Board certification outcomes for candidates who fail either the QE or CE examination are not well characterized, but this information could help candidates, policymakers, and other stakeholders make informed decisions about how to respond to examination failure. METHODS: We retrospectively examined ABS records for all surgeons who completed general surgery residency from 2000 to 2013 and attempted general surgery board certification. RESULTS: Among 14,483 surgeons who attempted general surgery certification, 13,566 (94%) passed both the QE and CE within the 7-year certification window. Of those who did ultimately obtain certification, 97% passed the QE within 2 attempts and 97% passed the CE within 2 attempts. For those who failed either the QE or the CE twice, 67% ultimately obtained certification. CONCLUSIONS: Most surgeons who obtained ABS general surgery board certification did so within 2 attempts at each board examination. Candidates who fail either examination twice are less likely to achieve board certification.
Asunto(s)
Cirugía General , Internado y Residencia , Cirujanos , Certificación , Evaluación Educacional , Cirugía General/educación , Humanos , Estudios Retrospectivos , Consejos de Especialidades , Estados UnidosRESUMEN
OBJECTIVE: Residency program faculty participate in clinical competency committee (CCC) meetings, which are designed to evaluate residents' performance and aid in the development of individualized learning plans. In preparation for the CCC meetings, faculty members synthesize performance information from a variety of sources. Natural language processing (NLP), a form of artificial intelligence, might facilitate these complex holistic reviews. However, there is little research involving the application of this technology to resident performance assessments. With this study, we examine whether NLP can be used to estimate CCC ratings. DESIGN: We analyzed end-of-rotation assessments and CCC assessments for all surgical residents who trained at one institution between 2014 and 2018. We created models of end-of-rotation assessment ratings and text to predict dichotomized CCC assessment ratings for 16 Accreditation Council for Graduate Medical Education (ACGME) Milestones. We compared the performance of models with and without predictors derived from NLP of end-of-rotation assessment text. RESULTS: We analyzed 594 end-of-rotation assessments and 97 CCC assessments for 24 general surgery residents. The mean (standard deviation) for area under the receiver operating characteristic curve (AUC) was 0.84 (0.05) for models with only non-NLP predictors, 0.83 (0.06) for models with only NLP predictors, and 0.87 (0.05) for models with both NLP and non-NLP predictors. CONCLUSIONS: NLP can identify language correlated with specific ACGME Milestone ratings. In preparation for CCC meetings, faculty could use information automatically extracted from text to focus attention on residents who might benefit from additional support and guide the development of educational interventions.
Asunto(s)
Competencia Clínica , Internado y Residencia , Acreditación , Inteligencia Artificial , Educación de Postgrado en Medicina , Evaluación Educacional , Procesamiento de Lenguaje NaturalRESUMEN
OBJECTIVE: To validate the performance of a natural language processing (NLP) model in characterizing the quality of feedback provided to surgical trainees. DESIGN: Narrative surgical resident feedback transcripts were collected from a large academic institution and classified for quality by trained coders. 75% of classified transcripts were used to train a logistic regression NLP model and 25% were used for testing the model. The NLP model was trained by uploading classified transcripts and tested using unclassified transcripts. The model then classified those transcripts into dichotomized high- and low- quality ratings. Model performance was primarily assessed in terms of accuracy and secondary performance measures including sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). SETTING: A surgical residency program based in a large academic medical center. PARTICIPANTS: All surgical residents who received feedback via the Society for Improving Medical Professional Learning smartphone application (SIMPL, Boston, MA) in August 2019. RESULTS: The model classified the quality (high vs. low) of 2,416 narrative feedback transcripts with an accuracy of 0.83 (95% confidence interval: 0.80, 0.86), sensitivity of 0.37 (0.33, 0.45), specificity of 0.97 (0.96, 0.98), and an area under the receiver operating characteristic curve of 0.86 (0.83, 0.87). CONCLUSIONS: The NLP model classified the quality of operative performance feedback with high accuracy and specificity. NLP offers residency programs the opportunity to efficiently measure feedback quality. This information can be used for feedback improvement efforts and ultimately, the education of surgical trainees.
Asunto(s)
Internado y Residencia , Aplicaciones Móviles , Retroalimentación , Retroalimentación Formativa , Humanos , Procesamiento de Lenguaje NaturalRESUMEN
OBJECTIVE: The profession of surgery is entering a new era of "big data," where analyses of longitudinal trainee assessment data will be used to inform ongoing efforts to improve surgical education. Given the high-stakes implications of these types of analyses, researchers must define the conditions under which estimates derived from these large datasets remain valid. With this study, we determine the number of assessments of residents' performances needed to reliably assess the difficulty of "Core" surgical procedures. DESIGN: Using the SIMPL smartphone application from the Procedural Learning and Safety Collaborative, 402 attending surgeons directly observed and provided workplace-based assessments for 488 categorical residents after 5259 performances of 87 Core surgical procedures performed at 14 institutions. We used these faculty ratings to construct a linear mixed model with resident performance as the outcome variable and multiple predictors including, most significantly, the operative procedure as a random effect. We interpreted the variance in performance ratings attributable to the procedure, after controlling for other variables, as the "difficulty" of performing the procedure. We conducted a generalizability analysis and decision study to estimate the number of SIMPL performance ratings needed to reliably estimate the difficulty of a typical Core procedure. RESULTS: Twenty-four faculty ratings of resident operative performance were necessary to reliably estimate the difficulty of a typical Core surgical procedure (mean dependability coefficient 0.80, 95% confidence interval 0.73-0.87). CONCLUSIONS: At least 24 operative performance ratings are required to reliably estimate the difficulty of a typical Core surgical procedure. Future research using performance ratings to establish procedure difficulty should include adequate numbers of ratings given the high-stakes implications of those results for curriculum design and policy.
Asunto(s)
Competencia Clínica , Evaluación del Rendimiento de Empleados , Cirugía General/educación , Procedimientos Quirúrgicos Operativos/normas , Adulto , Macrodatos , Evaluación Educacional , Femenino , Humanos , Internado y Residencia , Masculino , Aplicaciones Móviles , Autonomía Profesional , Reproducibilidad de los ResultadosRESUMEN
INTRODUCTION: The Syrian refugee crisis, now in its 6th year, has displaced millions. Refugees depend on support from host nation governments and humanitarian organizations like the Syrian American Medical Society (SAMS). We describe the delivery of pediatric care during a SAMS short-term medical mission to a refugee camp in Northern Jordan. METHODS: The medical mission team encompassed dozens of specialties. Teams visited many sites, including the Zaatari refugee camp near the Syrian border. For this study, we gathered quantitative data from one physician who provided pediatric care and anecdotes from multiple SAMS physicians who provided pediatric care in Zaatari during the same time period. The physician supplying the quantitative data recorded age, diagnoses, and prescriptions for each patient. RESULTS: The physician saw an average of 69 patients per day. Many of these were children aged 0-4 years. At least one diagnosis was recorded for 73.9% of patients, and at least one prescription was recorded for 85.5% of patients. DISCUSSION: Most presenting complaints involved acute infectious illnesses, but these seemed preventable and related to refugees' living situations. Mental health assessment was difficult. Referrals proved important for evaluation and management of both acute and chronic conditions. For the short term, we emphasize the importance of effective liaison with refugee camp authorities and outside health-care organizations. For the long term, we recommend increased health-care infrastructure development and more emphasis on preventative care. CONCLUSION: With this study, we provide new quantitative and qualitative insights into pediatric care during a short-term medical mission to a Syrian refugee camp in Northern Jordan.