Precision of Chatbot Generative Pretrained Transformer Version 4-Generated References for Colon and Rectal Surgical Literature.

INTRODUCTION: The objective is to assess the precision of references generated by Chatbot Generative Pretrained Transformer version 4 (ChatGPT-4) in scientific literature pertaining to colon and rectal surgery. METHODS: Ten frequently studied keywords pertaining to colon and rectal surgery were chosen: colon cancer, rectal cancer, anal cancer, total neoadjuvant therapy, diverticulitis, low anterior resection, transanal minimally invasive surgery, ileal pouch anal anastomosis, abdominoperineal resection, and hemorrhoidectomy. ChatGPT-4 was prompted to search for the most representative citations for all keywords. After this, two separate evaluators meticulously examined the outcomes each key element, awarding full accuracy to generated citations in which there was no discrepancies in any of the fields when cross-referenced with the Scopus, Google, and PubMed databases. References from ChatGPT-4 underwent a thorough review process, which involved careful examination of key elements such as the article title, authors, journal name, publication year, and Digital Object Identifier (DOI). RESULTS: Forty-one of the 100 references generated by were fully accurate; however, but none included a DOI. Partial accuracy was observed in 67 of the references, which were identifiable by title and journal. Performance varied across specific keywords; for example, references for colon and rectal cancer were 100% identifiable by title and journal, but no term had 100% accuracy across all categories. Notably, none of the generated references correctly listed all authors. Conducted within a short timeframe during which ChatGPT4 is rapidly evolving and updating its knowledge base. CONCLUSIONS: While ChatGPT-4 offers improvements over its predecessors and shows potential for use in academic literature, its inconsistent performance across categories, lack of DOIs, and irregularities in authorship listings raise concerns about its readiness for application in the field of colon and rectal surgery research.

Effect of the COVID-19 pandemic on the residency match among surgical specialties.

BACKGROUND: Despite unprecedented changes to undergraduate medical education and the residency selection process during the COVID-19 pandemic, there is little objective evidence on how the pandemic affected match outcomes such as matched applicant characteristics, interview distribution, geographic clustering, and associated costs. We investigated COVID-19's impact on the residency match by comparing surgery applicants' characteristics, interview distribution, and related costs from 2018 to 2020 to 2021. METHODS: Data from the Texas Seeking Transparency in Applications to Residency initiative were analyzed. Descriptive statistics, bivariate testing, and sensitivity analysis were performed to compare matched applicants in surgical specialties from 2018-2020 to 2021. RESULTS: This study included 5,258 applicants who matched into 10 surgical specialties from 2018 to 2021. In 2021, there was a decrease in proportion of students who reported a geographic connection to their matched program (38.4% vs 42.1%; P = .021) and no significant difference in number of interviews attended (mean [SD], 13.1 [6.2] vs 13.3 [4.7]; P = .136) compared to prior years. Applicants in 2021 had more research experiences and fewer honored clerkships (both P < .001), and these associations persisted in sensitivity analysis. Matched applicants in 2021 reported significantly lower total costs associated with the residency application process compared to 2018 to 2020 (mean [SD] $1,959 [1,275] vs $6,756 [4,081]; P < .001). CONCLUSION: Although COVID-19 appeared to result in a reduction in number of honored clerkships, it may have provided more opportunities for students to engage in research. Overall, the adoption of virtual interviews and away rotations may have successfully mitigated some of the adverse consequences of the pandemic on the residency match for surgical specialties.

Evaluation of Predictors for Successful Residency Match in General Surgery.

OBJECTIVE: To determine predictive factors for a successful residency match among general surgery applicants from 2018 to 2021. DESIGN: A retrospective cross-sectional study of general surgery applicants who matched and went unmatched in match years 2018 to 2021. Applicant characteristics, geographic connections to a program, and away rotations were compared among matched and unmatched applicants. SETTING: Data were sourced from the Texas Seeking Transparency in Applications to Residency initiative for general surgery applicants. PARTICIPANTS: All fourth-year medical students applying in the 2018 to 2021 cycles at participating U.S. medical schools were eligible to respond to the Texas Seeking Transparency in Applications to Residency survey. This study included a total of 1,425 general surgery applicants. RESULTS: Of 1,425 general surgery applicants, 88% matched and 12% went unmatched. Significant predictors for a successful match included Step 1 Score ≥237 (odds ratio (OR) 1.59 [95% CI 1.15-2.19]; p = 0.005); Step 2 CK Score ≥252 (OR 1.88 [95% CI 1.36-2.60]; p < 0.001); ≥3 Honored Clerkships (OR 1.84 [95% CI 1.33-2.53]; p < 0.001); Honors in General Surgery Clerkship (OR 1.73 [95% CI 1.33-2.53]; p = 0.001); AOA membership (OR 2.14 [95% CI 1.34-3.42]; p = 0.001); ≥4 abstracts, posters, or publications (OR 1.66 [95% CI 1.20-2.30]; p=0.002); and ≥1 peer-reviewed publications (OR 1.52 [95% CI 1.09-2.12]; p = 0.014). On average, matched applicants completed more away rotations than unmatched applicants (p = 0.004). Overall, 36% of matched applicants reported a geographic connection to the program where they matched. CONCLUSIONS: We found that Step 2 CK score, research productivity, honored clerkships, AOA status, and away rotations are significant predictors for successfully matching into general surgery residency. Medical schools can encourage students to prepare a holistic application incorporating variables quantified in this study in preparation for the Step 1 reporting change.

A comparison of match outcomes between traditional medical degree and dual-degree applicants.

BACKGROUND: Dual degrees combining and MD with another professional degree (MPH, MBA, or PhD) are becoming more common in an attempt to increase an applicant's competitivity for a residency. OBJECTIVE: This study was designed to assess differences in MD-only and dual degree MD applicants with respect to applicant characteristics and match outcomes. METHODS: Utilizing the voluntarily-reported publicly available 2017-2019 Texas STAR database, we assessed applicants from 115 medical schools. Texas STAR indicates that over this time period, there were 18,224 responses for a response rate of 43.8%. Comparisons were made between groups using student's t-test and chi-squared analysis. RESULTS: Compared to MD only students, MD/MPH applicants had a higher propensity towards primary care specialties. MD/PhD applicants did not differ versus MD only applicants in their selection of primary care specialties, or of competitive specialties. MD/MBA applicants chose more competitive specialties and less primary care specialties. Despite all these differences, match rates were not different comparing MD only and dual-degree students. CONCLUSIONS: Despite the growing popularity of combined MD programs, such programs do not appear to increase applicant match competitivity.