Barriers to Postdischarge Smartphone App Use Among Patients With Traumatic Rib Fractures.

Rib fractures commonly result from traumatic injury and often require hospitalization for pain control and supportive pulmonary care. Although the use of mobile health technology to share patient-generated health data has increased, it remains limited in patients with traumatic injuries. We sought to assess the feasibility of mobile health tracking in patients with rib fractures by using a smartphone app to monitor postdischarge recovery. We encountered patient, institutional, and process-related obstacles that limited app use. The success of future work requires the acknowledgment of these limitations and the use of an implementation science framework to effectively integrate technological tools for personalized trauma care.

Postoperative Opioid Prescribing via Rule-Based Guidelines Derived from In-Hospital Consumption: An Assessment of Efficacy Based on Postdischarge Opioid Use.

BACKGROUND: Many institutions have developed operation-specific guidelines for opioid prescribing. These guidelines rarely incorporate in-hospital opioid consumption, which is highly correlated with consumption. We compare outcomes of several patient-centered approaches to prescribing that are derived from in-hospital consumption, including several experimental, rule-based prescribing guidelines and our current institutional guideline. STUDY DESIGN: We performed a retrospective, cohort study of all adults undergoing surgery at a single-academic medical center. Several rule-based guidelines, derived from in-hospital consumption (quantity of opioids consumed within 24 hours of discharge), were used to specify the theoretical quantity of opioid prescribed on discharge. The efficacy of the experimental guidelines was compared with 3 references: an approximation of our institution's tailored prescribing guideline; prescribing all patients the typical quantity of opioids consumed for patients undergoing the same operation; and a representative rule-based, tiered framework. For each scenario, we calculated the penalized residual sum of squares (reflecting the composite deviation from actual patient consumption, with 15% penalty for overprescribing) and the proportion of opioids consumed relative to prescribed. RESULTS: A total of 1,048 patients met inclusion criteria. Mean (SD) and median (interquartile range [IQR]) quantity of opioids consumed within 24 hours of discharge were 11.2 (26.9) morphine milligram equivalents and 0 (0 to 15) morphine milligram equivalents. Median (IQR) postdischarge consumption was 16 (0 to 150) morphine milligram equivalents. Our institutional guideline and the previously validated rule-based guideline outperform alternate approaches, with median (IQR) differences in prescribed vs consumed opioids of 0 (-60 to 27.25) and 37.5 (-37.5 to 37.5), respectively, corresponding to penalized residual sum of squares of 39,817,602 and 38,336,895, respectively. CONCLUSIONS: Rather than relying on fixed quantities for defined operations, rule-based guidelines offer a simple yet effective method for tailoring opioid prescribing to in-hospital consumption.

Evaluating capabilities of large language models: Performance of GPT-4 on surgical knowledge assessments.

BACKGROUND: Artificial intelligence has the potential to dramatically alter health care by enhancing how we diagnose and treat disease. One promising artificial intelligence model is ChatGPT, a general-purpose large language model trained by OpenAI. ChatGPT has shown human-level performance on several professional and academic benchmarks. We sought to evaluate its performance on surgical knowledge questions and assess the stability of this performance on repeat queries. METHODS: We evaluated the performance of ChatGPT-4 on questions from the Surgical Council on Resident Education question bank and a second commonly used surgical knowledge assessment, referred to as Data-B. Questions were entered in 2 formats: open-ended and multiple-choice. ChatGPT outputs were assessed for accuracy and insights by surgeon evaluators. We categorized reasons for model errors and the stability of performance on repeat queries. RESULTS: A total of 167 Surgical Council on Resident Education and 112 Data-B questions were presented to the ChatGPT interface. ChatGPT correctly answered 71.3% and 67.9% of multiple choice and 47.9% and 66.1% of open-ended questions for Surgical Council on Resident Education and Data-B, respectively. For both open-ended and multiple-choice questions, approximately two-thirds of ChatGPT responses contained nonobvious insights. Common reasons for incorrect responses included inaccurate information in a complex question (n = 16, 36.4%), inaccurate information in a fact-based question (n = 11, 25.0%), and accurate information with circumstantial discrepancy (n = 6, 13.6%). Upon repeat query, the answer selected by ChatGPT varied for 36.4% of questions answered incorrectly on the first query; the response accuracy changed for 6/16 (37.5%) questions. CONCLUSION: Consistent with findings in other academic and professional domains, we demonstrate near or above human-level performance of ChatGPT on surgical knowledge questions from 2 widely used question banks. ChatGPT performed better on multiple-choice than open-ended questions, prompting questions regarding its potential for clinical application. Unique to this study, we demonstrate inconsistency in ChatGPT responses on repeat queries. This finding warrants future consideration including efforts at training large language models to provide the safe and consistent responses required for clinical application. Despite near or above human-level performance on question banks and given these observations, it is unclear whether large language models such as ChatGPT are able to safely assist clinicians in providing care.

Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning.

Objective: To overcome limitations of open surgery artificial intelligence (AI) models by curating the largest collection of annotated videos and to leverage this AI-ready data set to develop a generalizable multitask AI model capable of real-time understanding of clinically significant surgical behaviors in prospectively collected real-world surgical videos. Design, Setting, and Participants: The study team programmatically queried open surgery procedures on YouTube and manually annotated selected videos to create the AI-ready data set used to train a multitask AI model for 2 proof-of-concept studies, one generating surgical signatures that define the patterns of a given procedure and the other identifying kinematics of hand motion that correlate with surgeon skill level and experience. The Annotated Videos of Open Surgery (AVOS) data set includes 1997 videos from 23 open-surgical procedure types uploaded to YouTube from 50 countries over the last 15 years. Prospectively recorded surgical videos were collected from a single tertiary care academic medical center. Deidentified videos were recorded of surgeons performing open surgical procedures and analyzed for correlation with surgical training. Exposures: The multitask AI model was trained on the AI-ready video data set and then retrospectively applied to the prospectively collected video data set. Main Outcomes and Measures: Analysis of open surgical videos in near real-time, performance on AI-ready and prospectively collected videos, and quantification of surgeon skill. Results: Using the AI-ready data set, the study team developed a multitask AI model capable of real-time understanding of surgical behaviors-the building blocks of procedural flow and surgeon skill-across space and time. Through principal component analysis, a single compound skill feature was identified, composed of a linear combination of kinematic hand attributes. This feature was a significant discriminator between experienced surgeons and surgical trainees across 101 prospectively collected surgical videos of 14 operators. For each unit increase in the compound feature value, the odds of the operator being an experienced surgeon were 3.6 times higher (95% CI, 1.67-7.62; P = .001). Conclusions and Relevance: In this observational study, the AVOS-trained model was applied to analyze prospectively collected open surgical videos and identify kinematic descriptors of surgical skill related to efficiency of hand motion. The ability to provide AI-deduced insights into surgical structure and skill is valuable in optimizing surgical skill acquisition and ultimately improving surgical care.

An Institutional Curriculum for Opioid Prescribing Education: Outcomes From 2017 to 2022.

INTRODUCTION: Prescription opioids, including those prescribed after surgery, have greatly contributed to the US opioid epidemic. Educating opioid prescribers is a crucial component of ensuring the safe use of opioids among surgical patients. METHODS: An annual opioid prescribing education curriculum was implemented among new surgical prescribers at our institution between 2017 and 2022. The curriculum includes a single 75-min session which is comprised of several components: pain medications (dosing, indications, and contraindications); patients at high risk for uncontrolled pain and/or opioid misuse or abuse; patient monitoring and care plans; and state and federal regulations. Participants were asked to complete an opioid knowledge assessment before and after the didactic session. RESULTS: Presession and postsession assessments were completed by 197 (89.6%) prescribers. Across the five studied years, the median presession score was 54.5%. This increased to 63.6% after completion of the curriculum, representing a median relative knowledge increase of 18.2%. The median relative improvement was greatest for preinterns and interns (18.2% for both groups); smaller improvements were observed for postgraduate year 2-5 residents (9.1%) and advanced practice providers (9.1%). On a scale of 1 to 10 (with 5 being comfortable), median (interquartile range) self-reported comfort in prescribing opioids increased from 3 (2-5) before education to 5 (4-6) after education (P < 0.001). CONCLUSIONS: Each year, the curriculum substantially improved provider knowledge of and comfort in opioid prescribing. Despite increased national awareness of the opioid epidemic and increasing institutional initiatives to improve opioid prescribing practices, there was a sustained knowledge and comfort gap among new surgical prescribers. The observed effects of our opioid education curriculum highlight the value of a simple and efficient educational initiative.

Comparing Rationale for Opioid Prescribing Decisions after Surgery with Subsequent Patient Consumption: A Survey of the Highest Quartile of Prescribers.

BACKGROUND: Opioid prescribing patterns, including those after surgery, have been implicated as a significant contributor to the US opioid crisis. A plethora of interventions-from nudges to reminders-have been deployed to improve prescribing behavior, but reasons for persistent outlier behavior are often unknown. STUDY DESIGN: Our institution employs multiple prescribing resources and a near real-time, feedback-based intervention to promote appropriate opioid prescribing. Since 2019, an automated system has emailed providers when a prescription exceeds the 75th percentile of typical opioid consumption for a given procedure-as defined by institutional data collection. Emails include population consumption metrics and an optional survey on rationale for prescribing. Responses were analyzed to understand why providers choose to prescribe atypically large discharge opioid prescriptions. We then compared provider prescriptions against patient consumption. RESULTS: During the study period, 10,672 eligible postsurgical patients were discharged; 2,013 prescriptions (29.4% of opioid prescriptions) exceeded our institutional guideline. Surveys were completed by outlier prescribers for 414 (20.6%) encounters. Among patients where both consumption data and prescribing rationale surveys were available, 35.2% did not consume any opioids after discharge and 21.5% consumed <50% of their prescription. Only 93 (39.9%) patients receiving outlier prescriptions were outlier consumers. Most common reasons for prescribing outlier amounts were attending preference (34%) and prescriber analysis of patient characteristics (34%). CONCLUSIONS: The top quartile of opioid prescriptions did not align with, and often far exceeded, patient postdischarge opioid consumption. Providers cite assessment of patient characteristics as a common driver of decision-making, but this did not align with patient usage for approximately 50% of patients.

Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments.

Background: Artificial intelligence (AI) has the potential to dramatically alter healthcare by enhancing how we diagnosis and treat disease. One promising AI model is ChatGPT, a large general-purpose language model trained by OpenAI. The chat interface has shown robust, human-level performance on several professional and academic benchmarks. We sought to probe its performance and stability over time on surgical case questions. Methods: We evaluated the performance of ChatGPT-4 on two surgical knowledge assessments: the Surgical Council on Resident Education (SCORE) and a second commonly used knowledge assessment, referred to as Data-B. Questions were entered in two formats: open-ended and multiple choice. ChatGPT output were assessed for accuracy and insights by surgeon evaluators. We categorized reasons for model errors and the stability of performance on repeat encounters. Results: A total of 167 SCORE and 112 Data-B questions were presented to the ChatGPT interface. ChatGPT correctly answered 71% and 68% of multiple-choice SCORE and Data-B questions, respectively. For both open-ended and multiple-choice questions, approximately two-thirds of ChatGPT responses contained non-obvious insights. Common reasons for inaccurate responses included: inaccurate information in a complex question (n=16, 36.4%); inaccurate information in fact-based question (n=11, 25.0%); and accurate information with circumstantial discrepancy (n=6, 13.6%). Upon repeat query, the answer selected by ChatGPT varied for 36.4% of inaccurate questions; the response accuracy changed for 6/16 questions. Conclusion: Consistent with prior findings, we demonstrate robust near or above human-level performance of ChatGPT within the surgical domain. Unique to this study, we demonstrate a substantial inconsistency in ChatGPT responses with repeat query. This finding warrants future consideration and presents an opportunity to further train these models to provide safe and consistent responses. Without mental and/or conceptual models, it is unclear whether language models such as ChatGPT would be able to safely assist clinicians in providing care.

Quantifying the Prognostic Value of Preoperative Surgeon Intuition: Comparing Surgeon Intuition and Clinical Risk Prediction as Derived from the American College of Surgeons NSQIP Risk Calculator.

BACKGROUND: Surgical risk prediction models traditionally use patient attributes and measures of physiology to generate predictions about postoperative outcomes. However, the surgeon's assessment of the patient may be a valuable predictor, given the surgeon's ability to detect and incorporate factors that existing models cannot capture. We compare the predictive utility of surgeon intuition and a risk calculator derived from the American College of Surgeons (ACS) NSQIP. STUDY DESIGN: From January 10, 2021 to January 9, 2022, surgeons were surveyed immediately before performing surgery to assess their perception of a patient's risk of developing any postoperative complication. Clinical data were abstracted from ACS NSQIP. Both sources of data were independently used to build models to predict the likelihood of a patient experiencing any 30-day postoperative complication as defined by ACS NSQIP. RESULTS: Preoperative surgeon assessment was obtained for 216 patients. NSQIP data were available for 9,182 patients who underwent general surgery (January 1, 2017 to January 9, 2022). A binomial regression model trained on clinical data alone had an area under the receiver operating characteristic curve (AUC) of 0.83 (95% CI 0.80 to 0.85) in predicting any complication. A model trained on only preoperative surgeon intuition had an AUC of 0.70 (95% CI 0.63 to 0.78). A model trained on surgeon intuition and a subset of clinical predictors had an AUC of 0.83 (95% CI 0.77 to 0.89). CONCLUSIONS: Preoperative surgeon intuition alone is an independent predictor of patient outcomes; however, a risk calculator derived from ACS NSQIP is a more robust predictor of postoperative complication. Combining intuition and clinical data did not strengthen prediction.

Opinions of African American adults about the use of apolipoprotein L1 (ApoL1) genetic testing in living kidney donation and transplantation.

Apolipoprotein L1 (ApoL1) predictive genetic testing for kidney disease, and its emerging role in transplantation, remains controversial as it may exacerbate underlying disparities among African Americans (AAs) at increased risk. We conducted an online simulation among AAs (N = 585) about interest in ApoL1 testing and its cofactors, under 2 scenarios: as a potential living donor (PLD), and as a patient awaiting transplantation. Most respondents (61%) expressed high interest in genetic testing as a PLD: age ≥35 years (adjusted odds ratio [aOR], 1.75; 95% confidence interval [CI], 1.18, 2.60, P = .01), AA identity (aOR, 1.67; 95% CI, 1.02, 2.72, P = .04), perceived kidney disease risk following donation (aOR, 1.68; 95% CI, 1.03, 2.73, P = .03), interest in genetics (aOR, 2.89; 95% CI, 1.95, 4.29, P = .001), and genetics self-efficacy (aOR, 2.38; 95% CI, 1.54, 3.67, P = .001) were positively associated with ApoL1 test interest. If awaiting transplantation, most (89%) believed that ApoL1 testing should be done on AA deceased donors, and older age (aOR, 1.85; 95% CI, 1.03, 3.32, P = .04) and greater interest in genetics (aOR, 2.61; 95% CI, 1.41, 4.81, P = .002) were associated with interest in testing deceased donors. Findings highlight strong support for ApoL1 testing in AAs and the need to examine such opinions among PLDs and transplant patients to enhance patient education efforts.

Differences in Postoperative Morbidity among Obese Patients Undergoing Abdominal Versus Laparoscopic Hysterectomy for Benign Indications.

STUDY OBJECTIVE: To analyze the interaction between the route of hysterectomy for benign disease and postoperative morbidity among patients stratified by body mass index (BMI) and to test for a dose-dependent relationship between obesity severity and postoperative morbidity. DESIGN: A retrospective cohort study. PATIENTS: Benign hysterectomy cases were abstracted from the American College of Surgeons National Safety and Quality Improvement Program from 2005 to 2016. Cancer and prolapse surgeries were excluded by corresponding International Classification of Diseases and Current Procedural Terminology codes. INTERVENTIONS: Laparoscopic hysterectomy. MEASUREMENTS AND MAIN RESULTS: Associations between BMI, route of surgery, and categoric patient variables were examined using the chi-square test. Associations of BMI, route of surgery, and continuous patient variables were examined using 1-way analysis of variance. Associations of the route of surgery with binary outcomes were examined within BMI categories using the chi-square or Fisher exact test. Logistic regression and interaction tests were used for final outcomes of interest. There were 159 025 patients in the collected sample. Patients who underwent an abdominal hysterectomy had higher odds of composite morbidity if they were obese; the adjusted odds were 17% higher for class 1 obesity, 55% higher for class 2 obesity, and 163% higher for class 3 obesity. An abdominal hysterectomy was associated with worse postoperative outcomes when compared with a laparoscopic hysterectomy (p <.001). The risk of increased composite postoperative morbidity for patients undergoing a laparoscopic hysterectomy was not significantly different from the reference group until women had class 3 obesity; the odds of composite morbidity for class 3 obesity women become 31% higher than for nonobese patients. CONCLUSION: BMI directly impacts postoperative morbidity for both abdominal and laparoscopic hysterectomies although the effect is more pronounced after an abdominal hysterectomy. Roughly 40% of women undergoing a hysterectomy in the United States are obese. These data should motivate surgeons to consider ways to medically and surgically optimize patients, including weight reduction before hysterectomy and choosing a laparoscopic approach whenever possible to lower the risk of postoperative morbidity.

Engaging Adolescents With Sexual Health Messaging: A Qualitative Analysis.

PURPOSE: Personalized and interactive text messaging interventions may increase participant engagement; yet, how to design messages that retain adolescent attention and positively affect sexual health behaviors remains unclear. The purpose of this study was to identify the characteristics of sexual health text messages perceived as engaging by sexually active adolescent females. METHODS: We conducted semistructured, open-ended interviews with sexually active females aged 14-19 in one urban emergency department. Participants received automated sexual health information sent via an interactive, two-way texting format. The 343 messages viewed by participants were based on key stakeholder input, relevant theoretical models, and existing evidence-based guidelines. Interviews elicited feedback. Enrollment continued until saturation of themes. Interviews were recorded, transcribed, and coded based on thematic analysis using NVivo 10. RESULTS: Participants (n = 31) were predominantly Hispanic (28; 90%), insured (29; 94%), and recently sexually active (24; 77%). Themes were as follows: (1) Tone: messages should be direct, factual, entertaining, and respect adolescent autonomy; messages should not be intrusive, presumptive, or preachy. (2) Emotion evoked: participants preferred messages that provoked thought, validated feelings, and empowered. Messages from a reliable source felt comforting, making participants feel cared for and special. (3) Interactivity: participants favored messages that offered choices, such as a mini-conversation. (4) Personalization: messages should look similar to adolescent digital preferences but be individually tailored with relatable characters. CONCLUSIONS: This study informs the tone, structure, and style of sexual health text messages directed toward adolescent females in the emergency department. Future work should consider these characteristics when designing digital interventions to engage adolescent females.