RESUMEN
BACKGROUND: Few studies have examined the performance of artificial intelligence (AI) content detection in scientific writing. This study evaluates the performance of publicly available AI content detectors when applied to both human-written and AI-generated scientific articles. METHODS: Articles published in Annals of Surgical Oncology (ASO) during the year 2022, as well as AI-generated articles using OpenAI's ChatGPT, were analyzed by three AI content detectors to assess the probability of AI-generated content. Full manuscripts and their individual sections were evaluated. Group comparisons and trend analyses were conducted by using ANOVA and linear regression. Classification performance was determined using area under the curve (AUC). RESULTS: A total of 449 original articles met inclusion criteria and were evaluated to determine the likelihood of being generated by AI. Each detector also evaluated 47 AI-generated articles by using titles from ASO articles. Human-written articles had an average probability of being AI-generated of 9.4% with significant differences between the detectors. Only two (0.4%) human-written manuscripts were detected as having a 0% probability of being AI-generated by all three detectors. Completely AI-generated articles were evaluated to have a higher average probability of being AI-generated (43.5%) with a range from 12.0 to 99.9%. CONCLUSIONS: This study demonstrates differences in the performance of various AI content detectors with the potential to label human-written articles as AI-generated. Any effort toward implementing AI detectors must include a strategy for continuous evaluation and validation as AI models and detectors rapidly evolve.
RESUMEN
OBJECTIVES: While patient interest in telehealth increases, clinicians' perspectives may influence longer-term adoption. We sought to identify facilitators and barriers to continued clinician incorporation of telehealth into practice. METHODS: A cross-sectional 24-item web-based survey was emailed to 491 providers with ≥50 video visits (VVs) within an academic health system between 1 March 2020 and 31 December 2020. We quantitatively summarised the characteristics and perceptions of respondents by using descriptive and test statistics. We used systematic content analysis to qualitatively code open-ended responses, double coding at least 25%. RESULTS: 247 providers (50.3%) responded to the survey. Seventy-nine per cent were confident in their ability to deliver excellent clinical care through VV. In comparison, 48% were confident in their ability to troubleshoot technical issues. Most clinicians (87%) expressed various concerns about VV. Providers across specialties generally agreed that VV reduced infection risk (71%) and transportation barriers (71%). Three overarching themes in the qualitative data included infrastructure and training, usefulness and expectation setting for patients and providers. DISCUSSION: As healthcare systems plan for future delivery directions, they must address the tension between patients' and providers' expectations of care within the digital space. Telehealth creates new friction, one where the healthcare system must fit into the patient's life rather than the usual dynamic of the patient fitting into the healthcare system. CONCLUSION: Telehealth infrastructure and patient and clinician technological acumen continue to evolve. Clinicians in this survey offered valuable insights into the directions healthcare organisations can take to right-size this healthcare delivery modality.