The Clinical Utility of a 7-Gene Biosignature on Radiation Therapy Decision Making in Patients with Ductal Carcinoma In Situ Following Breast-Conserving Surgery: An Updated Analysis of the DCISionRT® PREDICT Study.

BACKGROUND: Breast-conserving surgery (BCS) followed by adjuvant radiotherapy (RT) is a standard treatment for ductal carcinoma in situ (DCIS). A low-risk patient subset that does not benefit from RT has not yet been clearly identified. The DCISionRT test provides a clinically validated decision score (DS), which is prognostic of 10-year in-breast recurrence rates (invasive and non-invasive) and is also predictive of RT benefit. This analysis presents final outcomes from the PREDICT prospective registry trial aiming to determine how often the DCISionRT test changes radiation treatment recommendations. METHODS: Overall, 2496 patients were enrolled from February 2018 to January 2022 at 63 academic and community practice sites and received DCISionRT as part of their care plan. Treating physicians reported their treatment recommendations pre- and post-test as well as the patient's preference. The primary endpoint was to identify the percentage of patients where testing led to a change in RT recommendation. The impact of the test on RT treatment recommendation was physician specialty, treatment settings, individual clinical/pathological features and RTOG 9804 like criteria. Multivariate logisitc regression analysis was used to estimate the odds ratio (ORs) for factors associated with the post-test RT recommendations. RESULTS: RT recommendation changed 38% of women, resulting in a 20% decrease in the overall recommendation of RT (p < 0.001). Of those women initially recommended no RT (n = 583), 31% were recommended RT post-test. The recommendation for RT post-test increased with increasing DS, from 29% to 66% to 91% for DS <2, DS 2-4, and DS >4, respectively. On multivariable analysis, DS had the strongest influence on final RT recommendation (odds ratio 22.2, 95% confidence interval 16.3-30.7), which was eightfold greater than clinicopathologic features. Furthermore, there was an overall change in the recommendation to receive RT in 42% of those patients meeting RTOG 9804-like low-risk criteria. CONCLUSIONS: The test results provided information that changes treatment recommendations both for and against RT use in large population of women with DCIS treated in a variety of clinical settings. Overall, clinicians changed their recommendations to include or omit RT for 38% of women based on the test results. Based on published clinical validations and the results from current study, DCISionRT may aid in preventing the over- and undertreatment of clinicopathological 'low-risk' and 'high-risk' DCIS patients. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT03448926 ( https://clinicaltrials.gov/study/NCT03448926 ).

Evaluating outcomes and toxicities for a newly implemented MRI-based brachytherapy program for cervical cancer.

OBJECTIVE: We report an updated analysis of the outcomes and toxicities of MRI-based brachytherapy for locally advanced cervical cancer from a U.S. academic center. METHODS: A retrospective review was performed on patients treated with MRI-based brachytherapy for cervical cancer. EBRT was standardly 45 Gy in 25 fractions with weekly cisplatin. MRI was performed with the brachytherapy applicator in situ. Dose specification was most commonly 7 Gy for 4 fractions with optimization aim of D90 HR-CTV EQD2 of 85-95 Gyα/ß=10 Gy in 2 implants each delivering 2 fractions. RESULTS: Ninety-eight patients were included with median follow up of 24.5 months (IQR 11.9-39.8). Stage IIIA-IVB accounted for 31.6% of cases. Dosimetry results include median GTV D98 of 101.0 Gy (IQR 93.3-118.8) and HR-CTV D90 of 89 Gy (IQR 86.1-90.6). Median D2cc bladder, rectum, sigmoid, and bowel doses were 82.1 Gy (IQR 75.9-88.0), 65.9 Gy (IQR 59.6-71.2), 65.1 Gy (IQR 57.7-69.6), and 55 Gy (IQR 48.9-60.9). Chronic grade 3+ toxicities were seen in the bladder (8.2%), rectosigmoid (4.1%), and vagina (1.0%). Three-year LC, PFS, and OS were estimated to be 84%, 61.7%, and 76.1%, respectively. CONCLUSION: MRI-based brachytherapy demonstrates excellent local control and acceptable rates of high-grade morbidity. These results are possible in our population with relatively large volume primary tumors and extensive local disease.

Geometric and dosimetric evaluation for breast and regional nodal auto-segmentation structures.

The accuracy of artificial intelligence (AI) generated contours for intact-breast and post-mastectomy radiotherapy plans was evaluated. Geometric and dosimetric comparisons were performed between auto-contours (ACs) and manual-contours (MCs) produced by physicians for target structures. Breast and regional nodal structures were manually delineated on 66 breast cancer patients. ACs were retrospectively generated. The characteristics of the breast/post-mastectomy chestwall (CW) and regional nodal structures (axillary [AxN], supraclavicular [SC], internal mammary [IM]) were geometrically evaluated by Dice similarity coefficient (DSC), mean surface distance, and Hausdorff Distance. The structures were also evaluated dosimetrically by superimposing the MC clinically delivered plans onto the ACs to assess the impact of utilizing ACs with target dose (Vx%) evaluation. Positive geometric correlations between volume and DSC for intact-breast, AxN, and CW were observed. Little or anti correlations between volume and DSC for IM and SC were shown. For intact-breast plans, insignificant dosimetric differences between ACs and MCs were observed for AxNV95% (p = 0.17) and SCV95% (p = 0.16), while IMNV90% ACs and MCs were significantly different. The average V95% for intact-breast MCs (98.4%) and ACs (97.1%) were comparable but statistically different (p = 0.02). For post-mastectomy plans, AxNV95% (p = 0.35) and SCV95% (p = 0.08) were consistent between ACs and MCs, while IMNV90% was significantly different. Additionally, 94.1% of AC-breasts met ΔV95% variation <5% when DSC > 0.7. However, only 62.5% AC-CWs achieved the same metrics, despite AC-CWV95% (p = 0.43) being statistically insignificant. The AC intact-breast structure was dosimetrically similar to MCs. The AC AxN and SC may require manual adjustments. Careful review should be performed for AC post-mastectomy CW and IMN before treatment planning. The findings of this study may guide the clinical decision-making process for the utilization of AI-driven ACs for intact-breast and post-mastectomy plans. Before clinical implementation of this auto-segmentation software, an in-depth assessment of agreement with each local facilities MCs is needed.

Leveraging Electronic Health Record Data and Measuring Interdependence in the Era of Precision Education and Assessment.

PURPOSE: The era of precision education is increasingly leveraging electronic health record (EHR) data to assess residents' clinical performance. But precision in what the EHR-based resident performance metrics are truly assessing is not fully understood. For instance, there is limited understanding of how EHR-based measures account for the influence of the team on an individual's performance-or conversely how an individual contributes to team performances. This study aims to elaborate on how the theoretical understandings of supportive and collaborative interdependence are captured in residents' EHR-based metrics. METHOD: Using a mixed methods study design, the authors conducted a secondary analysis of 5 existing quantitative and qualitative datasets used in previous EHR studies to investigate how aspects of interdependence shape the ways that team-based care is provided to patients. RESULTS: Quantitative analyses of 16 EHR-based metrics found variability in faculty and resident performance (both between and within resident). Qualitative analyses revealed that faculty lack awareness of their own EHR-based performance metrics, which limits their ability to act interdependently with residents in an evidence-informed fashion. The lens of interdependence elucidates how resident practice patterns develop across residency training, shifting from supportive to collaborative interdependence over time. Joint displays merging the quantitative and qualitative analyses showed that residents are aware of variability in faculty's practice patterns and that viewing resident EHR-based measures without accounting for the interdependence of residents with faculty is problematic, particularly within the framework of precision education. CONCLUSIONS: To prepare for this new paradigm of precision education, educators need to develop and evaluate theoretically robust models that measure interdependence in EHR-based metrics, affording more nuanced interpretation of such metrics when assessing residents throughout training.

ACR-ARS Practice Parameter for Communication: Radiation Oncology.

BACKGROUND: This practice parameter was revised collaboratively by the American College of Radiology (ACR), and the American Radium Society (ARS). Timely, accurate, and effective communications are critical to quality and safety in contemporary medical practices. Radiation oncology incorporates the science and technology of complex, integrated treatment delivery and the art of providing care to individual patients. Through written physical and/or electronic reports and direct communication, radiation oncologists convey their knowledge and evaluation regarding patient care, clinical workup, and treatment provided to others in the management of the patient. Applicable practice parameters need to be revised periodically regarding medical record documentation for professional and technical components of services delivered. METHODS: This practice parameter was developed and revised according to the process described under the heading "The Process for Developing ACR Practice Parameters and Technical Standards" on the ACR website ( https://www.acr.org/Clinical-Resources/Practice-Parameters-and-Technical-Standards ) by the Committee on Practice Parameters-Radiation Oncology of the ACR Commission on Radiation Oncology in collaboration with the ARS. Both societies have reviewed and approved the document. RESULTS: This practice parameter addresses radiation oncology communications in general, including (a) medical record, (b) electronic, and (c) doctor-patient communications, as well as specific documentation for radiation oncology reports such as (a) consultation, (b) clinical treatment management notes (including inpatient communication), (c) treatment (completion) summary, and (d) follow-up visits. CONCLUSIONS: The radiation oncologist's participation in the multidisciplinary management of patients is reflected in timely, medically appropriate, and informative communication with patients, caregivers, referring physician, and other members of the health care team. The ACR-ARS Practice Parameter for Communication: Radiation Oncology is an educational tool designed to assist practitioners in providing appropriate communication regarding radiation oncology care for patients.

ACR-ARS Practice Parameter for Radiation Oncology.

BACKGROUND: This practice parameter was revised collaboratively by the American College of Radiology (ACR), and the American Radium Society. This practice parameter provides updated reference literature regarding radiation oncology practice and its key personnel. METHODS: This practice parameter was developed according to the process described under the heading The Process for Developing ACR Practice Parameters and Technical Standards on the ACR website ( https://www.acr.org/Clinical-Resources/Practice-Parameters-and-Technical-Standards ) by the Committee on Practice Parameters-Radiation Oncology of the ACR Commission on Radiation Oncology in collaboration with the American Radium Society. RESULTS: This practice parameter provides a comprehensive update to the reference literature regarding radiation oncology practice in general. The overall roles of the radiation oncologist, the Qualified Medical Physicist, and other specialized personnel involved in the delivery of external-beam radiation therapy are discussed. The use of radiation therapy requires detailed attention to equipment, patient and personnel safety, equipment maintenance and quality assurance, and continuing staff education. Because the practice of radiation oncology occurs in a variety of clinical environments, the judgment of a qualified radiation oncologist should be used to apply these practice parameters to individual practices. Radiation oncologists should follow the guiding principle of limiting radiation exposure to patients and personnel while accomplishing therapeutic goals. CONCLUSION: This practice parameter can be used as an effective tool to guide radiation oncology practice by successfully incorporating the close interaction and coordination among radiation oncologists, medical physicists, dosimetrists, nurses, and radiation therapists.

Large Language Model-Based Responses to Patients' In-Basket Messages.

Importance: Virtual patient-physician communications have increased since 2020 and negatively impacted primary care physician (PCP) well-being. Generative artificial intelligence (GenAI) drafts of patient messages could potentially reduce health care professional (HCP) workload and improve communication quality, but only if the drafts are considered useful. Objectives: To assess PCPs' perceptions of GenAI drafts and to examine linguistic characteristics associated with equity and perceived empathy. Design, Setting, and Participants: This cross-sectional quality improvement study tested the hypothesis that PCPs' ratings of GenAI drafts (created using the electronic health record [EHR] standard prompts) would be equivalent to HCP-generated responses on 3 dimensions. The study was conducted at NYU Langone Health using private patient-HCP communications at 3 internal medicine practices piloting GenAI. Exposures: Randomly assigned patient messages coupled with either an HCP message or the draft GenAI response. Main Outcomes and Measures: PCPs rated responses' information content quality (eg, relevance), using a Likert scale, communication quality (eg, verbosity), using a Likert scale, and whether they would use the draft or start anew (usable vs unusable). Branching logic further probed for empathy, personalization, and professionalism of responses. Computational linguistics methods assessed content differences in HCP vs GenAI responses, focusing on equity and empathy. Results: A total of 16 PCPs (8 [50.0%] female) reviewed 344 messages (175 GenAI drafted; 169 HCP drafted). Both GenAI and HCP responses were rated favorably. GenAI responses were rated higher for communication style than HCP responses (mean [SD], 3.70 [1.15] vs 3.38 [1.20]; P = .01, U = 12 568.5) but were similar to HCPs on information content (mean [SD], 3.53 [1.26] vs 3.41 [1.27]; P = .37; U = 13 981.0) and usable draft proportion (mean [SD], 0.69 [0.48] vs 0.65 [0.47], P = .49, t = -0.6842). Usable GenAI responses were considered more empathetic than usable HCP responses (32 of 86 [37.2%] vs 13 of 79 [16.5%]; difference, 125.5%), possibly attributable to more subjective (mean [SD], 0.54 [0.16] vs 0.31 [0.23]; P < .001; difference, 74.2%) and positive (mean [SD] polarity, 0.21 [0.14] vs 0.13 [0.25]; P = .02; difference, 61.5%) language; they were also numerically longer (mean [SD] word count, 90.5 [32.0] vs 65.4 [62.6]; difference, 38.4%), but the difference was not statistically significant (P = .07) and more linguistically complex (mean [SD] score, 125.2 [47.8] vs 95.4 [58.8]; P = .002; difference, 31.2%). Conclusions: In this cross-sectional study of PCP perceptions of an EHR-integrated GenAI chatbot, GenAI was found to communicate information better and with more empathy than HCPs, highlighting its potential to enhance patient-HCP communication. However, GenAI drafts were less readable than HCPs', a significant concern for patients with low health or English literacy.

ACR Appropriateness Criteria® Pretreatment Evaluation and Follow-Up of Invasive Cancer of the Cervix: 2023 Update.

Cervical cancer is a common gynecological malignancy worldwide. Cervical cancer is staged based on the International Federation of Gynecology and Obstetrics (FIGO) classification system, which was revised in 2018 to incorporate radiologic and pathologic data. Imaging plays an important role in pretreatment assessment including initial staging and treatment response assessment of cervical cancer. Accurate determination of tumor size, local extension, and nodal and distant metastases is important for treatment selection and for prognostication. Although local recurrence can be diagnosed by physical examination, imaging plays a critical role in detection and follow-up of local and distant recurrence and subsequent treatment selection. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

The First Generative AI Prompt-A-Thon in Healthcare: A Novel Approach to Workforce Engagement with a Private Instance of ChatGPT.

BACKGROUND: Healthcare crowdsourcing events (e.g. hackathons) facilitate interdisciplinary collaboration and encourage innovation. Peer-reviewed research has not yet considered a healthcare crowdsourcing event focusing on generative artificial intelligence (GenAI), which generates text in response to detailed prompts and has vast potential for improving the efficiency of healthcare organizations. Our event, the New York University Langone Health (NYULH) Prompt-a-thon, primarily sought to inspire and build AI fluency within our diverse NYULH community, and foster collaboration and innovation. Secondarily, we sought to analyze how participants' experience was influenced by their prior GenAI exposure and whether they received sample prompts during the workshop. METHODS: Executing the event required the assembly of an expert planning committee, who recruited diverse participants, anticipated technological challenges, and prepared the event. The event was composed of didactics and workshop sessions, which educated and allowed participants to experiment with using GenAI on real healthcare data. Participants were given novel "project cards" associated with each dataset that illuminated the tasks GenAI could perform and, for a random set of teams, sample prompts to help them achieve each task (the public repository of project cards can be found at https://github.com/smallw03/NYULH-Generative-AI-Prompt-a-thon-Project-Cards). Afterwards, participants were asked to fill out a survey with 7-point Likert-style questions. RESULTS: Our event was successful in educating and inspiring hundreds of enthusiastic in-person and virtual participants across our organization on the responsible use of GenAI in a low-cost and technologically feasible manner. All participants responded positively, on average, to each of the survey questions (e.g., confidence in their ability to use and trust GenAI). Critically, participants reported a self-perceived increase in their likelihood of using and promoting colleagues' use of GenAI for their daily work. No significant differences were seen in the surveys of those who received sample prompts with their project task descriptions. CONCLUSION: The first healthcare Prompt-a-thon was an overwhelming success, with minimal technological failures, positive responses from diverse participants and staff, and evidence of post-event engagement. These findings will be integral to planning future events at our institution, and to others looking to engage their workforce in utilizing GenAI.

Executive Summary of the American Radium Society Appropriate Use Criteria for Neoadjuvant Therapy for Nonmetastatic Pancreatic Adenocarcinoma: Systematic Review and Guidelines.

For patients with locoregionally confined pancreatic ductal adenocarcinoma (PDAC), margin-negative surgical resection is the only known curative treatment; however, the majority of patients are not operable candidates at initial diagnosis. Among patients with resectable disease who undergo surgery alone, the 5-year survival remains poor. Adjuvant therapies, including systemic therapy or chemoradiation, are utilized as they improve locoregional control and overall survival. There has been increasing interest in the use of neoadjuvant therapy to obtain early control of occult metastatic disease, allow local tumor response to facilitate margin-negative resection, and provide a test of time and biology to assist with the selection of candidates most likely to benefit from radical surgical resection. However, limited guidance exists regarding the relative effectiveness of treatment options. In this systematic review, the American Radium Society multidisciplinary gastrointestinal expert panel convened to develop Appropriate Use Criteria evaluating the evidence regarding neoadjuvant treatment for patients with PDAC, including surgery, systemic therapy, and radiotherapy, in terms of oncologic outcomes and quality of life. The evidence was assessed using the Population, Intervention, Comparator, Outcome, and Study (PICOS) design framework and "Preferred Reporting Items for Systematic Reviews and Meta-analyses" 2020 methodology. Eligible studies included phases 2 to 3 trials, meta-analyses, and retrospective analyses published between January 1, 2012 and December 30, 2022 in the Ovid Medline database. A summary of recommendations based on the available literature is outlined to guide practitioners in the management of patients with PDAC.

Radiation Therapy With or Without Cisplatin for Local Recurrences of Endometrial Cancer: Results From an NRG Oncology/GOG Prospective Randomized Multicenter Clinical Trial.

PURPOSE: Pelvic recurrence is a frequent pattern of relapse for women with endometrial cancer. A randomized trial compared progression-free survival (PFS) after treatment with radiation therapy alone as compared with concurrent chemotherapy. MATERIALS AND METHODS: Between February 2008 and August 2020, 165 patients were randomly assigned 1:1 to receive either radiation treatment alone or a combination of chemotherapy and radiation treatment. The primary objective of this study was to determine whether chemoradiation therapy was more effective than radiation therapy alone at improving PFS. RESULTS: The majority of patients had low-grade (1 or 2) endometrioid histology (82%) and recurrences confined to the vagina (86%). External beam with either the three-dimensional or intensity modulated radiation treatment technique was followed by a boost delivered with brachytherapy or external beam. Patients randomly assigned to receive chemotherapy were treated with once weekly cisplatin (40 mg/m2). Rates of acute toxicity were higher in patients treated with chemoradiation as compared with radiation treatment alone. Median PFS was longer for patients treated with radiation therapy alone as compared with chemotherapy and radiation (median PFS was not reached for RT v 73 months for chemoradiation, hazard ratio of 1.25 (95% CI, 0.75 to 2.07). At 3 years, 73% of patients treated definitively with radiation and 62% of patients treated with chemoradiation were alive and free of disease progression. CONCLUSION: Excellent outcomes can be achieved for women with localized recurrences of endometrial cancer when treated with radiation therapy. The addition of chemotherapy does not improve PFS for patients treated with definitive radiation therapy for recurrent endometrial cancer and increases acute toxicity. Patients with low-grade and vaginal recurrences who constituted the majority of those enrolled are best treated with radiation therapy alone.

ACR-ACNM-ARS-ASTRO-SNMMI Practice Parameter for the Performance of Therapy With Radiopharmaceuticals.

OBJECTIVES: This practice parameter was revised collaboratively by the American College of Radiology (ACR), the American College of Nuclear Medicine, the American Radium Society, the American Society for Radiation Oncology, and the Society of Nuclear Medicine and Molecular Imaging. The document is intended to serve as a resource for appropriately trained and licensed physicians who perform therapeutic procedures with unsealed sources, referred to in the document using the more inclusive terminology of radiopharmaceuticals, for which a written directive is required for authorized users under NRC 10 CFR 35.300. METHODS: This practice parameter was developed according to the process described under the heading The Process for Developing ACR Practice Parameters and Technical Standards on the ACR website ( https://www.acr.org/Clinical-Resources/Practice-Parameters-and-Technical-Standards ) by the Committee on Practice Parameters-Radiation Oncology of the ACR Commission on Radiation Oncology in collaboration with the American Radium Society. RESULTS: This practice parameter addresses the overall role of the applicable physician-authorized user, Qualified Medical Physicist, and other specialized personnel involved in the delivery of radiopharmaceutical therapy. Therapeutic radiopharmaceuticals include those administered as elemental radioactive isotopes (radionuclides) or the radioactive element incorporated into a targeting molecule (ligand) by one or more chemical bonds. This document provides guidance regarding general principles of radionuclide therapies and indications of various alpha, beta, gamma, and mixed emission agents with references to several recent practice parameters on new and commonly performed radiopharmaceutical therapies. CONCLUSION: This document addresses clinical circumstances, elements of available agents, and the qualifications and responsibilities of various members of the radiation care team, specifications of consultation and other clinical documentation, post-therapy follow-up, radiation safety precautions, elements of quality control and improvement programs, infection control, and patient education to ensure optimal patient care and safety when utilizing radiopharmaceuticals.

Executive Summary of the American Radium Society on Appropriate Use Criteria for Nonoperative Management of Rectal Adenocarcinoma: Systematic Review and Guidelines.

For patients with rectal cancer, the standard approach of chemotherapy, radiation therapy, and surgery (trimodality therapy) is associated with significant long-term toxicity and/or colostomy for most patients. Patient options focused on quality of life (QOL) have dramatically improved, but there remains limited guidance regarding comparative effectiveness. This systematic review and associated guidelines evaluate how various treatment strategies compare to each other in terms of oncologic outcomes and QOL. Cochrane and Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology were used to search for prospective and retrospective trials and meta-analyses of adequate quality within the Ovid Medline database between January 1, 2012, and June 15, 2023. These studies informed the expert panel, which rated the appropriateness of various treatments in 6 clinical scenarios through a well-established consensus methodology (modified Delphi). The search process yielded 197 articles that advised voting. Increasing data have shown that nonoperative management (NOM) and primary surgery result in QOL benefits noted over trimodality therapy without detriment to oncologic outcomes. For patients with rectal cancer for whom total mesorectal excision would result in permanent colostomy or inadequate bowel continence, NOM was strongly recommended as usually appropriate. Restaging with tumor response assessment approximately 8 to 12 weeks after completion of radiation therapy/chemoradiation therapy was deemed a necessary component of NOM. The panel recommended active surveillance in the setting of a near-complete or complete response. In the setting of NOM, 54 to 56 Gy in 27 to 31 fractions concurrent with chemotherapy and followed by consolidation chemotherapy was recommended. The panel strongly recommends primary surgery as usually appropriate for a T3N0 high rectal tumor for which low anterior resection and adequate bowel function is possible, with adjuvant chemotherapy considered if N+. Recent data support NOM and primary surgery as important options that should be offered to eligible patients. Considering the complexity of multidisciplinary management, patients should be discussed in a multidisciplinary setting, and therapy should be tailored to individual patient goals/values.

ACR Appropriateness Criteria® Female Breast Cancer Screening: 2023 Update.

Early detection of breast cancer from regular screening substantially reduces breast cancer mortality and morbidity. Multiple different imaging modalities may be used to screen for breast cancer. Screening recommendations differ based on an individual's risk of developing breast cancer. Numerous factors contribute to breast cancer risk, which is frequently divided into three major categories: average, intermediate, and high risk. For patients assigned female at birth with native breast tissue, mammography and digital breast tomosynthesis are the recommended method for breast cancer screening in all risk categories. In addition to the recommendation of mammography and digital breast tomosynthesis in high-risk patients, screening with breast MRI is recommended. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores.

Importance: Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR. Objective: Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates. Methods: Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation. Results: For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date. Conclusions: In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

Electronic Health Record Messaging Patterns of Health Care Professionals in Inpatient Medicine.

This cross-sectional study examines secure messaging patterns among health care professionals in the internal medicine inpatient setting.

ACR appropriateness criteria staging and follow-up of vulvar cancer

Vulvar cancer is an uncommon gynecologic tumor and one of several human papillomavirus-associated malignancies. Squamous cell carcinoma is the most prevalent histologic subtype of vulvar cancer, accounting for the majority of cases. Imaging plays an important role in managing vulvar cancer. At initial diagnosis, imaging is useful to assess the size and extent of primary tumor and to evaluate the status of inguinofemoral lymph nodes. If recurrent disease is suspected, imaging is essential to demonstrate local extent of tumor and to identify lymph node and distant metastases. In this publication, we summarize the recent literature and describe the panel's recommendations about the appropriate use of imaging for various phases of patient management including initial staging, surveillance, and restaging of vulvar cancer. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision include an extensive analysis of current medical literature from peer reviewed journals and the application of well-established methodologies (RAND/UCLA Appropriateness Method and Grading of Recommendations Assessment, Development, and Evaluation or GRADE) to rate the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where evidence is lacking or equivocal, expert opinion may supplement the available evidence to recommend imaging or treatment.