RESUMO
PURPOSE: This study investigated the concordance of five different publicly available Large Language Models (LLM) with the recommendations of a multidisciplinary tumor board regarding treatment recommendations for complex breast cancer patient profiles. METHODS: Five LLM, including three versions of ChatGPT (version 4 and 3.5, with data access until September 3021 and January 2022), Llama2, and Bard were prompted to produce treatment recommendations for 20 complex breast cancer patient profiles. LLM recommendations were compared to the recommendations of a multidisciplinary tumor board (gold standard), including surgical, endocrine and systemic treatment, radiotherapy, and genetic testing therapy options. RESULTS: GPT4 demonstrated the highest concordance (70.6%) for invasive breast cancer patient profiles, followed by GPT3.5 September 2021 (58.8%), GPT3.5 January 2022 (41.2%), Llama2 (35.3%) and Bard (23.5%). Including precancerous lesions of ductal carcinoma in situ, the identical ranking was reached with lower overall concordance for each LLM (GPT4 60.0%, GPT3.5 September 2021 50.0%, GPT3.5 January 2022 35.0%, Llama2 30.0%, Bard 20.0%). GPT4 achieved full concordance (100%) for radiotherapy. Lowest alignment was reached in recommending genetic testing, demonstrating a varying concordance (55.0% for GPT3.5 January 2022, Llama2 and Bard up to 85.0% for GPT4). CONCLUSION: This early feasibility study is the first to compare different LLM in breast cancer care with regard to changes in accuracy over time, i.e., with access to more data or through technological upgrades. Methodological advancement, i.e., the optimization of prompting techniques, and technological development, i.e., enabling data input control and secure data processing, are necessary in the preparation of large-scale and multicenter studies to provide evidence on their safe and reliable clinical application. At present, safe and evidenced use of LLM in clinical breast cancer care is not yet feasible.
Assuntos
Neoplasias da Mama , Humanos , Feminino , Neoplasias da Mama/terapia , Neoplasias da Mama/genética , Tomada de Decisão Clínica , Técnicas de Apoio para a DecisãoRESUMO
BACKGROUND: Physicians have to make countless decisions every day. The medical, ethical and legal aspects are often intertwined and subject to change over time. Involving an ethics committee or arranging an ethical consultation are examples of potential aids to decision making. Whether and how artificial intelligence (AI) and the large language model (LLM) of the company OpenAI (San Francisco, CA, USA), known under the name ChatGPT, can also help and support ethical decision making is increasingly becoming a matter of controversial debate. MATERIAL AND METHODS: Based on a case example, in which a female physician is confronted with ethical and legal issues and presents these to ChatGPT to come up with answers, the first indications of the strengths and weaknesses are ascertained. CONCLUSION: Due to the rapid technical development and access to ever increasing quantities of data, the utilization should be closely observed and evaluated.
Assuntos
Inteligência Artificial , Comissão de Ética , Feminino , Humanos , Tomada de Decisão Clínica , Tomada de Decisões , Ética MédicaRESUMO
BACKGROUND: Physicians have to make countless decisions every day. The medical, ethical and legal aspects are often intertwined and subject to change over time. Involving an ethics committee or arranging an ethical consultation are examples of potential aids to decision making. Whether and how artificial intelligence (AI) and the large language model (LLM) of the company OpenAI (San Francisco, CA, USA), known under the name ChatGPT, can also help and support ethical decision making is increasingly becoming a matter of controversial debate. MATERIAL AND METHODS: Based on a case example, in which a female physician is confronted with ethical and legal issues and presents these to ChatGPT to come up with answers, the first indications of the strengths and weaknesses are ascertained. CONCLUSION: Due to the rapid technical development and access to ever increasing quantities of data, the utilization should be closely observed and evaluated.
