SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization.
J Biomed Inform
; 151: 104606, 2024 03.
Article
en En
| MEDLINE
| ID: mdl-38325698
ABSTRACT
Electronic health records (EHRs) store an extensive array of patient information, encompassing medical histories, diagnoses, treatments, and test outcomes. These records are crucial for enabling healthcare providers to make well-informed decisions regarding patient care. Summarizing clinical notes further assists healthcare professionals in pinpointing potential health risks and making better-informed decisions. This process contributes to reducing errors and enhancing patient outcomes by ensuring providers have access to the most pertinent and current patient data. Recent research has shown that incorporating instruction prompts with large language models (LLMs) substantially boosts the efficacy of summarization tasks. However, we show that this approach also leads to increased performance variance, resulting in significantly distinct summaries even when instruction prompts share similar meanings. To tackle this challenge, we introduce a model-agnostic Soft Prompt-BasedCalibration (SPeC) pipeline that employs soft prompts to lower variance while preserving the advantages of prompt-based summarization. Experimental findings on multiple clinical note tasks and LLMs indicate that our method not only bolsters performance but also effectively regulates variance across different LLMs, providing a more consistent and reliable approach to summarizing critical medical information.
Palabras clave
Texto completo:
1
Colección:
01-internacional
Base de datos:
MEDLINE
Contexto en salud:
1_ASSA2030
Problema de salud:
1_sistemas_informacao_saude
Asunto principal:
Procesamiento de Lenguaje Natural
/
Registros Electrónicos de Salud
Tipo de estudio:
Prognostic_studies
Límite:
Humans
Idioma:
En
Revista:
J Biomed Inform
Asunto de la revista:
INFORMATICA MEDICA
Año:
2024
Tipo del documento:
Article