Chat-ePRO: Development and pilot study of an electronic patient-reported outcomes system based on ChatGPT.
J Biomed Inform
; 154: 104651, 2024 Jun.
Article
em En
| MEDLINE
| ID: mdl-38703936
ABSTRACT
OBJECTIVE:
Chatbots have the potential to improve user compliance in electronic Patient-Reported Outcome (ePRO) system. Compared to rule-based chatbots, Large Language Model (LLM) offers advantages such as simplifying the development process and increasing conversational flexibility. However, there is currently a lack of practical applications of LLMs in ePRO systems. Therefore, this study utilized ChatGPT to develop the Chat-ePRO system and designed a pilot study to explore the feasibility of building an ePRO system based on LLM. MATERIALS ANDMETHODS:
This study employed prompt engineering and offline knowledge distillation to design a dialogue algorithm and built the Chat-ePRO system on the WeChat Mini Program platform. In order to compare Chat-ePRO with the form-based ePRO and rule-based chatbot ePRO used in previous studies, we conducted a pilot study applying the three ePRO systems sequentially at the Sir Run Run Shaw Hospital to collect patients' PRO data.RESULT:
Chat-ePRO is capable of correctly generating conversation based on PRO forms (success rate 95.7 %) and accurately extracting the PRO data instantaneously from conversation (Macro-F1 0.95). The majority of subjective evaluations from doctors (>70 %) suggest that Chat-ePRO is able to comprehend questions and consistently generate responses. Pilot study shows that Chat-ePRO demonstrates higher response rate (9/10, 90 %) and longer interaction time (10.86 s/turn) compared to the other two methods.CONCLUSION:
Our study demonstrated the feasibility of utilizing algorithms such as prompt engineering to drive LLM in completing ePRO data collection tasks, and validated that the Chat-ePRO system can effectively enhance patient compliance.Palavras-chave
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Algoritmos
/
Medidas de Resultados Relatados pelo Paciente
Limite:
Adult
/
Female
/
Humans
/
Male
/
Middle aged
Idioma:
En
Revista:
J Biomed Inform
Assunto da revista:
INFORMATICA MEDICA
Ano de publicação:
2024
Tipo de documento:
Article
País de afiliação:
China