Chat-ePRO: Development and pilot study of an electronic patient-reported outcomes system based on ChatGPT.

OBJECTIVE: Chatbots have the potential to improve user compliance in electronic Patient-Reported Outcome (ePRO) system. Compared to rule-based chatbots, Large Language Model (LLM) offers advantages such as simplifying the development process and increasing conversational flexibility. However, there is currently a lack of practical applications of LLMs in ePRO systems. Therefore, this study utilized ChatGPT to develop the Chat-ePRO system and designed a pilot study to explore the feasibility of building an ePRO system based on LLM. MATERIALS AND METHODS: This study employed prompt engineering and offline knowledge distillation to design a dialogue algorithm and built the Chat-ePRO system on the WeChat Mini Program platform. In order to compare Chat-ePRO with the form-based ePRO and rule-based chatbot ePRO used in previous studies, we conducted a pilot study applying the three ePRO systems sequentially at the Sir Run Run Shaw Hospital to collect patients' PRO data. RESULT: Chat-ePRO is capable of correctly generating conversation based on PRO forms (success rate: 95.7 %) and accurately extracting the PRO data instantaneously from conversation (Macro-F1: 0.95). The majority of subjective evaluations from doctors (>70 %) suggest that Chat-ePRO is able to comprehend questions and consistently generate responses. Pilot study shows that Chat-ePRO demonstrates higher response rate (9/10, 90 %) and longer interaction time (10.86 s/turn) compared to the other two methods. CONCLUSION: Our study demonstrated the feasibility of utilizing algorithms such as prompt engineering to drive LLM in completing ePRO data collection tasks, and validated that the Chat-ePRO system can effectively enhance patient compliance.

A configurable method for clinical quality measurement through electronic health records based on openEHR and CQL.

BACKGROUND: One of the primary obstacles to measure clinical quality is the lack of configurable solutions to make computers understand and compute clinical quality indicators. The paper presents a solution that can help clinical staff develop clinical quality measurement more easily and generate the corresponding data reports and visualization by a configurable method based on openEHR and Clinical Quality Language (CQL). METHODS: First, expression logic adopted from CQL was combined with openEHR to express clinical quality indicators. Archetype binding provides the clinical information models used in expression logic, terminology binding makes the medical concepts consistent used in clinical quality artifacts and metadata is regarded as the essential component for sharing and management. Then, a systematic approach was put forward to facilitate the development of clinical quality indicators and the generation of corresponding data reports and visualization. Finally, clinical physicians were invited to test our system and give their opinions. RESULTS: With the combination of openEHR and CQL, 64 indicators from Centers for Medicare & Medicaid Services (CMS) were expressed for verification and a complicated indicator was shown as an example. 68 indicators from 17 different scenes in the local environment were also expressed and computed in our system. A platform was built to support the development of indicators in a unified way. Also, an execution engine can parse and compute these indicators. Based on a clinical data repository (CDR), indicators were used to generate data reports and visualization and shown in a dashboard. CONCLUSION: Our method is capable of expressing clinical quality indicators formally. With the computer-interpretable indicators, a systematic approach can make it more easily to define clinical indicators and generate medical data reports and visualization, and facilitate the adoption of clinical quality measurements.

Automatic RadLex coding of Chinese structured radiology reports based on text similarity ensemble.

BACKGROUND: Standardized coding of plays an important role in radiology reports' secondary use such as data analytics, data-driven decision support, and personalized medicine. RadLex, a standard radiological lexicon, can reduce subjective variability and improve clarity in radiology reports. RadLex coding of radiology reports is widely used in many countries, but translation and localization of RadLex in China are far from being established. Although automatic RadLex coding is a common way for non-standard radiology reports, the high-accuracy cross-language RadLex coding is hardly achieved due to the limitation of up-to-date auto-translation and text similarity algorithms and still requires further research. METHODS: We present an effective approach that combines a hybrid translation and a Multilayer Perceptron weighting text similarity ensemble algorithm for automatic RadLex coding of Chinese structured radiology reports. Firstly, a hybrid way to integrate Google neural machine translation and dictionary translation helps to optimize the translation of Chinese radiology phrases to English. The dictionary is made up of 21,863 Chinese-English radiological term pairs extracted from several free medical dictionaries. Secondly, four typical text similarity algorithms are introduced, which are Levenshtein distance, Jaccard similarity coefficient, Word2vec Continuous bag-of-words model, and WordNet Wup similarity algorithms. Lastly, the Multilayer Perceptron model has been used to synthesize the contextual, lexical, character and syntactical information of four text similarity algorithms to promote precision, in which four similarity scores of two terms are taken as input and the output presents whether the two terms are synonyms. RESULTS: The results show the effectiveness of the approach with an F1-score of 90.15%, a precision of 91.78% and a recall of 88.59%. The hybrid translation algorithm has no negative effect on the final coding, F1-score has increased by 21.44% and 8.12% compared with the GNMT algorithm and dictionary translation. Compared with the single similarity, the result of the MLP weighting similarity algorithm is satisfactory that has a 4.48% increase compared with the best single similarity algorithm, WordNet Wup. CONCLUSIONS: The paper proposed an innovative automatic cross-language RadLex coding approach to solve the standardization of Chinese structured radiology reports, that can be taken as a reference to automatic cross-language coding.

Development of an openEHR Template for COVID-19 Based on Clinical Guidelines.

BACKGROUND: The coronavirus disease (COVID-19) was discovered in China in December 2019. It has developed into a threatening international public health emergency. With the exception of China, the number of cases continues to increase worldwide. A number of studies about disease diagnosis and treatment have been carried out, and many clinically proven effective results have been achieved. Although information technology can improve the transferring of such knowledge to clinical practice rapidly, data interoperability is still a challenge due to the heterogeneous nature of hospital information systems. This issue becomes even more serious if the knowledge for diagnosis and treatment is updated rapidly as is the case for COVID-19. An open, semantic-sharing, and collaborative-information modeling framework is needed to rapidly develop a shared data model for exchanging data among systems. openEHR is such a framework and is supported by many open software packages that help to promote information sharing and interoperability. OBJECTIVE: This study aims to develop a shared data model based on the openEHR modeling approach to improve the interoperability among systems for the diagnosis and treatment of COVID-19. METHODS: The latest Guideline of COVID-19 Diagnosis and Treatment in China was selected as the knowledge source for modeling. First, the guideline was analyzed and the data items used for diagnosis and treatment, and management were extracted. Second, the data items were classified and further organized into domain concepts with a mind map. Third, searching was executed in the international openEHR Clinical Knowledge Manager (CKM) to find the existing archetypes that could represent the concepts. New archetypes were developed for those concepts that could not be found. Fourth, these archetypes were further organized into a template using Ocean Template Editor. Fifth, a test case of data exchanging between the clinical data repository and clinical decision support system based on the template was conducted to verify the feasibility of the study. RESULTS: A total of 203 data items were extracted from the guideline in China, and 16 domain concepts (16 leaf nodes in the mind map) were organized. There were 22 archetypes used to develop the template for all data items extracted from the guideline. All of them could be found in the CKM and reused directly. The archetypes and templates were reviewed and finally released in a public project within the CKM. The test case showed that the template can facilitate the data exchange and meet the requirements of decision support. CONCLUSIONS: This study has developed the openEHR template for COVID-19 based on the latest guideline from China using openEHR modeling methodology. It represented the capability of the methodology for rapidly modeling and sharing knowledge through reusing the existing archetypes, which is especially useful in a new and fast-changing area such as with COVID-19.

A Semi-Automatic Data Cleaning & Coding Tool for Chinese Clinical Data Standardization.

The clinical data often have limited usefulness because of the diversified expression. Chinese clinical data standardization can improve the usability of clinical data. The complexity of data cleaning and coding for Chinese clinical data prompted the turn of low-effective manual coding into the computer-aided tool. This study established the universal data cleaning and coding process and tool for Chinese clinical data standardization, which can greatly improve human efficiency. The process included the preprocessing, text similarity algorithm, and manual review. The standardization process proved effective for the diagnosis, drug, and examination data standardization task and can be used gradually in other clinical domains. The semi-automatic data cleaning and coding can reduce the half time for standardization, and it was used in hospitals in Beijing.

A Patient-Screening Tool for Clinical Research Based on Electronic Health Records Using OpenEHR: Development Study.

BACKGROUND: The widespread adoption of electronic health records (EHRs) has facilitated the secondary use of EHR data for clinical research. However, screening eligible patients from EHRs is a challenging task. The concepts in eligibility criteria are not completely matched with EHRs, especially derived concepts. The lack of high-level expression of Structured Query Language (SQL) makes it difficult and time consuming to express them. The openEHR Expression Language (EL) as a domain-specific language based on clinical information models shows promise to represent complex eligibility criteria. OBJECTIVE: The study aims to develop a patient-screening tool based on EHRs for clinical research using openEHR to solve concept mismatch and improve query performance. METHODS: A patient-screening tool based on EHRs using openEHR was proposed. It uses the advantages of information models and EL in openEHR to provide high-level expressions and improve query performance. First, openEHR archetypes and templates were chosen to define concepts called simple concepts directly from EHRs. Second, openEHR EL was used to generate derived concepts by combining simple concepts and constraints. Third, a hierarchical index corresponding to archetypes in Elasticsearch (ES) was generated to improve query performance for subqueries and join queries related to the derived concepts. Finally, we realized a patient-screening tool for clinical research. RESULTS: In total, 500 sentences randomly selected from 4691 eligibility criteria in 389 clinical trials on stroke from the Chinese Clinical Trial Registry (ChiCTR) were evaluated. An openEHR-based clinical data repository (CDR) in a grade A tertiary hospital in China was considered as an experimental environment. Based on these, 589 medical concepts were found in the 500 sentences. Of them, 513 (87.1%) concepts could be represented, while the others could not be, because of a lack of information models and coarse-grained requirements. In addition, our case study on 6 queries demonstrated that our tool shows better query performance among 4 cases (66.67%). CONCLUSIONS: We developed a patient-screening tool using openEHR. It not only helps solve concept mismatch but also improves query performance to reduce the burden on researchers. In addition, we demonstrated a promising solution for secondary use of EHR data using openEHR, which can be referenced by other researchers.

Extending CQL with openEHR to Express Clinical Quality Indicators.

Clinical Quality Language (CQL), a HL7 authoring language to express clinical quality indicators, provides the capability to express logic that is human readable yet structured enough for processing a query electronically. OpenEHR is a widely used, modeling methodology, but currently CQL cannot support openEHR archetypes which hinders its usage in openEHR environment. This paper presents a method to express and compute clinical quality indicators by extending CQL with openEHR archetypes. To verify the feasibility of this method, 64 indicators from the Centers for Medicare & Medicaid Services (CMS) and 118 indicators from local environment in China were utilized. The results show that those indicators can be well represented and computed in openEHR environment.