RESUMEN
BACKGROUND: Discharge instructions are a key form of documentation and patient communication in the time of transition from the emergency department (ED) to home. Discharge instructions are time-consuming and often underprioritized, especially in the ED, leading to discharge delays and possibly impersonal patient instructions. Generative artificial intelligence and large language models (LLMs) offer promising methods of creating high-quality and personalized discharge instructions; however, there exists a gap in understanding patient perspectives of LLM-generated discharge instructions. OBJECTIVE: We aimed to assess the use of LLMs such as ChatGPT in synthesizing accurate and patient-accessible discharge instructions in the ED. METHODS: We synthesized 5 unique, fictional ED encounters to emulate real ED encounters that included a diverse set of clinician history, physical notes, and nursing notes. These were passed to GPT-4 in Azure OpenAI Service (Microsoft) to generate LLM-generated discharge instructions. Standard discharge instructions were also generated for each of the 5 unique ED encounters. All GPT-generated and standard discharge instructions were then formatted into standardized after-visit summary documents. These after-visit summaries containing either GPT-generated or standard discharge instructions were randomly and blindly administered to Amazon MTurk respondents representing patient populations through Amazon MTurk Survey Distribution. Discharge instructions were assessed based on metrics of interpretability of significance, understandability, and satisfaction. RESULTS: Our findings revealed that survey respondents' perspectives regarding GPT-generated and standard discharge instructions were significantly (P=.01) more favorable toward GPT-generated return precautions, and all other sections were considered noninferior to standard discharge instructions. Of the 156 survey respondents, GPT-generated discharge instructions were assigned favorable ratings, "agree" and "strongly agree," more frequently along the metric of interpretability of significance in discharge instruction subsections regarding diagnosis, procedures, treatment, post-ED medications or any changes to medications, and return precautions. Survey respondents found GPT-generated instructions to be more understandable when rating procedures, treatment, post-ED medications or medication changes, post-ED follow-up, and return precautions. Satisfaction with GPT-generated discharge instruction subsections was the most favorable in procedures, treatment, post-ED medications or medication changes, and return precautions. Wilcoxon rank-sum test of Likert responses revealed significant differences (P=.01) in the interpretability of significant return precautions in GPT-generated discharge instructions compared to standard discharge instructions but not for other evaluation metrics and discharge instruction subsections. CONCLUSIONS: This study demonstrates the potential for LLMs such as ChatGPT to act as a method of augmenting current documentation workflows in the ED to reduce the documentation burden of physicians. The ability of LLMs to provide tailored instructions for patients by improving readability and making instructions more applicable to patients could improve upon the methods of communication that currently exist.
Asunto(s)
Servicio de Urgencia en Hospital , Alta del Paciente , Humanos , Servicio de Urgencia en Hospital/estadística & datos numéricos , Alta del Paciente/estadística & datos numéricos , Femenino , Masculino , Encuestas y Cuestionarios , Adulto , Persona de Mediana Edad , Inteligencia ArtificialRESUMEN
The objective of this study is to use statistical techniques for the identification of transition points along the life course, aiming to identify fundamental changes in patient multimorbidity burden across phases of clinical care. This retrospective cohort analysis utilized 5.2 million patient encounters from 2013 to 2022, collected from a large academic institution and its affiliated hospitals. Structured information was systematically gathered for each encounter and three methodologies - clustering analysis, False Nearest Neighbor, and transitivity analysis - were employed to pinpoint transitions in patients' clinical phase. Clustering analysis identified transition points at age 2, 17, 41, and 66, FNN at 4.27, 5.83, 5.85, 14.12, 20.62, 24.30, 25.10, 29.08, 33.12, 35.7, 38.69, 55.66, 70.03, and transitivity analysis at 7.27, 23.58, 29.04, 35.00, 61.29, 67.03, 77.11. Clustering analysis identified transition points that align with the current clinical gestalt of pediatric, adult, and geriatric phases of care. Notably, over half of the transition points identified by FNN and transitivity analysis were between ages 20 and 40, a population that is traditionally considered to be clinically homogeneous. Few transition points were identified between ages 3 and 17. Despite large social and developmental transition at those ages, the burden of multimorbidities may be consistent across the age range. Transition points derived through unsupervised machine learning approaches identify changes in the clinical phase that align with true differences in underlying multimorbidity burden. These transitions may be different from conventional pediatric and geriatric phases, which are often influenced by policy rather than clinical changes.
RESUMEN
INTRODUCTION: Multimorbidity rates are both increasing in prevalence across age ranges, and also increasing in diagnostic importance within and outside the family medicine clinic. Here we aim to describe the course of multimorbidity across the lifespan. METHODS: This was a retrospective cohort study across 211,953 patients from a large northeastern health care system. Past medical histories were collected in the form of ICD-10 diagnostic codes. Rates of multimorbidity were calculated from comorbid diagnoses defined from the ICD10 codes identified in the past medical histories. RESULTS: We identify 4 main age groups of diagnosis and multimorbidity. Ages 0 to 10 contain diagnoses which are infectious or respiratory, whereas ages 10 to 40 are related to mental health. From ages 40 to 70 there is an emergence of alcohol use disorders and cardiometabolic disorders. And ages 70 to 90 are predominantly long-term sequelae of the most common cardiometabolic disorders. The mortality of the whole population over the study period was 5.7%, whereas the multimorbidity with the highest mortality across the study period was Circulatory Disorders-Circulatory Disorders at 23.1%. CONCLUSION: The results from this study provide a comparison for the presence of multimorbidity within age cohorts longitudinally across the population. These patterns of comorbidity can assist in the allocation to practice resources that will best support the common conditions that patients need assistance with, especially as the patients transition between pediatric, adult, and geriatric care. Future work examining and comparing multimorbidity indices is warranted.
Asunto(s)
Medicina Familiar y Comunitaria , Multimorbilidad , Humanos , Estudios Retrospectivos , Anciano , Adulto , Persona de Mediana Edad , Adolescente , Anciano de 80 o más Años , Medicina Familiar y Comunitaria/estadística & datos numéricos , Masculino , Femenino , Adulto Joven , Niño , Preescolar , Lactante , Recién Nacido , Factores de Edad , Prevalencia , New England/epidemiologíaRESUMEN
Objectives: This study presents a design framework to enhance the accuracy by which large language models (LLMs), like ChatGPT can extract insights from clinical notes. We highlight this framework via prompt refinement for the automated determination of HEART (History, ECG, Age, Risk factors, Troponin risk algorithm) scores in chest pain evaluation. Methods: We developed a pipeline for LLM prompt testing, employing stochastic repeat testing and quantifying response errors relative to physician assessment. We evaluated the pipeline for automated HEART score determination across a limited set of 24 synthetic clinical notes representing four simulated patients. To assess whether iterative prompt design could improve the LLMs' ability to extract complex clinical concepts and apply rule-based logic to translate them to HEART subscores, we monitored diagnostic performance during prompt iteration. Results: Validation included three iterative rounds of prompt improvement for three HEART subscores with 25 repeat trials totaling 1200 queries each for GPT-3.5 and GPT-4. For both LLM models, from initial to final prompt design, there was a decrease in the rate of responses with erroneous, non-numerical subscore answers. Accuracy of numerical responses for HEART subscores (discrete 0-2 point scale) improved for GPT-4 from the initial to final prompt iteration, decreasing from a mean error of 0.16-0.10 (95% confidence interval: 0.07-0.14) points. Conclusion: We established a framework for iterative prompt design in the clinical space. Although the results indicate potential for integrating LLMs in structured clinical note analysis, translation to real, large-scale clinical data with appropriate data privacy safeguards is needed.
RESUMEN
Algorithms are a ubiquitous part of modern life. Despite being a component of medicine since early efforts to deploy computers in medicine, clinicians' resistance to using decision support and use algorithms to address cognitive biases has been limited. This resistance is not just limited to the use of algorithmic clinical decision support, but also evidence and stochastic reasoning and the implications of the forcing function of the electronic medical record. Physician resistance to algorithmic support in clinical decision making is in stark contrast to their general acceptance of algorithmic support in other aspects of life.
Asunto(s)
Algoritmos , Toma de Decisiones Clínicas , Sistemas de Apoyo a Decisiones Clínicas , Registros Electrónicos de Salud , Humanos , Médicos/psicología , Actitud del Personal de SaludRESUMEN
[This corrects the article DOI: 10.2196/45312.].
RESUMEN
OBJECTIVE: We aimed to discover computationally-derived phenotypes of opioid-related patient presentations to the ED via clinical notes and structured electronic health record (EHR) data. METHODS: This was a retrospective study of ED visits from 2013-2020 across ten sites within a regional healthcare network. We derived phenotypes from visits for patients ≥18 years of age with at least one prior or current documentation of an opioid-related diagnosis. Natural language processing was used to extract clinical entities from notes, which were combined with structured data within the EHR to create a set of features. We performed latent dirichlet allocation to identify topics within these features. Groups of patient presentations with similar attributes were identified by cluster analysis. RESULTS: In total 82,577 ED visits met inclusion criteria. The 30 topics were discovered ranging from those related to substance use disorder, chronic conditions, mental health, and medical management. Clustering on these topics identified nine unique cohorts with one-year survivals ranging from 84.2-96.8%, rates of one-year ED returns from 9-34%, rates of one-year opioid event 10-17%, rates of medications for opioid use disorder from 17-43%, and a median Carlson comorbidity index of 2-8. Two cohorts of phenotypes were identified related to chronic substance use disorder, or acute overdose. CONCLUSIONS: Our results indicate distinct phenotypic clusters with varying patient-oriented outcomes which provide future targets better allocation of resources and therapeutics. This highlights the heterogeneity of the overall population, and the need to develop targeted interventions for each population.
Asunto(s)
Analgésicos Opioides , Trastornos Relacionados con Opioides , Humanos , Analgésicos Opioides/efectos adversos , Estudios Retrospectivos , Servicio de Urgencia en Hospital , FenotipoRESUMEN
Large language models (LLMs) such as ChatGPT have sparked extensive discourse within the medical education community, spurring both excitement and apprehension. Written from the perspective of medical students, this editorial offers insights gleaned through immersive interactions with ChatGPT, contextualized by ongoing research into the imminent role of LLMs in health care. Three distinct positive use cases for ChatGPT were identified: facilitating differential diagnosis brainstorming, providing interactive practice cases, and aiding in multiple-choice question review. These use cases can effectively help students learn foundational medical knowledge during the preclinical curriculum while reinforcing the learning of core Entrustable Professional Activities. Simultaneously, we highlight key limitations of LLMs in medical education, including their insufficient ability to teach the integration of contextual and external information, comprehend sensory and nonverbal cues, cultivate rapport and interpersonal interaction, and align with overarching medical education and patient care goals. Through interacting with LLMs to augment learning during medical school, students can gain an understanding of their strengths and weaknesses. This understanding will be pivotal as we navigate a health care landscape increasingly intertwined with LLMs and artificial intelligence.
RESUMEN
BACKGROUND: The accelerating pace of biomedical publication has made it impractical to manually, systematically identify papers containing specific information and extract this information. This is especially challenging when the information itself resides beyond titles or abstracts. For emerging science, with a limited set of known papers of interest and an incomplete information model, this is of pressing concern. A timely example in retrospect is the identification of immune signatures (coherent sets of biomarkers) driving differential SARS-CoV-2 infection outcomes. IMPLEMENTATION: We built a classifier to identify papers containing domain-specific information from the document embeddings of the title and abstract. To train this classifier with limited data, we developed an iterative process leveraging pre-trained SPECTER document embeddings, SVM classifiers and web-enabled expert review to iteratively augment the training set. This training set was then used to create a classifier to identify papers containing domain-specific information. Finally, information was extracted from these papers through a semi-automated system that directly solicited the paper authors to respond via a web-based form. RESULTS: We demonstrate a classifier that retrieves papers with human COVID-19 immune signatures with a positive predictive value of 86%. The type of immune signature (e.g., gene expression vs. other types of profiling) was also identified with a positive predictive value of 74%. Semi-automated queries to the corresponding authors of these publications requesting signature information achieved a 31% response rate. CONCLUSIONS: Our results demonstrate the efficacy of using a SVM classifier with document embeddings of the title and abstract, to retrieve papers with domain-specific information, even when that information is rarely present in the abstract. Targeted author engagement based on classifier predictions offers a promising pathway to build a semi-structured representation of such information. Through this approach, partially automated literature mining can help rapidly create semi-structured knowledge repositories for automatic analysis of emerging health threats.
Asunto(s)
COVID-19 , Humanos , SARS-CoV-2RESUMEN
Physician progress notes are frequently organized into Subjective, Objective, Assessment, and Plan (SOAP) sections. The Assessment section synthesizes information recorded in the Subjective and Objective sections, and the Plan section documents tests and treatments to narrow the differential diagnosis and manage symptoms. Classifying the relationship between the Assessment and Plan sections has been suggested to provide valuable insight into clinical reasoning. In this work, we use a novel human-in-the-loop pipeline to classify the relationships between the Assessment and Plan sections of SOAP notes as a part of the n2c2 2022 Track 3 Challenge. In particular, we use a clinical information model constructed from both the entailment logic expected from the aforementioned Challenge and the problem-oriented medical record. This information model is used to label named entities as primary and secondary problems/symptoms, events and complications in all four SOAP sections. We iteratively train separate Named Entity Recognition models and use them to annotate entities in all notes/sections. We fine-tune a downstream RoBERTa-large model to classify the Assessment-Plan relationship. We evaluate multiple language model architectures, preprocessing parameters, and methods of knowledge integration, achieving a maximum macro-F1 score of 82.31%. Our initial model achieves top-2 performance during the challenge (macro-F1: 81.52%, competitors' macro-F1 range: 74.54%-82.12%). We improved our model by incorporating post-challenge annotations (S&O sections), outperforming the top model from the Challenge. We also used Shapley additive explanations to investigate the extent of language model clinical logic, under the lens of our clinical information model. We find that the model often uses shallow heuristics and nonspecific attention when making predictions, suggesting language model knowledge integration requires further research.
Asunto(s)
Médicos , Humanos , Atención , Registros Electrónicos de Salud , Registros , Procesamiento de Lenguaje NaturalRESUMEN
BACKGROUND: Chat Generative Pre-trained Transformer (ChatGPT) is a 175-billion-parameter natural language processing model that can generate conversation-style responses to user input. OBJECTIVE: This study aimed to evaluate the performance of ChatGPT on questions within the scope of the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 exams, as well as to analyze responses for user interpretability. METHODS: We used 2 sets of multiple-choice questions to evaluate ChatGPT's performance, each with questions pertaining to Step 1 and Step 2. The first set was derived from AMBOSS, a commonly used question bank for medical students, which also provides statistics on question difficulty and the performance on an exam relative to the user base. The second set was the National Board of Medical Examiners (NBME) free 120 questions. ChatGPT's performance was compared to 2 other large language models, GPT-3 and InstructGPT. The text output of each ChatGPT response was evaluated across 3 qualitative metrics: logical justification of the answer selected, presence of information internal to the question, and presence of information external to the question. RESULTS: Of the 4 data sets, AMBOSS-Step1, AMBOSS-Step2, NBME-Free-Step1, and NBME-Free-Step2, ChatGPT achieved accuracies of 44% (44/100), 42% (42/100), 64.4% (56/87), and 57.8% (59/102), respectively. ChatGPT outperformed InstructGPT by 8.15% on average across all data sets, and GPT-3 performed similarly to random chance. The model demonstrated a significant decrease in performance as question difficulty increased (P=.01) within the AMBOSS-Step1 data set. We found that logical justification for ChatGPT's answer selection was present in 100% of outputs of the NBME data sets. Internal information to the question was present in 96.8% (183/189) of all questions. The presence of information external to the question was 44.5% and 27% lower for incorrect answers relative to correct answers on the NBME-Free-Step1 (P<.001) and NBME-Free-Step2 (P=.001) data sets, respectively. CONCLUSIONS: ChatGPT marks a significant improvement in natural language processing models on the tasks of medical question answering. By performing at a greater than 60% threshold on the NBME-Free-Step-1 data set, we show that the model achieves the equivalent of a passing score for a third-year medical student. Additionally, we highlight ChatGPT's capacity to provide logic and informational context across the majority of answers. These facts taken together make a compelling case for the potential applications of ChatGPT as an interactive medical education tool to support learning.
RESUMEN
BACKGROUND: With the advent of competency-based medical education, as well as Canadian efforts to include clinical informatics within undergraduate medical education, competency frameworks in the United States have not emphasized the skills associated with clinical informatics pertinent to the broader practice of medicine. OBJECTIVE: By examining the competency frameworks with which undergraduate medical education in clinical informatics has been developed in Canada and the United States, we hypothesized that there is a gap: the lack of a unified competency set and frame for clinical informatics education across North America. METHODS: We performed directional competency mapping between Canadian and American graduate clinical informatics competencies and general graduate medical education competencies. Directional competency mapping was performed between Canadian roles and American common program requirements using keyword matching at the subcompetency and enabling competency levels. In addition, for general graduate medical education competencies, the Physician Competency Reference Set developed for the Liaison Committee on Medical Education was used as a direct means of computing the ontological overlap between competency frameworks. RESULTS: Upon mapping Canadian roles to American competencies via both undergraduate and graduate medical education competency frameworks, the difference in focus between the 2 countries can be thematically described as a difference between the concepts of clinical and management reasoning. CONCLUSIONS: We suggest that the development or deployment of informatics competencies in undergraduate medical education should focus on 3 items: the teaching of diagnostic reasoning, such that the information tasks that comprise both clinical and management reasoning can be discussed; precision medical education, where informatics can provide for more fine-grained evaluation; and assessment methods to support traditional pedagogical efforts (both at the bedside and beyond). Assessment using cases or structured assessments (eg, Objective Structured Clinical Examinations) would help students draw parallels between clinical informatics and fundamental clinical subjects and would better emphasize the cognitive techniques taught through informatics.
RESUMEN
Undergraduate medical education serves as a foundation for the medical student to develop the skills of a generalist physician. Given the "blurring" of the demarcations between childhood and adulthood and the increased scope of pediatric practice, an extra layer has been added to medical education which seeks to address care across the lifespan. While approaches have been developed to teach this layer, clerkship reform has not focused on advancing the clinical science of adolescence. Furthermore, as we look towards the vanguard of entrustable professional activities (EPA), specific attention to transition care for the adolescent has seen minimal attention. Drawing on prior examples of curriculum integration between specialties as well as solutions to complex care management from clinical reasoning, we suggest that attention to the development of the generalist physician requires attention to the combined medicine-pediatrics specialty.
RESUMEN
Identifying patient risk factors leading to adverse opioid-related events (AOEs) may enable targeted risk-based interventions, uncover potential causal mechanisms, and enhance prognosis. In this article, we aim to discover patient diagnosis, procedure, and medication event trajectories associated with AOEs using large-scale data mining methods. The individual temporally preceding factors associated with the highest relative risk (RR) for AOEs were opioid withdrawal therapy agents, toxic encephalopathy, problems related to housing and economic circumstances, and unspecified viral hepatitis, with RR of 33.4, 26.1, 19.9, and 18.7, respectively. Patient cohorts with a socioeconomic or mental health code had a larger RR for over 75% of all identified trajectories compared to the average population. By analyzing health trajectories leading to AOEs, we discover novel, temporally-connected combinations of diagnoses and health service events that significantly increase risk of AOEs, including natural histories marked by socioeconomic and mental health diagnoses.
Asunto(s)
Analgésicos Opioides , Analgésicos Opioides/efectos adversos , HumanosRESUMEN
STUDY OBJECTIVE: While patient-centered communication and shared decisionmaking are increasingly recognized as vital aspects of clinical practice, little is known about their characteristics in real-world emergency department (ED) settings. We constructed a natural language processing tool to identify patient-centered communication as documented in ED notes and to describe visit-level, site-level, and temporal patterns within a large health system. METHODS: This was a 2-part study involving (1) the development and validation of an natural language processing tool using regular expressions to identify shared decisionmaking and (2) a retrospective analysis using mixed effects logistic regression and trend analysis of shared decisionmaking and general patient discussion using the natural language processing tool to assess ED physician and advanced practice provider notes from 2013 to 2020. RESULTS: Compared to chart review of 600 ED notes, the accuracy rates of the natural language processing tool for identification of shared decisionmaking and general patient discussion were 96.7% (95% CI 94.9% to 97.9%) and 88.9% (95% confidence interval [CI] 86.1% to 91.3%), respectively. The natural language processing tool identified shared decisionmaking in 58,246 (2.2%) and general patient discussion in 590,933 (22%) notes. From 2013 to 2020, natural language processing-detected shared decisionmaking increased 300% and general patient discussion increased 50%. We observed higher odds of shared decisionmaking documentation among physicians versus advanced practice providers (odds ratio [OR] 1.14, 95% CI 1.07 to 1.23) and among female versus male patients (OR 1.13, 95% CI 1.11 to 1.15). Black patients had lower odds of shared decisionmaking (OR 0.8, 95% CI 0.84 to 0.88) compared with White patients. Shared decisionmaking and general patient discussion were also associated with higher levels of triage and commercial insurance status. CONCLUSION: In this study, we developed and validated an natural language processing tool using regular expressions to extract shared decisionmaking from ED notes and found multiple potential factors contributing to variation, including social, demographic, temporal, and presentation characteristics.
Asunto(s)
Comunicación , Toma de Decisiones Conjunta , Registros Electrónicos de Salud , Medicina de Emergencia/normas , Procesamiento de Lenguaje Natural , Relaciones Médico-Paciente , Adolescente , Adulto , Anciano , Anciano de 80 o más Años , Niño , Preescolar , Femenino , Humanos , Lactante , Recién Nacido , Masculino , Persona de Mediana Edad , Estudios Retrospectivos , Encuestas y Cuestionarios , Adulto JovenRESUMEN
BACKGROUND: Research shows that female trainees experience evaluation penalties for gender non-conforming behaviour during medical training. Studies of medical education evaluations and performance scores do reflect a gender bias, though studies are of varying methodology and results have not been consistent. OBJECTIVE: We sought to examine the differences in word use, competency themes and length within written evaluations of internal medicine residents at scale, considering the impact of both faculty and resident gender. We hypothesised that female internal medicine residents receive more negative feedback, and different thematic feedback than male residents. METHODS: This study utilised a corpus of 3864 individual responses to positive and negative questions over the course of six years (2012-2018) within Yale University School of Medicine's internal medicine residency. Researchers developed a sentiment model to assess the valence of evaluation responses. We then used natural language processing (NLP) to evaluate whether female versus male residents received more positive or negative feedback and if that feedback focussed on different Accreditation Council for Graduate Medical Education (ACGME) core competencies based on their gender. Evaluator-evaluatee gender dyad was analysed to see how it impacted quantity and quality of feedback. RESULTS: We found that female and male residents did not have substantively different numbers of positive or negative comments. While certain competencies were discussed more than others, gender did not seem to influence which competencies were discussed. Neither gender trainee received more written feedback, though female evaluators tended to write longer evaluations. CONCLUSIONS: We conclude that when examined at scale, quantitative gender differences are not as prevalent as has been seen in qualitative work. We suggest that further investigation of linguistic phenomena (such as context) is warranted to reconcile this finding with prior work.
Asunto(s)
Internado y Residencia , Sexismo , Competencia Clínica , Educación de Postgrado en Medicina , Femenino , Humanos , Masculino , Procesamiento de Lenguaje NaturalRESUMEN
Buprenorphine (BUP) can safely and effectively reduce craving, overdose, and mortality rates in people with opioid use disorder (OUD). However, adoption of ED-initiation of BUP has been slow partly due to physician perception this practice is too complex and disruptive. We report progress of the ongoing EMBED (EMergency department-initiated BuprenorphinE for opioid use Disorder) project. This project is a five-year collaboration across five healthcare systems with the goal to develop, integrate, study, and disseminate user-centered Clinical Decision Support (CDS) to promote the adoption of Emergency Department (ED)-initiation of buprenorphine/naloxone (BUP) into routine emergency care. Soon to enter its third year, the project has already completed multiple milestones to achieve its goals including (1) user-centered design of the CDS prototype, (2) integration of the CDS into an automated electronic health record (EHR) workflow, (3) data coordination including derivation and validation of an EHR-based computable phenotype, (4) meeting all ethical and regulatory requirements to achieve a waiver of informed consent, (5) pilot testing of the intervention at a single site, and (6) launching a parallel group-randomized 18-month pragmatic trial in 20 EDs across 5 healthcare systems. Pilot testing of the intervention in a single ED was associated with increased rates of ED-initiated BUP and naloxone prescribing and a doubling of the number of unique physicians adopting the practice. The ongoing multi-center pragmatic trial will assess the intervention's effectiveness, scalability, and generalizability with a goal to shift the emergency care paradigm for OUD towards early identification and treatment. TRIAL REGISTRATION: Clinicaltrials.gov # NCT03658642.