Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study.

BACKGROUND: Emergency departments (EDs) are frequently overcrowded and increasingly used by nonurgent patients. Symptom checkers (SCs) offer on-demand access to disease suggestions and recommended actions, potentially improving overall patient flow. Contrary to the increasing use of SCs, there is a lack of supporting evidence based on direct patient use. OBJECTIVE: This study aimed to compare the diagnostic accuracy, safety, usability, and acceptance of 2 SCs, Ada and Symptoma. METHODS: A randomized, crossover, head-to-head, double-blinded study including consecutive adult patients presenting to the ED at University Hospital Erlangen. Patients completed both SCs, Ada and Symptoma. The primary outcome was the diagnostic accuracy of SCs. In total, 6 blinded independent expert raters classified diagnostic concordance of SC suggestions with the final discharge diagnosis as (1) identical, (2) plausible, or (3) diagnostically different. SC suggestions per patient were additionally classified as safe or potentially life-threatening, and the concordance of Ada's and physician-based triage category was assessed. Secondary outcomes were SC usability (5-point Likert-scale: 1=very easy to use to 5=very difficult to use) and SC acceptance net promoter score (NPS). RESULTS: A total of 450 patients completed the study between April and November 2021. The most common chief complaint was chest pain (160/437, 37%). The identical diagnosis was ranked first (or within the top 5 diagnoses) by Ada and Symptoma in 14% (59/437; 27%, 117/437) and 4% (16/437; 13%, 55/437) of patients, respectively. An identical or plausible diagnosis was ranked first (or within the top 5 diagnoses) by Ada and Symptoma in 58% (253/437; 75%, 329/437) and 38% (164/437; 64%, 281/437) of patients, respectively. Ada and Symptoma did not suggest potentially life-threatening diagnoses in 13% (56/437) and 14% (61/437) of patients, respectively. Ada correctly triaged, undertriaged, and overtriaged 34% (149/437), 13% (58/437), and 53% (230/437) of patients, respectively. A total of 88% (385/437) and 78% (342/437) of participants rated Ada and Symptoma as very easy or easy to use, respectively. Ada's NPS was -34 (55% [239/437] detractors; 21% [93/437] promoters) and Symptoma's NPS was -47 (63% [275/437] detractors and 16% [70/437]) promoters. CONCLUSIONS: Ada demonstrated a higher diagnostic accuracy than Symptoma, and substantially more patients would recommend Ada and assessed Ada as easy to use. The high number of unrecognized potentially life-threatening diagnoses by both SCs and inappropriate triage advice by Ada was alarming. Overall, the trustworthiness of SC recommendations appears questionable. SC authorization should necessitate rigorous clinical evaluation studies to prevent misdiagnoses, fatal triage advice, and misuse of scarce medical resources. TRIAL REGISTRATION: German Register of Clinical Trials DRKS00024830; https://drks.de/search/en/trial/DRKS00024830.

Psychometric properties of the Reintegration to Normal Living Index for sepsis survivors.

PURPOSE: Return to a normal state of living is a key patient-relevant outcome for sepsis survivors. The Reintegration to Normal Living Index (RNLI) assesses self-perceived participation in patients with chronic disease, but its psychometric properties have been analyzed neither for patients after sepsis nor in a German patient cohort. This study aims to analyze the psychometric properties of the German version of the RNLI in sepsis survivors. METHODS: In a prospective multicenter survey study, 287 sepsis survivors were interviewed 6 and 12 months after hospital discharge. Multiple-group categorical confirmatory factor analyses with three competing models were used to explore the factor structure of the RNLI. Concurrent validity was evaluated in relation to the EQ-5D-3L and the Barthel Index of Activities of Daily Living (ADL). RESULTS: Regarding structural validity, all models showed an acceptable model fit. Because of high correlation between the latent variables in the two-factor models (up to r = 0.969) and for reason of parsimony, we opted for the common factor model to analyze the concurrent validity. Our analyses showed moderate positive correlations between RNLI score and ADL score (r ≥ 0.630), EQ-5D-3L visual analogue scale (r ≥ 0.656) and EQ-5D-3L utility score (r ≥ 0.548). The reliability assessed by McDonald's Omega was 0.94. CONCLUSION: We found convincing evidence for good reliability, structural and concurrent validity of the RNLI in German sepsis survivors. We propose to use the RNLI in addition to generic health-related quality of life measures to assess the reintegration to normal living after sepsis.

Staff perspectives on the influence of patient characteristics on alarm management in the intensive care unit: a cross-sectional survey study.

BACKGROUND: High rates of clinical alarms in the intensive care unit can result in alarm fatigue among staff. Individualization of alarm thresholds is regarded as one measure to reduce non-actionable alarms. The aim of this study was to investigate staff's perceptions of alarm threshold individualization according to patient characteristics and disease status. METHODS: This is a cross-sectional survey study (February-July 2020). Intensive care nurses and physicians were sampled by convenience. Data was collected using an online questionnaire. RESULTS: Staff view the individualization of alarm thresholds in the monitoring of vital signs as important. The extent to which alarm thresholds are adapted from the normal range varies depending on the vital sign monitored, the reason for clinical deterioration, and the professional group asked. Vital signs used for hemodynamic monitoring (heart rate and blood pressure) were most subject to alarm individualizations. Staff are ambivalent regarding the integration of novel technological features into alarm management. CONCLUSIONS: All relevant stakeholders, including clinicians, hospital management, and industry, must collaborate to establish a "standard for individualization," moving away from ad hoc alarm management to an intelligent, data-driven alarm management. Making alarms meaningful and trustworthy again has the potential to mitigate alarm fatigue - a major cause of stress in clinical staff and considerable hazard to patient safety. TRIAL REGISTRATION: The study was registered at ClinicalTrials.gov (NCT03514173) on 02/05/2018.

Characteristics of Users and Nonusers of Symptom Checkers in Germany: Cross-Sectional Survey Study.

BACKGROUND: Previous studies have revealed that users of symptom checkers (SCs, apps that support self-diagnosis and self-triage) are predominantly female, are younger than average, and have higher levels of formal education. Little data are available for Germany, and no study has so far compared usage patterns with people's awareness of SCs and the perception of usefulness. OBJECTIVE: We explored the sociodemographic and individual characteristics that are associated with the awareness, usage, and perceived usefulness of SCs in the German population. METHODS: We conducted a cross-sectional online survey among 1084 German residents in July 2022 regarding personal characteristics and people's awareness and usage of SCs. Using random sampling from a commercial panel, we collected participant responses stratified by gender, state of residence, income, and age to reflect the German population. We analyzed the collected data exploratively. RESULTS: Of all respondents, 16.3% (177/1084) were aware of SCs and 6.5% (71/1084) had used them before. Those aware of SCs were younger (mean 38.8, SD 14.6 years, vs mean 48.3, SD 15.7 years), were more often female (107/177, 60.5%, vs 453/907, 49.9%), and had higher formal education levels (eg, 72/177, 40.7%, vs 238/907, 26.2%, with a university/college degree) than those unaware. The same observation applied to users compared to nonusers. It disappeared, however, when comparing users to nonusers who were aware of SCs. Among users, 40.8% (29/71) considered these tools useful. Those considering them useful reported higher self-efficacy (mean 4.21, SD 0.66, vs mean 3.63, SD 0.81, on a scale of 1-5) and a higher net household income (mean EUR 2591.63, SD EUR 1103.96 [mean US $2798.96, SD US $1192.28], vs mean EUR 1626.60, SD EUR 649.05 [mean US $1756.73, SD US $700.97]) than those who considered them not useful. More women considered SCs unhelpful (13/44, 29.5%) compared to men (4/26, 15.4%). CONCLUSIONS: Concurring with studies from other countries, our findings show associations between sociodemographic characteristics and SC usage in a German sample: users were on average younger, of higher socioeconomic status, and more commonly female compared to nonusers. However, usage cannot be explained by sociodemographic differences alone. It rather seems that sociodemographics explain who is or is not aware of the technology, but those who are aware of SCs are equally likely to use them, independently of sociodemographic differences. Although in some groups (eg, people with anxiety disorder), more participants reported to know and use SCs, they tended to perceive them as less useful. In other groups (eg, male participants), fewer respondents were aware of SCs, but those who used them perceived them to be more useful. Thus, SCs should be designed to fit specific user needs, and strategies should be developed to help reach individuals who could benefit but are not aware of SCs yet.

Triage Accuracy of Symptom Checker Apps: 5-Year Follow-up Evaluation.

BACKGROUND: Symptom checkers are digital tools assisting laypersons in self-assessing the urgency and potential causes of their medical complaints. They are widely used but face concerns from both patients and health care professionals, especially regarding their accuracy. A 2015 landmark study substantiated these concerns using case vignettes to demonstrate that symptom checkers commonly err in their triage assessment. OBJECTIVE: This study aims to revisit the landmark index study to investigate whether and how symptom checkers' capabilities have evolved since 2015 and how they currently compare with laypersons' stand-alone triage appraisal. METHODS: In early 2020, we searched for smartphone and web-based applications providing triage advice. We evaluated these apps on the same 45 case vignettes as the index study. Using descriptive statistics, we compared our findings with those of the index study and with publicly available data on laypersons' triage capability. RESULTS: We retrieved 22 symptom checkers providing triage advice. The median triage accuracy in 2020 (55.8%, IQR 15.1%) was close to that in 2015 (59.1%, IQR 15.5%). The apps in 2020 were less risk averse (odds 1.11:1, the ratio of overtriage errors to undertriage errors) than those in 2015 (odds 2.82:1), missing >40% of emergencies. Few apps outperformed laypersons in either deciding whether emergency care was required or whether self-care was sufficient. No apps outperformed the laypersons on both decisions. CONCLUSIONS: Triage performance of symptom checkers has, on average, not improved over the course of 5 years. It decreased in 2 use cases (advice on when emergency care is required and when no health care is needed for the moment). However, triage capability varies widely within the sample of symptom checkers. Whether it is beneficial to seek advice from symptom checkers depends on the app chosen and on the specific question to be answered. Future research should develop resources (eg, case vignette repositories) to audit the capabilities of symptom checkers continuously and independently and provide guidance on when and to whom they should be recommended.

Correction: Benchmarking Triage Capability of Symptom Checkers Against That of Medical Laypersons: Survey Study.

[This corrects the article DOI: 10.2196/24475.].

Benchmarking Triage Capability of Symptom Checkers Against That of Medical Laypersons: Survey Study.

BACKGROUND: Symptom checkers (SCs) are tools developed to provide clinical decision support to laypersons. Apart from suggesting probable diagnoses, they commonly advise when users should seek care (triage advice). SCs have become increasingly popular despite prior studies rating their performance as mediocre. To date, it is unclear whether SCs can triage better than those who might choose to use them. OBJECTIVE: This study aims to compare triage accuracy between SCs and their potential users (ie, laypersons). METHODS: On Amazon Mechanical Turk, we recruited 91 adults from the United States who had no professional medical background. In a web-based survey, the participants evaluated 45 fictitious clinical case vignettes. Data for 15 SCs that had processed the same vignettes were obtained from a previous study. As main outcome measures, we assessed the accuracy of the triage assessments made by participants and SCs for each of the three triage levels (ie, emergency care, nonemergency care, self-care) and overall, the proportion of participants outperforming each SC in terms of accuracy, and the risk aversion of participants and SCs by comparing the proportion of cases that were overtriaged. RESULTS: The mean overall triage accuracy was similar for participants (60.9%, SD 6.8%; 95% CI 59.5%-62.3%) and SCs (58%, SD 12.8%). Most participants outperformed all but 5 SCs. On average, SCs more reliably detected emergencies (80.6%, SD 17.9%) than laypersons did (67.5%, SD 16.4%; 95% CI 64.1%-70.8%). Although both SCs and participants struggled with cases requiring self-care (the least urgent triage category), SCs more often wrongly classified these cases as emergencies (43/174, 24.7%) compared with laypersons (56/1365, 4.10%). CONCLUSIONS: Most SCs had no greater triage capability than an average layperson, although the triage accuracy of the five best SCs was superior to the accuracy of most participants. SCs might improve early detection of emergencies but might also needlessly increase resource utilization in health care. Laypersons sometimes require support in deciding when to rely on self-care but it is in that very situation where SCs perform the worst. Further research is needed to determine how to best combine the strengths of humans and SCs.

Improvements in Patient Monitoring in the Intensive Care Unit: Survey Study.

BACKGROUND: Due to demographic change and, more recently, coronavirus disease (COVID-19), the importance of modern intensive care units (ICU) is becoming apparent. One of the key components of an ICU is the continuous monitoring of patients' vital parameters. However, existing advances in informatics, signal processing, or engineering that could alleviate the burden on ICUs have not yet been applied. This could be due to the lack of user involvement in research and development. OBJECTIVE: This study focused on the satisfaction of ICU staff with current patient monitoring and their suggestions for future improvements. We aimed to identify aspects of monitoring that interrupt patient care, display devices for remote monitoring, use cases for artificial intelligence (AI), and whether ICU staff members are willing to improve their digital literacy or contribute to the improvement of patient monitoring. We further aimed to identify differences in the responses of different professional groups. METHODS: This survey study was performed with ICU staff from 4 ICUs of a German university hospital between November 2019 and January 2020. We developed a web-based 36-item survey questionnaire, by analyzing a preceding qualitative interview study with ICU staff, about the clinical requirements of future patient monitoring. Statistical analyses of questionnaire results included median values with their bootstrapped 95% confidence intervals, and chi-square tests to compare the distributions of item responses of the professional groups. RESULTS: In total, 86 of the 270 ICU physicians and nurses completed the survey questionnaire. The majority stated they felt confident using the patient monitoring equipment, but that high rates of false-positive alarms and the many sensor cables interrupted patient care. Regarding future improvements, respondents asked for wireless sensors, a reduction in the number of false-positive alarms, and hospital standard operating procedures for alarm management. Responses to the display devices proposed for remote patient monitoring were divided. Most respondents indicated it would be useful for earlier alerting or when they were responsible for multiple wards. AI for ICUs would be useful for early detection of complications and an increased risk of mortality; in addition, the AI could propose guidelines for therapy and diagnostics. Transparency, interoperability, usability, and staff training were essential to promote the use of AI. The majority wanted to learn more about new technologies for the ICU and required more time for learning. Physicians had fewer reservations than nurses about AI-based intelligent alarm management and using mobile phones for remote monitoring. CONCLUSIONS: This survey study of ICU staff revealed key improvements for patient monitoring in intensive care medicine. Hospital providers and medical device manufacturers should focus on reducing false alarms, implementing hospital alarm standard operating procedures, introducing wireless sensors, preparing for the use of AI, and enhancing the digital literacy of ICU staff. Our results may contribute to the user-centered transfer of digital technologies into practice to alleviate challenges in intensive care medicine. TRIAL REGISTRATION: ClinicalTrials.gov NCT03514173; https://clinicaltrials.gov/ct2/show/NCT03514173.

Home Hospital Outcomes for Acute Decompensated Heart Failure and Factors Associated With Escalation of Care.

BACKGROUND: Overall outcomes and the escalation rate for home hospital admissions for heart failure (HF) are not known. We report overall outcomes, predict escalation, and describe care provided after escalation among patients admitted to home hospital for HF. METHODS: Our retrospective analysis included all patients admitted for HF to 2 home hospital programs in Massachusetts between February 2020 and October 2022. Escalation of care was defined as transfer to an inpatient hospital setting (emergency department, inpatient medical unit) for at least 1 overnight stay. Unexpected mortality was defined as mortality excluding those who desired to pass away at home on admission or transitioned to hospice. We performed the least absolute shrinkage and selection operator logistic regression to predict escalation. RESULTS: We included 437 hospitalizations; patients had a median age of 80 (interquartile range, 69-89) years, 58.1% were women, and 64.8% were White. Of the cohort, 29.2% had reduced ejection fraction, 50.9% had chronic kidney disease, and 60.6% had atrial fibrillation. Median admission Get With The Guidelines HF score was 39 (interquartile range, 35-45; 1%-5% predicted inpatient mortality). Escalation occurred in 10.3% of hospitalizations. Thirty-day readmission occurred in 15.1%, 90-day readmission occurred in 33.8%, and 6-month mortality occurred in 11.5%. There was no unexpected mortality during home hospitalization. Patients who experienced escalation had significantly longer median length of stays (19 versus 7.5 days, P<0.001). The most common reason for escalation was progressive renal dysfunction (36.2%). A low mean arterial pressure at the time of admission to home hospital was the most significant predictor of escalation in the least absolute shrinkage and selection operator regression. CONCLUSIONS: About 1 in 10 home hospital patients with HF required escalation; none had unexpected mortality. Patients requiring escalation had longer length of stays. A low mean arterial pressure at the time of admission to home hospital was the most important predictor of escalation of care in the least absolute shrinkage and selection operator logistic regression model.

How suitable are clinical vignettes for the evaluation of symptom checker apps? A test theoretical perspective.

Objective: To evaluate the ability of case vignettes to assess the performance of symptom checker applications and to suggest refinements to the methodology used in case vignette-based audit studies. Methods: We re-analyzed the publicly available data of two prominent case vignette-based symptom checker audit studies by calculating common metrics of test theory. Furthermore, we developed a new metric, the Capability Comparison Score (CCS), which compares symptom checker capability while controlling for the difficulty of the set of cases each symptom checker evaluated. We then scrutinized whether applying test theory and the CCS altered the performance ranking of the investigated symptom checkers. Results: In both studies, most symptom checkers changed their rank order when adjusting the triage capability for item difficulty (ID) with the CCS. The previously reported triage accuracies commonly overestimated the capability of symptom checkers because they did not account for the fact that symptom checkers tend to selectively appraise easier cases (i.e., with high ID values). Also, many case vignettes in both studies showed insufficient (very low and even negative) values of item-total correlation (ITC), suggesting that individual items or the composition of item sets are of low quality. Conclusions: A test-theoretic perspective helps identify previously undetected threats to the validity of case vignette-based symptom checker assessments and provides guidance and specific metrics to improve the quality of case vignettes, in particular by controlling for the difficulty of the vignettes an app was (not) able to evaluate correctly. Such measures might prove more meaningful than accuracy alone for the competitive assessment of symptom checkers. Our approach helps elaborate and standardize the methodology used for appraising symptom checker capability, which, ultimately, may yield more reliable results.

The Triage Capability of Laypersons: Retrospective Exploratory Analysis.

BACKGROUND: Although medical decision-making may be thought of as a task involving health professionals, many decisions, including critical health-related decisions are made by laypersons alone. Specifically, as the first step to most care episodes, it is the patient who determines whether and where to seek health care (triage). Overcautious self-assessments (ie, overtriaging) may lead to overutilization of health care facilities and overcrowded emergency departments, whereas imprudent decisions (ie, undertriaging) constitute a risk to the patient's health. Recently, patient-facing decision support systems, commonly known as symptom checkers, have been developed to assist laypersons in these decisions. OBJECTIVE: The purpose of this study is to identify factors influencing laypersons' ability to self-triage and their risk averseness in self-triage decisions. METHODS: We analyzed publicly available data on 91 laypersons appraising 45 short fictitious patient descriptions (case vignettes; N=4095 appraisals). Using signal detection theory and descriptive and inferential statistics, we explored whether the type of medical decision laypersons face, their confidence in their decision, and sociodemographic factors influence their triage accuracy and the type of errors they make. We distinguished between 2 decisions: whether emergency care was required (decision 1) and whether self-care was sufficient (decision 2). RESULTS: The accuracy of detecting emergencies (decision 1) was higher (mean 82.2%, SD 5.9%) than that of deciding whether any type of medical care is required (decision 2, mean 75.9%, SD 5.25%; t>90=8.4; P<.001; Cohen d=0.9). Sensitivity for decision 1 was lower (mean 67.5%, SD 16.4%) than its specificity (mean 89.6%, SD 8.6%) whereas sensitivity for decision 2 was higher (mean 90.5%, SD 8.3%) than its specificity (mean 46.7%, SD 15.95%). Female participants were more risk averse and overtriaged more often than male participants, but age and level of education showed no association with participants' risk averseness. Participants' triage accuracy was higher when they were certain about their appraisal (2114/3381, 62.5%) than when being uncertain (378/714, 52.9%). However, most errors occurred when participants were certain of their decision (1267/1603, 79%). Participants were more commonly certain of their overtriage errors (mean 80.9%, SD 23.8%) than their undertriage errors (mean 72.5%, SD 30.9%; t>89=3.7; P<.001; d=0.39). CONCLUSIONS: Our study suggests that laypersons are overcautious in deciding whether they require medical care at all, but they miss identifying a considerable portion of emergencies. Our results further indicate that women are more risk averse than men in both types of decisions. Layperson participants made most triage errors when they were certain of their own appraisal. Thus, they might not follow or even seek advice (eg, from symptom checkers) in most instances where advice would be useful.

Interactive Versus Static Decision Support Tools for COVID-19: Randomized Controlled Trial.

BACKGROUND: During the COVID-19 pandemic, medical laypersons with symptoms indicative of a COVID-19 infection commonly sought guidance on whether and where to find medical care. Numerous web-based decision support tools (DSTs) have been developed, both by public and commercial stakeholders, to assist their decision making. Though most of the DSTs' underlying algorithms are similar and simple decision trees, their mode of presentation differs: some DSTs present a static flowchart, while others are designed as a conversational agent, guiding the user through the decision tree's nodes step-by-step in an interactive manner. OBJECTIVE: This study aims to investigate whether interactive DSTs provide greater decision support than noninteractive (ie, static) flowcharts. METHODS: We developed mock interfaces for 2 DSTs (1 static, 1 interactive), mimicking patient-facing, freely available DSTs for COVID-19-related self-assessment. Their underlying algorithm was identical and based on the Centers for Disease Control and Prevention's guidelines. We recruited adult US residents online in November 2020. Participants appraised the appropriate social and care-seeking behavior for 7 fictitious descriptions of patients (case vignettes). Participants in the experimental groups received either the static or the interactive mock DST as support, while the control group appraised the case vignettes unsupported. We determined participants' accuracy, decision certainty (after deciding), and mental effort to measure the quality of decision support. Participants' ratings of the DSTs' usefulness, ease of use, trust, and future intention to use the tools served as measures to analyze differences in participants' perception of the tools. We used ANOVAs and t tests to assess statistical significance. RESULTS: Our survey yielded 196 responses. The mean number of correct assessments was higher in the intervention groups (interactive DST group: mean 11.71, SD 2.37; static DST group: mean 11.45, SD 2.48) than in the control group (mean 10.17, SD 2.00). Decisional certainty was significantly higher in the experimental groups (interactive DST group: mean 80.7%, SD 14.1%; static DST group: mean 80.5%, SD 15.8%) compared to the control group (mean 65.8%, SD 20.8%). The differences in these measures proved statistically significant in t tests comparing each intervention group with the control group (P<.001 for all 4 t tests). ANOVA detected no significant differences regarding mental effort between the 3 study groups. Differences between the 2 intervention groups were of small effect sizes and nonsignificant for all 3 measures of the quality of decision support and most measures of participants' perception of the DSTs. CONCLUSIONS: When the decision space is limited, as is the case in common COVID-19 self-assessment DSTs, static flowcharts might prove as beneficial in enhancing decision quality as interactive tools. Given that static flowcharts reveal the underlying decision algorithm more transparently and require less effort to develop, they might prove more efficient in providing guidance to the public. Further research should validate our findings on different use cases, elaborate on the trade-off between transparency and convenience in DSTs, and investigate whether subgroups of users benefit more with 1 type of user interface than the other. TRIAL REGISTRATION: Deutsches Register Klinischer Studien DRKS00028136; https://tinyurl.com/4bcfausx (retrospectively registered).

Determinants of Laypersons' Trust in Medical Decision Aids: Randomized Controlled Trial.

BACKGROUND: Symptom checker apps are patient-facing decision support systems aimed at providing advice to laypersons on whether, where, and how to seek health care (disposition advice). Such advice can improve laypersons' self-assessment and ultimately improve medical outcomes. Past research has mainly focused on the accuracy of symptom checker apps' suggestions. To support decision-making, such apps need to provide not only accurate but also trustworthy advice. To date, only few studies have addressed the question of the extent to which laypersons trust symptom checker app advice or the factors that moderate their trust. Studies on general decision support systems have shown that framing automated systems (anthropomorphic or emphasizing expertise), for example, by using icons symbolizing artificial intelligence (AI), affects users' trust. OBJECTIVE: This study aims to identify the factors influencing laypersons' trust in the advice provided by symptom checker apps. Primarily, we investigated whether designs using anthropomorphic framing or framing the app as an AI increases users' trust compared with no such framing. METHODS: Through a web-based survey, we recruited 494 US residents with no professional medical training. The participants had to first appraise the urgency of a fictitious patient description (case vignette). Subsequently, a decision aid (mock symptom checker app) provided disposition advice contradicting the participants' appraisal, and they had to subsequently reappraise the vignette. Participants were randomized into 3 groups: 2 experimental groups using visual framing (anthropomorphic, 160/494, 32.4%, vs AI, 161/494, 32.6%) and a neutral group without such framing (173/494, 35%). RESULTS: Most participants (384/494, 77.7%) followed the decision aid's advice, regardless of its urgency level. Neither anthropomorphic framing (odds ratio 1.120, 95% CI 0.664-1.897) nor framing as AI (odds ratio 0.942, 95% CI 0.565-1.570) increased behavioral or subjective trust (P=.99) compared with the no-frame condition. Even participants who were extremely certain in their own decisions (ie, 100% certain) commonly changed it in favor of the symptom checker's advice (19/34, 56%). Propensity to trust and eHealth literacy were associated with increased subjective trust in the symptom checker (propensity to trust b=0.25; eHealth literacy b=0.2), whereas sociodemographic variables showed no such link with either subjective or behavioral trust. CONCLUSIONS: Contrary to our expectation, neither the anthropomorphic framing nor the emphasis on AI increased trust in symptom checker advice compared with that of a neutral control condition. However, independent of the interface, most participants trusted the mock app's advice, even when they were very certain of their own assessment. Thus, the question arises as to whether laypersons use such symptom checkers as substitutes rather than as aids in their own decision-making. With trust in symptom checkers already high at baseline, the benefit of symptom checkers depends on interface designs that enable users to adequately calibrate their trust levels during usage. TRIAL REGISTRATION: Deutsches Register Klinischer Studien DRKS00028561; https://tinyurl.com/rv4utcfb (retrospectively registered).

Examining the impact of a symptom assessment application on patient-physician interaction among self-referred walk-in patients in the emergency department (AKUSYM): study protocol for a multi-center, randomized controlled, parallel-group superiority trial.

BACKGROUND: Due to the increasing use of online health information, symptom checkers have been developed to provide an individualized assessment of health complaints and provide potential diagnoses and an urgency estimation. It is assumed that they support patient empowerment and have a positive impact on patient-physician interaction and satisfaction with care. Particularly in the emergency department (ED), symptom checkers could be integrated to bridge waiting times in the ED, and patients as well as physicians could take advantage of potential positive effects. Our study therefore aims to assess the impact of symptom assessment application (SAA) usage compared to no SAA usage on the patient-physician interaction in self-referred walk-in patients in the ED population. METHODS: In this multi-center, 1:1 randomized, controlled, parallel-group superiority trial, 440 self-referred adult walk-in patients with a non-urgent triage category will be recruited in three EDs in Berlin. Eligible participants in the intervention group will use a SAA directly after initial triage. The control group receives standard care without using a SAA. The primary endpoint is patients' satisfaction with the patient-physician interaction assessed by the Patient Satisfaction Questionnaire. DISCUSSION: The results of this trial could influence the implementation of SAA into acute care to improve the satisfaction with the patient-physician interaction. TRIAL REGISTRATION: German Clinical Trials Registry DRKS00028598 . Registered on 25.03.2022.

Predicting lethal courses in critically ill COVID-19 patients using a machine learning model trained on patients with non-COVID-19 viral pneumonia.

In a pandemic with a novel disease, disease-specific prognosis models are available only with a delay. To bridge the critical early phase, models built for similar diseases might be applied. To test the accuracy of such a knowledge transfer, we investigated how precise lethal courses in critically ill COVID-19 patients can be predicted by a model trained on critically ill non-COVID-19 viral pneumonia patients. We trained gradient boosted decision tree models on 718 (245 deceased) non-COVID-19 viral pneumonia patients to predict individual ICU mortality and applied it to 1054 (369 deceased) COVID-19 patients. Our model showed a significantly better predictive performance (AUROC 0.86 [95% CI 0.86-0.87]) than the clinical scores APACHE2 (0.63 [95% CI 0.61-0.65]), SAPS2 (0.72 [95% CI 0.71-0.74]) and SOFA (0.76 [95% CI 0.75-0.77]), the COVID-19-specific mortality prediction models of Zhou (0.76 [95% CI 0.73-0.78]) and Wang (laboratory: 0.62 [95% CI 0.59-0.65]; clinical: 0.56 [95% CI 0.55-0.58]) and the 4C COVID-19 Mortality score (0.71 [95% CI 0.70-0.72]). We conclude that lethal courses in critically ill COVID-19 patients can be predicted by a machine learning model trained on non-COVID-19 patients. Our results suggest that in a pandemic with a novel disease, prognosis models built for similar diseases can be applied, even when the diseases differ in time courses and in rates of critical and lethal courses.