Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation.

BACKGROUND: Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis. METHODS: To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm. RESULTS: A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ2 = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals' recommendations (χ2 = 2.49, P = 0.11). LIMITATIONS: Cases were from a single institution. CI was solicited using a single algorithm/platform. CONCLUSIONS: When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.

Impact of digitally acquired peer diagnostic input on diagnostic confidence in outpatient cases: A pragmatic randomized trial.

OBJECTIVE: The study sought to evaluate if peer input on outpatient cases impacted diagnostic confidence. MATERIALS AND METHODS: This randomized trial of a peer input intervention occurred among 28 clinicians with case-level randomization. Encounters with diagnostic uncertainty were entered onto a digital platform to collect input from ≥5 clinicians. The primary outcome was diagnostic confidence. We used mixed-effects logistic regression analyses to assess for intervention impact on diagnostic confidence. RESULTS: Among the 509 cases (255 control; 254 intervention), the intervention did not impact confidence (odds ratio [OR], 1.46; 95% confidence interval [CI], 0.999-2.12), but after adjusting for clinician and case traits, the intervention was associated with higher confidence (OR, 1.53; 95% CI, 1.01-2.32). The intervention impact was greater in cases with high uncertainty (OR, 3.23; 95% CI, 1.09- 9.52). CONCLUSIONS: Peer input increased diagnostic confidence primarily in high-uncertainty cases, consistent with findings that clinicians desire input primarily in cases with continued uncertainty.

Preparing for and responding to Covid-19's 'second hit'.

While already sobering, Covid-19 mortality projections only account for a portion of morbidity and mortality we should expect from the current outbreak - patients directly affected by Covid-19. Largely missing from current discussions is the indirect impact on a much broader set of patients affected the epidemic - patients who will experience greater morbidity and mortality from a wide range of clinical conditions due to disruptions in the provision of health care and other essential services - what we are describing here as the 'second hit' of Covid-19.

Comparative Accuracy of Diagnosis by Collective Intelligence of Multiple Physicians vs Individual Physicians.

Importance: The traditional approach of diagnosis by individual physicians has a high rate of misdiagnosis. Pooling multiple physicians' diagnoses (collective intelligence) is a promising approach to reducing misdiagnoses, but its accuracy in clinical cases is unknown to date. Objective: To assess how the diagnostic accuracy of groups of physicians and trainees compares with the diagnostic accuracy of individual physicians. Design, Setting, and Participants: Cross-sectional study using data from the Human Diagnosis Project (Human Dx), a multicountry data set of ranked differential diagnoses by individual physicians, graduate trainees, and medical students (users) solving user-submitted, structured clinical cases. From May 7, 2014, to October 5, 2016, groups of 2 to 9 randomly selected physicians solved individual cases. Data analysis was performed from March 16, 2017, to July 30, 2018. Main Outcomes and Measures: The primary outcome was diagnostic accuracy, assessed as a correct diagnosis in the top 3 ranked diagnoses for an individual; for groups, the top 3 diagnoses were a collective differential generated using a weighted combination of user diagnoses with a variety of approaches. A version of the McNemar test was used to account for clustering across repeated solvers to compare diagnostic accuracy. Results: Of the 2069 users solving 1572 cases from the Human Dx data set, 1228 (59.4%) were residents or fellows, 431 (20.8%) were attending physicians, and 410 (19.8%) were medical students. Collective intelligence was associated with increasing diagnostic accuracy, from 62.5% (95% CI, 60.1%-64.9%) for individual physicians up to 85.6% (95% CI, 83.9%-87.4%) for groups of 9 (23.0% difference; 95% CI, 14.9%-31.2%; P < .001). The range of improvement varied by the specifications used for combining groups' diagnoses, but groups consistently outperformed individuals regardless of approach. Absolute improvement in accuracy from individuals to groups of 9 varied by presenting symptom from an increase of 17.3% (95% CI, 6.4%-28.2%; P = .002) for abdominal pain to 29.8% (95% CI, 3.7%-55.8%; P = .02) for fever. Groups from 2 users (77.7% accuracy; 95% CI, 70.1%-84.6%) to 9 users (85.5% accuracy; 95% CI, 75.1%-95.9%) outperformed individual specialists in their subspecialty (66.3% accuracy; 95% CI, 59.1%-73.5%; P < .001 vs groups of 2 and 9). Conclusions and Relevance: A collective intelligence approach was associated with higher diagnostic accuracy compared with individuals, including individual specialists whose expertise matched the case diagnosis, across a range of medical cases. Given the few proven strategies to address misdiagnosis, this technique merits further study in clinical settings.

Assessment of a Simulated Case-Based Measurement of Physician Diagnostic Performance.

Importance: Diagnostic acumen is a fundamental skill in the practice of medicine. Scalable, practical, and objective tools to assess diagnostic performance are lacking. Objective: To validate a new method of assessing diagnostic performance that uses automated techniques to assess physicians' diagnostic performance on brief, open-ended case simulations. Design, Setting, and Participants: Retrospective cohort study of 11 023 unique attempts to solve case simulations on an online software platform, The Human Diagnosis Project (Human Dx). A total of 1738 practicing physicians, residents (internal medicine, family medicine, and emergency medicine), and medical students throughout the United States voluntarily used Human Dx software between January 21, 2016, and January 15, 2017. Main Outcomes and Measures: Internal structure validity was assessed by 3 measures of diagnostic performance: accuracy, efficiency, and a combined score (Diagnostic Acumen Precision Performance [DAPP]). These were each analyzed by level of training. Association with other variables' validity evidence was evaluated by correlating diagnostic performance and affiliation with an institution ranked in the top 25 medical schools by US News and World Report. Results: Data were analyzed for 239 attending physicians, 926 resident physicians, 347 intern physicians, and 226 medical students. Attending physicians had higher mean accuracy scores than medical students (difference, 8.1; 95% CI, 4.2-12.0; P < .001), as did residents (difference, 8.0; 95% CI, 4.8-11.2; P < .001) and interns (difference, 5.9; 95% CI, 2.3-9.6; P < .001). Attending physicians had higher mean efficiency compared with residents (difference, 4.8; 95% CI, 1.8-7.8; P < .001), interns (difference, 5.0; 95% CI, 1.5-8.4; P = .001), and medical students (difference, 5.4; 95% CI, 1.4-9.3; P = .003). Attending physicians also had significantly higher mean DAPP scores than residents (difference, 2.6; 95% CI, 0.0-5.2; P = .05), interns (difference, 3.6; 95% CI, 0.6-6.6; P = .01), and medical students (difference, 6.7; 95% CI, 3.3-10.2; P < .001). Attending physicians affiliated with a US News and World Report-ranked institution had higher mean DAPP scores compared with nonaffiliated attending physicians (80 [95% CI, 77-83] vs 72 [95% CI, 70-74], respectively; P < .001). Resident physicians affiliated with an institution ranked in the top 25 medical schools by US News and World Report also had higher mean DAPP scores compared with nonaffiliated peers (75 [95% CI, 73-77] vs 71 [95% CI, 69-72], respectively; P < .001). Conclusions and Relevance: The data suggest that diagnostic performance is higher in those with more training and that DAPP scores may be a valid measure to appraise diagnostic performance. This diagnostic assessment tool allows individuals to receive immediate feedback on performance through an openly accessible online platform.

Bolstering diagnostic reasoning skills with adaptive learning.

PURPOSE: Adaptive learning emerges when precise assessment informs delivery of educational materials. This study will demonstrate how data from Human Dx, a case-based e-learning platform, can characterize an individual's diagnostic reasoning skills, and deliver tailored content to improve accuracy. METHODS: Pearson Chi-square analysis was used to assess variability in accuracy across three groups of participants (attendings, residents, and medical students) and three categories of cases (core medical, surgical, and other). Logistic regression analyses were conducted to explore the relationship between solve duration and accuracy. Mean accuracy and duration were calculated for 370 individuals. Repeated measures analysis of variance (ANOVA) were used to assess variability for an individual solver across the three categories. RESULTS: There were significant differences in accuracy across the three groups and the three categories (p < 0.001). Individual solvers have significant variance in accuracy across the three categories. Shorter solve duration predicted higher accuracy. Patterns of performance were identified; four profiles are highlighted to demonstrate potential adaptive learning interventions. CONCLUSIONS: Human Dx can assess diagnostic reasoning skills. When weaknesses are identified, adaptive learning strategies can push content to promote skill development. This has implications for customizing curricular elements to improve the diagnostic skills of healthcare professionals.

How do mobile phone diabetes programs drive behavior change? Evidence from a mixed methods observational cohort study.

PURPOSE: The purpose of this study was to investigate the behavioral effects of a theory-driven, mobile phone-based intervention that combines automated text messaging and remote nursing, using an automated, interactive text messaging system. METHODS: This was a mixed methods observational cohort study. Study participants were members of the University of Chicago Health Plan (UCHP) who largely reside in a working-class, urban African American community. Surveys were conducted at baseline, 3 months (mid-intervention), and 6 months (postintervention) to test the hypothesis that the intervention would be associated with improvements in self-efficacy, social support, health beliefs, and self-care. In addition, in-depth individual interviews were conducted with 14 participants and then analyzed using the constant comparative method to identify new behavioral constructs affected by the intervention. RESULTS: The intervention was associated with improvements in 5 of 6 domains of self-care (medication taking, glucose monitoring, foot care, exercise, and healthy eating) and improvements in 1 or more measures of self-efficacy, social support, and health beliefs (perceived control). Qualitatively, participants reported that knowledge, attitudes, and ownership were also affected by the program. Together these findings were used to construct a new behavioral model. CONCLUSIONS: This study's findings challenge the prevailing assumption that mobile phones largely affect behavior change through reminders and support the idea that behaviorally driven mobile health interventions can address multiple behavioral pathways associated with sustained behavior change.

Incidence of remission in adults with type 2 diabetes: the diabetes & aging study.

OBJECTIVE: To estimate the incidence of remission in adults with type 2 diabetes not treated with bariatric surgery and to identify variables associated with remission. RESEARCH DESIGN AND METHODS: We quantified the incidence of diabetes remission and examined its correlates among 122,781 adults with type 2 diabetes in an integrated healthcare delivery system. Remission required the absence of ongoing drug therapy and was defined as follows: 1) partial: at least 1 year of subdiabetic hyperglycemia (hemoglobin A1c [HbA1c] level 5.7-6.4% [39-46 mmol/mol]); 2) complete: at least 1 year of normoglycemia (HbA1c level <5.7% [<39 mmol/mol]); and 3) prolonged: complete remission for at least 5 years. RESULTS: The incidence density (remissions per 1,000 person-years; 95% CI) of partial, complete, or prolonged remission was 2.8 (2.6-2.9), 0.24 (0.20-0.28), and 0.04 (0.01-0.06), respectively. The 7-year cumulative incidence of partial, complete, or prolonged remission was 1.47% (1.40-1.54%), 0.14% (0.12-0.16%), and 0.007% (0.003-0.020%), respectively. The 7-year cumulative incidence of achieving any remission was 1.60% in the whole cohort (1.53-1.68%) and 4.6% in the subgroup with new-onset diabetes (<2 years since diagnosis) (4.3-4.9%). After adjusting for demographic and clinical characteristics, correlates of remission included age >65 years, African American race, <2 years since diagnosis, baseline HbA1c level <5.7% (<39 mmol/mol), and no diabetes medication at baseline. CONCLUSIONS: In community settings, remission of type 2 diabetes does occur without bariatric surgery, but it is very rare.

Using Patient-Generated Health Data From Mobile Technologies for Diabetes Self-Management Support: Provider Perspectives From an Academic Medical Center.

BACKGROUND: Mobile health and patient-generated health data are promising health IT tools for delivering self-management support in diabetes, but little is known about provider perspectives on how best to integrate these programs into routine care. We explored provider perceptions of a patient-generated health data report from a text-message-based diabetes self-management program. The report was designed to relay clinically relevant data obtained from participants' responses to self-assessment questions delivered over text message. METHODS: Likert-type scale response surveys and in-depth interviews were conducted with primary care physicians and endocrinologists who pilot tested the patient-generated health data report in an actual clinical encounter. Interview guides were designed to assess providers' perceptions of the feasibility and utility of patient-generated health data in routine clinical practice. Interviews were audiotaped, transcribed, and analyzed using the constant comparative method. RESULTS: Twelve providers successfully piloted the summary report in clinic. Although only a minority of providers felt the report changed the care they provided (3 of 12 or 25%), most were willing to use the summary report in a future clinical encounter (9 of 12 or 75%). Perceived benefits of patient-generated health data included agenda setting, assessment of self-care, and identification of patient barriers. Major themes discussed included patient selection, reliability of patient-generated health information, and integration into clinical workflow. CONCLUSION: Providers perceived multiple benefits of patient-generated health data in overcoming common barriers to self-management support in clinical practice and found the summary report feasible and usable in a clinical context.

Mobile phone diabetes project led to improved glycemic control and net savings for Chicago plan participants.

Even with the best health care available, patients with chronic illnesses typically spend no more than a few hours a year in a health care setting, while their outcomes are largely determined by their activities during the remaining 5,000 waking hours of the year. As a widely available, low-cost technology, mobile phones are a promising tool to use in engaging patients in behavior change and facilitating self-care between visits. We examined the impact of a six-month mobile health (mHealth) demonstration project among adults with diabetes who belonged to an academic medical center's employee health plan. In addition to pre-post improvements in glycemic control (p=0.01) and patients' satisfaction with overall care (p=0.04), we observed a net cost savings of 8.8 percent. Those early results suggest that mHealth programs can support health care organizations' pursuit of the triple aim of improving patients' experiences with care, improving population health, and reducing the per capita cost of health care

Relationship-centered care: A new paradigm for population health management.

At the center of population health management services are the relationships between the patient and their care providers. The spread of relationship-centered care has resulted from the need to develop and nurture these relationships. As a model of coordinated and team-based care, relationship-based care is able to strengthen population health management. This paper explores why relationship-centered care is fundamental to population health management, describes compatibilities with patient-centered care and discusses examples of early applications of this paradigm. Using our experience in a population health management organization, we describe applications and lessons learned using this paradigm.

A text messaging intervention to improve heart failure self-management after hospital discharge in a largely African-American population: before-after study.

BACKGROUND: There is increasing interest in finding novel approaches to reduce health disparities in readmissions for acute decompensated heart failure (ADHF). Text messaging is a promising platform for improving chronic disease self-management in low-income populations, yet is largely unexplored in ADHF. OBJECTIVE: The purpose of this pre-post study was to assess the feasibility and acceptability of a text message-based (SMS: short message service) intervention in a largely African American population with ADHF and explore its effects on self-management. METHODS: Hospitalized patients with ADHF were enrolled in an automated text message-based heart failure program for 30 days following discharge. Messages provided self-care reminders and patient education on diet, symptom recognition, and health care navigation. Demographic and cell phone usage data were collected on enrollment, and an exit survey was administered on completion. The Self-Care of Heart Failure Index (SCHFI) was administered preintervention and postintervention and compared using sample t tests (composite) and Wilcoxon rank sum tests (individual). Clinical data were collected through chart abstraction. RESULTS: Of 51 patients approached for recruitment, 27 agreed to participate and 15 were enrolled (14 African-American, 1 White). Barriers to enrollment included not owning a personal cell phone (n=12), failing the Mini-Mental exam (n=3), needing a proxy (n=2), hard of hearing (n=1), and refusal (n=3). Another 3 participants left the study for health reasons and 3 others had technology issues. A total of 6 patients (5 African-American, 1 White) completed the postintervention surveys. The mean age was 50 years (range 23-69) and over half had Medicaid or were uninsured (60%, 9/15). The mean ejection fraction for those with systolic dysfunction was 22%, and at least two-thirds had a prior hospitalization in the past year. Participants strongly agreed that the program was easy to use (83%), reduced pills missed (66%), and decreased salt intake (66%). Maintenance (mean composite score 49 to 78, P=.003) and management (57 to 86, P=.002) improved at 4 weeks, whereas confidence did not change (57 to 75, P=.11). Of the 6 SCHFI items that showed a statistically significant improvement, 5 were specifically targeted by the texting intervention. CONCLUSIONS: Over half of ADHF patients in an urban, largely African American community were eligible and interested in participating in a text messaging program following discharge. Access to mobile phones was a significant barrier that should be addressed in future interventions. Among the participants who completed the study, we observed a high rate of satisfaction and preliminary evidence of improvements in heart failure self-management.

Developing a behavioral model for mobile phone-based diabetes interventions.

OBJECTIVES: Behavioral models for mobile phone-based diabetes interventions are lacking. This study explores the potential mechanisms by which a text message-based diabetes program affected self-management among African-Americans. METHODS: We conducted in-depth, individual interviews among 18 African-American patients with type 2 diabetes who completed a 4-week text message-based diabetes program. Each interview was audio-taped, transcribed verbatim, and imported into Atlas.ti software. Coding was done iteratively. Emergent themes were mapped onto existing behavioral constructs and then used to develop a novel behavioral model for mobile phone-based diabetes self-management programs. RESULTS: The effects of the text message-based program went beyond automated reminders. The constant, daily communications reduced denial of diabetes and reinforced the importance of self-management (Rosenstock Health Belief Model). Responding positively to questions about self-management increased mastery experience (Bandura Self-Efficacy). Most surprisingly, participants perceived the automated program as a "friend" and "support group" that monitored and supported their self-management behaviors (Barrera Social Support). CONCLUSIONS: A mobile phone-based diabetes program affected self-management through multiple behavioral constructs including health beliefs, self-efficacy, and social support. PRACTICE IMPLICATIONS: Disease management programs that utilize mobile technologies should be designed to leverage existing models of behavior change and can address barriers to self-management associated with health disparities.