Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

OBJECTIVES: Usable tools to support individual primary care clinicians in their diagnostic processes could help to reduce preventable harm from diagnostic errors. We conducted a formative study with primary care providers to identify key requisites to optimize the acceptability of 1 online collective intelligence platform (Human Diagnosis Project; Human Dx). MATERIALS AND METHODS: We conducted semistructured interviews with practicing primary care clinicians in a sample of the US community-based clinics to examine the acceptability and early usability of the collective intelligence online platform using standardized clinical cases and real-world clinical cases from the participants' own practice. We used an integrated inductive-deductive qualitative analysis approach to analyze the interview transcripts. RESULTS AND DISCUSSION: Perceived usefulness, perceived accuracy, quality assurance, trust, and ease of use emerged as essential domains of acceptability required for providers to use a collective intelligence tool in clinical practice. Participants conveyed that the collective opinion should: (1) contribute to their clinical reasoning, (2) boost their confidence, (3) be generated in a timely manner, and (4) be relevant to their clinical settings and use cases. Trust in the technology platform and the clinical accuracy of its collective intelligence output emerged as an incontrovertible requirement for user acceptance and engagement. CONCLUSION: We documented key requisites to building a collective intelligence technology platform that is trustworthy, useful, and acceptable to target end users for assistance in the diagnostic process. These key lessons may be applicable to other provider-facing decision support platforms.

Automating Guidelines for Clinical Decision Support: Knowledge Engineering and Implementation.

As utilization of clinical decision support (CDS) increases, it is important to continue the development and refinement of methods to accurately translate the intention of clinical practice guidelines (CPG) into a computable form. In this study, we validate and extend the 13 steps that Shiffman et al.5 identified for translating CPG knowledge for use in CDS. During an implementation project of ATHENA-CDS, we encoded complex CPG recommendations for five common chronic conditions for integration into an existing clinical dashboard. Major decisions made during the implementation process were recorded and categorized according to the 13 steps. During the implementation period, we categorized 119 decisions and identified 8 new categories required to complete the project. We provide details on an updated model that outlines all of the steps used to translate CPG knowledge into a CDS integrated with existing health information technology.

Automating Performance Measures and Clinical Practice Guidelines: Differences and Complementarities.

Through close analysis of two pairs of systems that implement the automated evaluation of performance measures (PMs) and guideline-based clinical decision support (CDS), we contrast differences in their knowledge encoding and necessary changes to a CDS system that provides management recommendations for patients failing performance measures. We trace the sources of differences to the implementation environments and goals of PMs and CDS.

Test Case Selection in Pre-Deployment Testing of Complex Clinical Decision Support Systems.

Clinical decision support (CDS) systems with complex logic are being developed. Ensuring the quality of CDS is imperative, but there is no consensus on testing standards. We tested ATHENA-HTN CDS after encoding updated hypertension guidelines into the system. A logic flow and a complexity analysis of the encoding were performed to guide testing. 100 test cases were selected to test the major pathways in the CDS logic flow, and the effectiveness of the testing was analyzed. The encoding contained 26 decision points and 3120 possible output combinations. The 100 cases selected tested all of the major pathways in the logic, but only 1% of the possible output combinations. Test case selection is one of the most challenging aspects in CDS testing and has a major impact on testing coverage. A test selection strategy should take into account the complexity of the system, identification of major logic pathways, and available resources.

Plasma oxytocin concentrations are lower in depressed vs. healthy control women and are independent of cortisol.

The neuropeptide oxytocin (OT) promotes social behavior and attenuates stress responsivity in mammals. Recent clinical evidence suggests OT concentrations may be dysregulated in major depression. This study extends previous research by testing whether: 1) OT concentrations vary systematically in depressive disorders with and without hypercortisolemia, 2) gender differences in OT concentrations are observed in depressed vs. healthy control participants, and 3) OT concentrations are predictive of clinical phenotypes. Plasma OT concentrations of psychotic major depressive (PMD; n = 14: 10 female, 4 male), non-psychotic major depressive (NPMD; n = 17: 12 female, 5 male), and non-depressed, healthy control (n = 19: 11 female, 8 male) participants were assayed at 2000, 2400, 0400, and 0800 h. Plasma cortisol concentrations were quantified at 2300 h, and clinical phenotypes were determined. As expected, PMD participants, compared to NPMD and healthy control participants, showed higher plasma cortisol concentrations. Although both depressed groups showed similar OT concentrations, a significant interaction effect between group and gender was observed. Specifically, depressed females exhibited lower mean OT concentrations than depressed males. Further, depressed vs. healthy control female participants exhibited lower mean OT concentrations, whereas depressed vs. healthy control male participants showed a trend in the opposite direction. OT concentrations were also predictive of desirability, drug dependence, and compulsivity scores as measured by the Million Clinical Multiaxial Inventory-III. All findings were independent of cortisol. These data suggest that OT signaling may provide a mechanism by which to better understand female-biased risk to develop depressive disorders and that plasma OT concentrations may be a useful biomarker of certain clinical phenotypes.