An Untapped Potential in Primary Care: Semi-Structured Interviews with Clinicians on How Patient Portals Will Work for Caregivers in the Safety Net.

BACKGROUND: Patients within safety-net settings are less likely to access health information on patient portals, despite expressed interest. Family and friends are important resources to assist these patients (ie, Medicaid recipients, older patients, patients with limited English proficiency) in navigating health systems, and provider support of the use of patient portals among these groups may also facilitate caregivers' use of their patients' portal. OBJECTIVE: Because safety net providers work closely with caregivers to care for their patients, we used qualitative methods to explore safety net providers' perspectives on portal use among caregivers for their patients, especially as there is limited literature about caregivers' use of portals in the safety net. METHODS: We conducted 45- to 60-min semistructured telephone interviews with providers from three large California safety-net health systems. The interviews focused on providers' experiences with caregivers, caregiver roles, and how the portal could be leveraged as a tool to support caregivers in their responsibilities. A total of three coders analyzed the interview transcripts using both deductive and inductive approaches and established a consensus regarding major themes. RESULTS: Of the 16 participants interviewed, 4 specialized in geriatrics, and all held a leadership or administrative role. We described themes highlighting providers' recognition of potential benefits associated with caregiver portal use and specific challenges to caregiver engagement. CONCLUSIONS: Providers recognized the potential for portals to improve information delivery and communication by helping caregivers assist socially and medically complex patients in the safety net. Providers in safety net sites also discussed a clear need for better ways to keep in touch with patients and connect with caregivers, yet security and privacy are perhaps of higher importance in these settings and may pose challenges to portal adoption. They noted that caregivers of patients in the safety net likely face similar communication barriers as patients, especially with respect to digital literacy, health literacy, and English proficiency. Further research is needed to assess and support caregivers' interest and ability to access portals across barriers in health and digital literacy, and English proficiency. Portal platforms and health systems must also address specific strategies to uphold patient preferences while maintaining privacy and security.

Disparities in Patient-Reported Interest in Web-Based Patient Portals: Survey at an Urban Academic Safety-Net Hospital.

BACKGROUND: Offering hospitalized patients' enrollment into a health system's patient portal may improve patient experience and engagement throughout the care continuum, especially across care transitions, but this process is less studied than portal engagement in the ambulatory setting. Patient portal disparities exist and may lead to differences in access or outcomes. As such, it is important to study upstream factors in a typical hospital workflow that could lead to those disparities in safety-net settings. OBJECTIVE: The objective of this study was to evaluate sociodemographic characteristics associated with interest in a health care system's portal among hospitalized patients and reasons for no interest. METHODS: Nurses assessed interest in a Web-based patient portal, expressed by the patient as "yes" or "no," as part of the admission nursing assessment among patients at an academic urban safety-net hospital and recorded responses in the electronic health record (EHR), including reasons for no interest. We extracted patient responses from the EHR. RESULTS: Among 23,994 hospitalizations over a 2-year period, 35.90% (8614/ 23,994) reported an interest in a Web-based portal. Reasons for no interest included the following: not interested/other reason 41.68% (6410/15,380), no ability to use/access computers/internet 29.59% (4551/15,380), doesn't speak English 11.15% (1715/15,380), physically or mentally unable 8.70% (1338/15,380), does not want to say 8.70% (1338/15,380), security concerns 0.03% (4/15,380), and not useful 0.16% (24/15,380). Among the 16,507 unique patients included in this sample, portal interest was lower in older, African American, non-English speaking, and homeless patient populations. CONCLUSIONS: In a safety-net system, patient interest at the time of hospitalization in a Web-based enterprise portal-a required step before enrollment-is low with significant disparities by sociodemographic characteristics. To avoid worsening the digital divide, new strategies are needed and should be embedded within routine workflows to engage vulnerable safety-net patients in the use of Web-based health technologies.

Perspectives of English, Chinese, and Spanish-Speaking Safety-Net Patients on Clinician Computer Use: Qualitative Analysis.

BACKGROUND: Safety-net systems serve patients with limited health literacy and limited English proficiency (LEP) who face communication barriers. However, little is known about how diverse safety-net patients feel about increasing clinician electronic health record (EHR) use. OBJECTIVE: The aim of this study was to better understand how safety-net patients, including those with LEP, view clinician EHR use. METHODS: We conducted focus groups in English, Spanish, and Cantonese (N=37) to elicit patient perspectives on how clinicians use EHRs during clinic visits. Using a grounded theory approach, we coded transcripts to identify key themes. RESULTS: Across multiple language groups, participants accepted multitasking and silent clinician EHR use if focused on their care. However, participants desired more screen share and eye contact, especially when demonstrating physical concerns. All participants, including LEP participants, wanted clinicians to include them in EHR use. CONCLUSIONS: Linguistically diverse patients accept the value of EHR use during outpatient visits but desire more eye contact, verbal warnings before EHR use, and screen-sharing. Safety-net health systems should support clinicians in completing EHR-related tasks during the visit using patient-centered strategies for all patients.

Language-concordant automated telephone queries to assess medication adherence in a diverse population: a cross-sectional analysis of convergent validity with pharmacy claims.

BACKGROUND: Clinicians have difficulty accurately assessing medication non-adherence within chronic disease care settings. Health information technology (HIT) could offer novel tools to assess medication adherence in diverse populations outside of usual health care settings. In a multilingual urban safety net population, we examined the validity of assessing adherence using automated telephone self-management (ATSM) queries, when compared with non-adherence using continuous medication gap (CMG) on pharmacy claims. We hypothesized that patients reporting greater days of missed pills to ATSM queries would have higher rates of non-adherence as measured by CMG, and that ATSM adherence assessments would perform as well as structured interview assessments. METHODS: As part of an ATSM-facilitated diabetes self-management program, low-income health plan members typed numeric responses to rotating weekly ATSM queries: "In the last 7 days, how many days did you MISS taking your …" diabetes, blood pressure, or cholesterol pill. Research assistants asked similar questions in computer-assisted structured telephone interviews. We measured continuous medication gap (CMG) by claims over 12 preceding months. To evaluate convergent validity, we compared rates of optimal adherence (CMG ≤ 20%) across respondents reporting 0, 1, and ≥ 2 missed pill days on ATSM and on structured interview. RESULTS: Among 210 participants, 46% had limited health literacy, 57% spoke Cantonese, and 19% Spanish. ATSM respondents reported ≥1 missed day for diabetes (33%), blood pressure (19%), and cholesterol (36%) pills. Interview respondents reported ≥1 missed day for diabetes (28%), blood pressure (21%), and cholesterol (26%) pills. Optimal adherence rates by CMG were lower among ATSM respondents reporting more missed days for blood pressure (p = 0.02) and cholesterol (p < 0.01); by interview, differences were significant for cholesterol (p = 0.01). CONCLUSIONS: Language-concordant ATSM demonstrated modest potential for assessing adherence. Studies should evaluate HIT assessments of medication beliefs and concerns in diverse populations. TRIAL REGISTRATION: NCT00683020 , registered May 21, 2008.

Barriers and Facilitators to Online Portal Use Among Patients and Caregivers in a Safety Net Health Care System: A Qualitative Study.

BACKGROUND: Patient portals have the potential to support self-management for chronic diseases and improve health outcomes. With the rapid rise in adoption of patient portals spurred by meaningful use incentives among safety net health systems (a health system or hospital providing a significant level of care to low-income, uninsured, and vulnerable populations), it is important to understand the readiness and willingness of patients and caregivers in safety net settings to access their personal health records online. OBJECTIVE: To explore patient and caregiver perspectives on online patient portal use before its implementation at San Francisco General Hospital, a safety net hospital. METHODS: We conducted 16 in-depth interviews with chronic disease patients and caregivers who expressed interest in using the Internet to manage their health. Discussions focused on health care experiences, technology use, and interest in using an online portal to manage health tasks. We used open coding to categorize all the barriers and facilitators to portal use, followed by a second round of coding that compared the categories to previously published findings. In secondary analyses, we also examined specific barriers among 2 subgroups: those with limited health literacy and caregivers. RESULTS: We interviewed 11 patients and 5 caregivers. Patients were predominantly male (82%, 9/11) and African American (45%, 5/11). All patients had been diagnosed with diabetes and the majority had limited health literacy (73%, 8/11). The majority of caregivers were female (80%, 4/5), African American (60%, 3/5), caregivers of individuals with diabetes (60%, 3/5), and had adequate health literacy (60%, 3/5). A total of 88% (14/16) of participants reported interest in using the portal after viewing a prototype. Major perceived barriers included security concerns, lack of technical skills/interest, and preference for in-person communication. Facilitators to portal use included convenience, health monitoring, and improvements in patient-provider communication. Participants with limited health literacy discussed more fundamental barriers to portal use, including challenges with reading and typing, personal experience with online security breaches/viruses, and distrust of potential security measures. Caregivers expressed high interest in portal use to support their roles in interpreting health information, advocating for quality care, and managing health behaviors and medical care. CONCLUSIONS: Despite concerns about security, difficulty understanding medical information, and satisfaction with current communication processes, respondents generally expressed enthusiasm about portal use. Our findings suggest a strong need for training and support to assist vulnerable patients with portal registration and use, particularly those with limited health literacy. Efforts to encourage portal use among vulnerable patients should directly address health literacy and security/privacy issues and support access for caregivers.

A multicenter study of physician mindfulness and health care quality.

PURPOSE: Mindfulness (ie, purposeful and nonjudgmental attentiveness to one's own experience, thoughts, and feelings) is associated with physician well-being. We sought to assess whether clinician self-rated mindfulness is associated with the quality of patient care. METHODS: We conducted an observational study of 45 clinicians (34 physicians, 8 nurse practitioners, and 3 physician assistants) caring for patients infected with the human immunodeficiency virus (HIV) who completed the Mindful Attention Awareness Scale and 437 HIV-infected patients at 4 HIV specialty clinic sites across the United States. We measured patient-clinician communication quality with audio-recorded encounters coded using the Roter Interaction Analysis System (RIAS) and patient ratings of care. RESULTS: In adjusted analyses comparing clinicians with highest and lowest tertile mindfulness scores, patient visits with high-mindfulness clinicians were more likely to be characterized by a patient-centered pattern of communication (adjusted odds ratio of a patient-centered visit was 4.14; 95% CI, 1.58-10.86), in which both patients and clinicians engaged in more rapport building and discussion of psychosocial issues. Clinicians with high-mindfulness scores also displayed more positive emotional tone with patients (adjusted ß = 1.17; 95% CI, 0.46-1.9). Patients were more likely to give high ratings on clinician communication (adjusted prevalence ratio [APR] = 1.48; 95% CI, 1.17-1.86) and to report high overall satisfaction (APR = 1.45; 95 CI, 1.15-1.84) with high-mindfulness clinicians. There was no association between clinician mindfulness and the amount of conversation about biomedical issues. CONCLUSIONS: Clinicians rating themselves as more mindful engage in more patient-centered communication and have more satisfied patients. Interventions should determine whether improving clinician mindfulness can also improve patient health outcomes.

Laboratory Results Release to Patients under the 21st Century Cures Act: The Eight Stakeholders Who Should Care.

A major aim of the 21st Century Cures Act is to support patients' access to their electronic health data and to prevent information blocking practices by health care organizations and health information technology developers. Prior to the Cures Act, significant variation existed in patient access to laboratory test results, key pieces of health data which enable timely self-management and engagement in care. Although many health care systems began releasing test results immediately through patient portals because of the Cures Act, implementation remains challenging due to variations in state regulations around electronic results release, local interpretations of allowable exceptions to Cures information blocking, concerns about privacy of sensitive laboratory results, and technological limitations. This paper outlines the eight stakeholder groups involved in implementation of electronic laboratory result release to patients and describes recommendations for these groups to consider in achieving the Cures Act goals to support a patient's access to their health information and control of their health care.

What patients say about their doctors online: a qualitative content analysis.

BACKGROUND: Doctor rating websites are a burgeoning trend, yet little is known about their content. OBJECTIVE: To explore the content of Internet reviews about primary care physicians. DESIGN: Qualitative content analysis of 712 online reviews from two rating websites. We purposively sampled reviews of 445 primary care doctors (internists and family practitioners) from four geographically dispersed U.S. urban locations. We report the major themes, and because this is a large sample, the frequencies of domains within our coding scheme. RESULTS: Most reviews (63%) were positive, recommending the physician. We found a major distinction between global reviews, "Dr. B is a great doctor." vs. specific descriptions which included interpersonal manner, "She always listens to what I have to say and answers all my questions."; technical competence "No matter who she has recommended re: MD specialists, this MD has done everything right."; and/or systems issues such as appointment and telephone access. Among specific reviews, interpersonal manner "Dr. A is so compassionate." and technical competence "He is knowledgeable, will research your case before giving you advice." comments tended to be more positive (69% and 80%, respectively), whereas systems-issues comments "Staff is so-so, less professional than should be…" were more mixed (60% positive, 40% negative). CONCLUSIONS: The majority of Internet reviews of primary care physicians are positive in nature. Our findings reaffirm that the care encounter extends beyond the patient-physician dyad; staff, access, and convenience all affect patient's reviews of physicians. In addition, negative interpersonal reviews underscore the importance of well-perceived bedside manner for a successful patient-physician interaction.

Clinician stress and patient-clinician communication in HIV care.

BACKGROUND: Clinician stress is common, but few studies have examined its relationship with communication behaviors. OBJECTIVE: To investigate associations between clinician stress and patient-clinician communication in primary HIV care. DESIGN: Observational study. PARTICIPANTS: Thirty-three primary HIV clinicians and 350 HIV-infected adult, English-speaking patients at three U.S. HIV specialty clinic sites. MAIN MEASURES: Clinicians completed the Perceived Stress Scale, and we categorized scores in tertiles. Audio-recordings of patient-clinician encounters were coded using the Roter Interaction Analysis System. Patients rated the quality of their clinician's communication and overall quality of medical care. We used regression with generalized estimating equations to examine associations between clinician stress and communication outcomes, controlling for clinician gender, clinic site, and visit length. KEY RESULTS: Among the 33 clinicians, 70 % were physicians, 64 % were women, 67 % were non-Hispanic white, and the mean stress score was 3.9 (SD 2.4, range 0-8). Among the 350 patients, 34 % were women, 55 % were African American, 23 % were non-Hispanic white, 16 % were Hispanic, and 30 % had been with their clinicians >5 years. Verbal dominance was higher for moderate-stress clinicians (ratio=1.93, p<0.01) and high-stress clinicians (ratio=1.76, p=0.01), compared with low-stress clinicians (ratio 1.45). More medical information was offered by moderate-stress clinicians (145.5 statements, p <0.01) and high-stress clinicians (125.9 statements, p=0.02), compared with low-stress clinicians (97.8 statements). High-stress clinicians offered less psychosocial information (17.1 vs. 19.3, p=0.02), and patients of high-stress clinicians rated their quality of care as excellent less frequently than patients of low-stress clinicians (49.5 % vs. 66.9 %, p<0.01). However, moderate-stress clinicians offered more partnering statements (27.7 vs. 18.2, p=0.04) and positive affect (3.88 vs. 3.78 score, p=0.02) than low-stress clinicians, and their patients' ratings did not differ. CONCLUSIONS: Although higher stress was associated with verbal dominance and lower patient ratings, moderate stress was associated with some positive communication behaviors. Prospective mixed methods studies should examine the complex relationships across the continuum of clinician well-being and health communication.

Effects of a focused patient-centered care curriculum on the experiences of internal medicine residents and their patients.

BACKGROUND: Traditional residency training may not promote competencies in patient-centered care. AIM: To improve residents' competencies in delivering patient-centered care. SETTING/PARTICIPANTS: Internal medicine residents at a university-based teaching hospital in Baltimore, Maryland. PROGRAM DESCRIPTION: One inpatient team admitted half the usual census and was exposed to a multi-modal patient-centered care curriculum to promote knowledge of patients as individuals, improve patient transitions of care, and reduce barriers to medication adherence. PROGRAM EVALUATION: Annual resident surveys (N = 40) revealed that the intervention was judged as professionally valuable (90%) and important to their training (90%) and offered experiences not available during other rotations (88%). Compared to standard inpatient rotation evaluations (n = 163), intervention rotation evaluations (n = 51) showed no differences in ratings for traditional medical learning, but higher ratings for improving how housestaff address patient medication adherence, communicate with patients about post-hospital transition of care, and know their patients as people (all p < 0.01). On post-discharge surveys, patients from the intervention team (N = 177, score 90.4, percentile ranking 97%) reported greater satisfaction with physicians than patients on standard teams (N = 924, score 86.1, percentile ranking 47%) p < 0.01). DISCUSSION: A patient-centered inpatient curriculum was associated with higher satisfaction ratings in patient-centered domains by internal medicine residents and with higher satisfaction ratings of their physicians by patients. Future research will explore the intervention's impact on clinical outcomes.

Quasi-experimental trial of diabetes Self-Management Automated and Real-Time Telephonic Support (SMARTSteps) in a Medicaid managed care plan: study protocol.

BACKGROUND: Health information technology can enhance self-management and quality of life for patients with chronic disease and overcome healthcare barriers for patients with limited English proficiency. After a randomized controlled trial of a multilingual automated telephone self-management support program (ATSM) improved patient-centered dimensions of diabetes care in safety net clinics, we collaborated with a nonprofit Medicaid managed care plan to translate research into practice, offering ATSM as a covered benefit and augmenting ATSM to promote medication activation. This paper describes the protocol of the Self-Management Automated and Real-Time Telephonic Support Project (SMARTSteps). METHODS/DESIGN: This controlled quasi-experimental trial used a wait-list variant of a stepped wedge design to enroll 362 adult health plan members with diabetes who speak English, Cantonese, or Spanish and receive care at 4 publicly-funded clinics. Through language-stratified randomization, participants were assigned to four intervention statuses: SMARTSteps-ONLY, SMARTSteps-PLUS, or wait-list for either intervention. In addition to usual primary care, intervention participants received 27 weekly calls in their preferred language with rotating queries and response-triggered education about self-care, medication adherence, safety concerns, psychological issues, and preventive services. Health coaches from the health plan called patients with out-of-range responses for collaborative goal setting and action planning. SMARTSteps-PLUS also included health coach calls to promote medication activation, adherence and intensification, if triggered by ATSM-reported non-adherence, refill non-adherence from pharmacy claims, or suboptimal cardiometabolic indicators. Wait-list patients crossed-over to SMARTSteps-ONLY or -PLUS at 6 months. For participants who agreed to structured telephone interviews at baseline and 6 months (n = 252), primary outcomes will be changes in quality of life and functional status with secondary outcomes of 6-month changes in self-management behaviors/efficacy and patient-centered processes of care. We will also evaluate 6-month changes in cardiometabolic (HbA1c, blood pressure, and LDL) and utilization indicators for all participants. DISCUSSION: Outcomes will provide evidence regarding real-world implementation of ATSM within a Medicaid managed care plan serving safety net settings. The evaluation trial will provide insight into translating and scaling up health information technology interventions for linguistically and culturally diverse vulnerable populations with chronic disease. TRIAL REGISTRATION: ClinicalTrials.gov: NCT00683020.

A SBIRT curriculum for medical residents: development of a performance feedback tool to build learner confidence.

A major barrier to actualizing the public health impact potential of screening, brief intervention, and referral to treatment (SBIRT) is the suboptimal development and implementation of evidence-based training curricula for healthcare providers. As part of a federal grant to develop and implement SBIRT training in medical residency programs, the authors assessed 95 internal medicine residents before they received SBIRT training to identify self-reported characteristics and behaviors that would inform curriculum development. Residents' confidence in their SBIRT skills significantly predicted SBIRT practice. Lack of experience dealing with alcohol or drug problems and discomfort in dealing with these issues were significantly associated with low confidence. To target these barriers, the authors revised their SBIRT curriculum to increase residents' confidence in their skills and developed an innovative SBIRT Proficiency Checklist and Feedback Protocol for skills practice observations. Qualitative feedback suggests that, despite the discomfort residents experience in being observed, a proficiency checklist and feedback protocol appear to boost learner confidence.

A video-intervention to improve clinician attitudes toward patients with sickle cell disease: the results of a randomized experiment.

BACKGROUND: Clinician attitudes toward patients are associated with variability in the quality of health care. Attitudes are typically considered difficult to change, and few interventions have attempted to do so. Negative attitudes toward adults with sickle cell disease have been identified as an important barrier to the receipt of appropriate pain management for this patient population. OBJECTIVE: To test the effect of a video-intervention designed to improve clinician attitudes toward adults with sickle cell disease. INTERVENTIONS: An 8-minute video depicting a clinician expert and patients discussing challenges in seeking treatment for sickle cell pain. DESIGN AND PARTICIPANTS: A randomized post-test only control group design was used to assess the impact of the intervention on the attitudes of 276 nurses and housestaff working at a large, urban, academic medical center. MAIN MEASURES: Attitudes toward adult sickle cell patients assessed using 5- and 6-point Likert-scale items. Exploratory factor analysis was used to identify underlying attitudinal domains and develop scales. Examples of the negative and positive attitudes assessed include clinician estimates of the percentage of SCD patients that exaggerate pain (negative) or make clinicians glad they went into medicine (positive). KEY RESULTS: Compared to the control group, the intervention group exhibited decreased negative attitudes (Difference in means = -8.9, 95%CI [-14.2, -3.6]; Cohen's d = 0.41), decreased endorsement of certain patient behaviors as "concern-raising" (Difference in means = -7.8, 95%CI [-13.1, -2.5]; Cohen's d = 0.36), and increased positive attitudes toward sickle cell patients (Difference in means = 6.6, 95% CI [0.6, 12.6]; Cohen's d = 0.27). CONCLUSIONS: Our results suggest that the attitudes of clinicians toward sickle cell patients may be improved through a short and relatively easy to implement intervention. Whether the attitudinal differences associated with our intervention are sustainable or are linked to clinical outcomes remains to be seen.

Religious coping and hospital admissions among adults with sickle cell disease.

Although a well-established literature implicates religiosity as a central element of the African American experience, little is known about how individuals from this group utilize religion to cope with specific health-related stressors. The present study examined the relation between religious coping and hospital admissions among a cohort of 95 adults with sickle cell disease-a genetic blood disorder that, in the United States, primarily affects people of African ancestry. Multiple regression analyses indicated that positive religious coping uniquely accounted for variance in hospital admissions after adjusting for other demographic and diagnostic variables. Specifically, greater endorsement of positive religious coping was associated with significantly fewer hospital admissions (ß = -.29, P < .05). These results indicate a need for further investigation of the roles that religion and spirituality play in adjustment to sickle cell disease and their influence on health care utilization patterns and health outcomes.

Informatics response to address the COVID-19 pandemic in a safety net healthcare system.

In service of particularly vulnerable populations, safety net healthcare systems must nimbly leverage health information technology (IT), including electronic health records (EHRs), to coordinate the medical and public health response to the novel coronavirus (COVID-19). Six months after the San Francisco Department of Public Health implemented a new EHR across its hospitals and citywide clinics, California declared a state of emergency in response to COVID-19. This paper describes how the IT and informatics teams supported San Francisco Department of Public Health's goals of expanding the safety net healthcare system capacity, meeting the needs of specific vulnerable populations, increasing equity in COVID-19 testing access, and expanding public health analytics and research capacity. Key enabling factors included critical partnerships with operational leaders, early identification of priorities, a clear governance structure, agility in the face of rapidly changing circumstances, and a commitment to vulnerable populations.

The association of provider communication with trust among adults with sickle cell disease.

BACKGROUND: Adults with sickle cell disease often report poor interpersonal healthcare experiences, including poor communication with providers. However, the effect of these experiences on patient trust is unknown. OBJECTIVE: To determine the association between patient ratings of the previous quality of provider communication and current trust in the medical profession among adults with sickle cell disease. RESEARCH DESIGN: Cross-sectional survey. PARTICIPANTS: A total of 95 adults with sickle cell disease. MEASUREMENTS: The four-item Provider Communication Subscale from the Consumer Assessment of Healthcare Plans and Systems Survey; The five-item Wake Forest Trust in the Medical Profession Scale. MAIN RESULTS: Better ratings of previous provider communication were significantly associated with higher levels of trust toward the medical profession. A 10% increase in provider communication rating was associated with a 3.76% increase in trust scores (p < 0.001, 95% CI [1.76%, 5.76%]), adjusting for patient-level demographic, clinical, and attitudinal characteristics. CONCLUSIONS: Poorer patient ratings of provider communication are associated with lower trust toward the medical profession among adults with sickle cell disease. Future research should examine the impact of low trust in the medical profession on clinical outcomes in this population of patients.

Evaluation of a Health Information Technology-Enabled Collective Intelligence Platform to Improve Diagnosis in Primary Care and Urgent Care Settings: Protocol for a Pragmatic Randomized Controlled Trial.

BACKGROUND: Diagnostic error in ambulatory care, a frequent cause of preventable harm, may be mitigated using the collective intelligence of multiple clinicians. The National Academy of Medicine has identified enhanced clinician collaboration and digital tools as a means to improve the diagnostic process. OBJECTIVE: This study aims to assess the efficacy of a collective intelligence output to improve diagnostic confidence and accuracy in ambulatory care cases (from primary care and urgent care clinic visits) with diagnostic uncertainty. METHODS: This is a pragmatic randomized controlled trial of using collective intelligence in cases with diagnostic uncertainty from clinicians at primary care and urgent care clinics in 2 health care systems in San Francisco. Real-life cases, identified for having an element of diagnostic uncertainty, will be entered into a collective intelligence digital platform to acquire collective intelligence from at least 5 clinician contributors on the platform. Cases will be randomized to an intervention group (where clinicians will view the collective intelligence output) or control (where clinicians will not view the collective intelligence output). Clinicians will complete a postvisit questionnaire that assesses their diagnostic confidence for each case; in the intervention cases, clinicians will complete the questionnaire after reviewing the collective intelligence output for the case. Using logistic regression accounting for clinician clustering, we will compare the primary outcome of diagnostic confidence and the secondary outcome of time with diagnosis (the time it takes for a clinician to reach a diagnosis), for intervention versus control cases. We will also assess the usability and satisfaction with the digital tool using measures adapted from the Technology Acceptance Model and Net Promoter Score. RESULTS: We have recruited 32 out of our recruitment goal of 33 participants. This study is funded until May 2020 and is approved by the University of California San Francisco Institutional Review Board until January 2020. We have completed data collection as of June 2019 and will complete our proposed analysis by December 2019. CONCLUSIONS: This study will determine if the use of a digital platform for collective intelligence is acceptable, useful, and efficacious in improving diagnostic confidence and accuracy in outpatient cases with diagnostic uncertainty. If shown to be valuable in improving clinicians' diagnostic process, this type of digital tool may be one of the first innovations used for reducing diagnostic errors in outpatient care. The findings of this study may provide a path forward for improving the diagnostic process. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/13151.

Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

OBJECTIVES: Usable tools to support individual primary care clinicians in their diagnostic processes could help to reduce preventable harm from diagnostic errors. We conducted a formative study with primary care providers to identify key requisites to optimize the acceptability of 1 online collective intelligence platform (Human Diagnosis Project; Human Dx). MATERIALS AND METHODS: We conducted semistructured interviews with practicing primary care clinicians in a sample of the US community-based clinics to examine the acceptability and early usability of the collective intelligence online platform using standardized clinical cases and real-world clinical cases from the participants' own practice. We used an integrated inductive-deductive qualitative analysis approach to analyze the interview transcripts. RESULTS AND DISCUSSION: Perceived usefulness, perceived accuracy, quality assurance, trust, and ease of use emerged as essential domains of acceptability required for providers to use a collective intelligence tool in clinical practice. Participants conveyed that the collective opinion should: (1) contribute to their clinical reasoning, (2) boost their confidence, (3) be generated in a timely manner, and (4) be relevant to their clinical settings and use cases. Trust in the technology platform and the clinical accuracy of its collective intelligence output emerged as an incontrovertible requirement for user acceptance and engagement. CONCLUSION: We documented key requisites to building a collective intelligence technology platform that is trustworthy, useful, and acceptable to target end users for assistance in the diagnostic process. These key lessons may be applicable to other provider-facing decision support platforms.

A Randomized Trial to Train Vulnerable Primary Care Patients to Use a Patient Portal.

BACKGROUND: Patient portals are becoming ubiquitous. Previous research has documented substantial barriers, especially among vulnerable patient subgroups such as those with lower socioeconomic status or limited health literacy (LHL). We tested the effectiveness of delivering online, video-based portal training to patients in a safety net setting. METHODS: We created an online video curriculum about accessing the San Francisco Health Network portal, and then randomized 93 English-speaking patients with 1+ chronic diseases to receive 1) an in-person tutorial with a research assistant, or 2) a link to view the videos on their own. We also examined a third, nonrandomized usual care comparison group. The primary outcome was portal log-in (yes/no) 3 to 6 months post-training, assessed via the electronic health record. Secondary outcomes were self-reported attitudes and skills collected via baseline and follow-up surveys. RESULTS: Mean age was 54 years, 51% had LHL, 60% were nonwhite, 52% were female, 45% reported fair/poor health, and 76% reported daily Internet use. At followup, 21% logged into the portal, with no differences by arm (P = .41), but this was higher than the overall clinic rate of 9% (P < .01) during the same time period. We found significant prepost improvements in self-rated portal skills (P = .03) and eHealth literacy (P < .01). Those with LHL were less likely to log in post-training (P < .01). CONCLUSIONS: Both modalities of online training were comparable, and neither mode enabled a majority of vulnerable patients to use portals, especially those with LHL. This suggests that portal training will need to be more intensive or portals need improved usability to meaningfully increase use among diverse patients.