Feasibility study of an EHR-integrated mobile shared decision making application.

INTRODUCTION: Integrating mobile applications (apps) into users' standard electronic health record (EHR) workflows may be valuable, especially for apps that both read and write data. This report details the lessons learned during the integration of a patient decision aid - prostate specific antigen (PSA) testing for prostate cancer screening - into our users' standard EHR workflow for a small usability assessment. MATERIALS AND METHODS: This feasibility study included two steps. First we enabled realtime, secure bidirectional data exchange between the mobile app and EHR for 14 data elements, and second we pilot tested the production environment app with 9 primary care patients aged 60-65 years. Our primary usability metric was a net promoter score (NPS), based on users' recommendation of the app to a friend or family member; we also assessed the proportion of users who 1) updated their prostate cancer risk factor information present in the EHR and 2) submitted more than one unique response regarding their preference to have PSA testing. RESULTS: The seven web services necessary to read and write data required considerable configuration, but successfully delivered risk factor-specific educational content and recorded patients' values and decision preference directly within the EHR. Seven of the 9 patients (78%) would recommend this app to a friend/family member (NPS = 55.6%), one patient used the app to update risk factor information, and 4/9 (44%) changed their decision preference while using the app. CONCLUSIONS: It is feasible to implement a decision aid directly into users' standard EHR workflow for limited usability testing. Broad scale implementation may have a positive effect on patient engagement and improve shared decision making, but several challenges exist with proprietary EHR vendor application programming interfaces (API)s.

Increasing confidence and changing behaviors in primary care providers engaged in genetic counselling.

BACKGROUND: Screening and counseling for genetic conditions is an increasingly important part of primary care practice, particularly given the paucity of genetic counselors in the United States. However, primary care physicians (PCPs) often have an inadequate understanding of evidence-based screening; communication approaches that encourage shared decision-making; ethical, legal, and social implication (ELSI) issues related to screening for genetic mutations; and the basics of clinical genetics. This study explored whether an interactive, web-based genetics curriculum directed at PCPs in non-academic primary care settings was superior at changing practice knowledge, attitudes, and behaviors when compared to a traditional educational approach, particularly when discussing common genetic conditions. METHODS: One hundred twenty one PCPs in California and Pennsylvania physician practices were randomized to either an Intervention Group (IG) or Control Group (CG). IG physicians completed a 6 h interactive web-based curriculum covering communication skills, basics of genetic testing, risk assessment, ELSI issues and practice behaviors. CG physicians were provided with a traditional approach to Continuing Medical Education (CME) (clinical review articles) offering equivalent information. RESULTS: PCPs in the Intervention Group showed greater increases in knowledge compared to the Control Group. Intervention PCPs were also more satisfied with the educational materials, and more confident in their genetics knowledge and skills compared to those receiving traditional CME materials. Intervention PCPs felt that the web-based curriculum covered medical management, genetics, and ELSI issues significantly better than did the Control Group, and in comparison with traditional curricula. The Intervention Group felt the online tools offered several advantages, and engaged in better shared decision making with standardized patients, however, there was no difference in behavior change between groups with regard to increases in ELSI discussions between PCPs and patients. CONCLUSION: While our intervention was deemed more enjoyable, demonstrated significant factual learning and retention, and increased shared decision making practices, there were few differences in behavior changes around ELSI discussions. Unfortunately, barriers to implementing behavior change in clinical genetics is not unique to our intervention. Perhaps the missing element is that busy physicians need systems-level support to engage in meaningful discussions around genetics issues. The next step in promoting active engagement between doctors and patients may be to put into place the tools needed for PCPs to easily access the materials they need at the point-of-care to engage in joint discussions around clinical genetics.

Impact of a randomized controlled educational trial to improve physician practice behaviors around screening for inherited breast cancer.

BACKGROUND: Many primary care physicians (PCPs) are ill-equipped to provide screening and counseling for inherited breast cancer. OBJECTIVE: To evaluate the outcomes of an interactive web-based genetics curriculum versus text curriculum for primary care physicians. DESIGN: Randomized two-group design. PARTICIPANTS: 121 California and Pennsylvania community physicians. INTERVENTION: Web-based interactive genetics curriculum, evaluated against a control group of physicians who studied genetics review articles. After education, physicians interacted with an announced standardized patient (SP) at risk for inherited breast cancer. MAIN MEASURES: Transcripts of visit discussions were coded for presence or absence of 69 topics relevant to inherited breast cancer. KEY RESULTS: Across all physicians, history-taking, discussions of test result implications, and exploration of ethical and legal issues were incomplete. Approximately half of physicians offered a genetic counseling referral (54.6%), and fewer (43.8%) recommended testing. Intervention physicians were more likely than controls to explore genetic counseling benefits (78.3% versus 60.7%, P = 0.048), encourage genetic counseling before testing (38.3% versus 21.3%, P = 0.048), ask about a family history of prostate cancer (25.0% versus 6.6%, P = 0.006), and report that a positive result indicated an increased risk of prostate cancer for male relatives (20.0% versus 1.6%, P = 0.001). Intervention-group physicians were less likely than controls to ask about Ashkenazi heritage (13.3% versus 34.4%, P = 0.01) or to reply that they would get tested when asked, "What would you do?" (33.3% versus 54.1%, P = 0.03). CONCLUSIONS: Physicians infrequently performed key counseling behaviors, and this was true regardless of whether they had completed the web-based interactive training or read clinical reviews.

A comparison of Web-based and small-group palliative and end-of-life care curricula: a quasi-randomized controlled study at one institution.

PURPOSE: Few studies have compared the effect of Web-based eLearning versus small-group learning on medical student outcomes. Palliative and end-of-life (PEOL) education is ideal for this comparison, given uneven access to PEOL experts and content nationally. METHOD: In 2010, the authors enrolled all third-year medical students at the University of California, Davis School of Medicine into a quasi-randomized controlled trial of Web-based interactive education (eDoctoring) compared with small-group education (Doctoring) on PEOL clinical content over two months. Students participated in three 3-hour PEOL sessions with similar content. Outcomes included a 24-item PEOL-specific self-efficacy scale with three domains (diagnosis/treatment [Cronbach alpha=0.92; CI: 0.91-0.93], communication/prognosis [alpha=0.95; CI: 0.93-0.96], and social impact/self-care [alpha=0.91; CI: 0.88-0.92]); 8 knowledge items; 10 curricular advantage/disadvantages; and curricular satisfaction (both students and faculty). RESULTS: Students were randomly assigned to Web-based eDoctoring (n=48) or small-group Doctoring (n=71) curricula. Self-efficacy and knowledge improved equivalently between groups (e.g., prognosis self-efficacy, 19%; knowledge, 10%-42%). Student and faculty ratings of the Web-based eDoctoring curriculum and the small-group Doctoring curriculum were equivalent for most goals, and overall satisfaction was equivalent for each, with a trend toward decreased eDoctoring student satisfaction. CONCLUSIONS: Findings showed equivalent gains in self-efficacy and knowledge between students participating in a Web-based PEOL curriculum in comparison with students learning similar content in a small-group format. Web-based curricula can standardize content presentation when local teaching expertise is limited, but it may lead to decreased user satisfaction.

Physician communication regarding prostate cancer screening: analysis of unannounced standardized patient visits.

PURPOSE: Prostate cancer screening with prostate-specific antigen (PSA) is a controversial issue. The present study aimed to explore physician behaviors during an unannounced standardized patient encounter that was part of a randomized controlled trial to educate physicians using a prostate cancer screening, interactive, Web-based module. METHODS: Participants included 118 internal medicine and family medicine physicians from 5 health systems in California, in 2007-2008. Control physicians received usual education about prostate cancer screening (brochures from the Center for Disease Control and Prevention). Intervention physicians participated in the prostate cancer screening module. Within 3 months, all physicians saw unannounced standardized patients who prompted prostate cancer screening discussions in clinic. The encounter was audio-recorded, and the recordings were transcribed. Authors analyzed physician behaviors around screening: (1) engagement after prompting, (2) degree of shared decision making, and (3) final recommendations for prostate cancer screening. RESULTS: After prompting, 90% of physicians discussed prostate cancer screening. In comparison with control physicians, intervention physicians showed somewhat more shared decision making behaviors (intervention 14 items vs control 11 items, P <.05), were more likely to mention no screening as an option (intervention 63% vs control 26%, P <.05), to encourage patients to consider different screening options (intervention 62% vs control 39%, P <.05) and seeking input from others (intervention 25% vs control 7%, P<.05). CONCLUSIONS: A brief Web-based interactive educational intervention can improve shared decision making, neutrality in recommendation, and reduce PSA test ordering. Engaging patients in discussion of the uses and limitations of tests with uncertain value can decrease utilization of the tests.

Pairing physician education with patient activation to improve shared decisions in prostate cancer screening: a cluster randomized controlled trial.

BACKGROUND: Most expert groups recommend shared decision making for prostate cancer screening. Most primary care physicians, however, routinely order a prostate-specific antigen (PSA) test with little or no discussion about whether they believe the potential benefits justify the risk of harm. We sought to assess whether educating primary care physicians and activating their patients to ask about prostate cancer screening had a synergistic effect on shared decision making, rates and types of discussions about prostate cancer screening, and the physician's final recommendations. METHODS: Our study was a cluster randomized controlled trial among primary care physicians and their patients, comparing usual education (control), with physician education alone (MD-Ed), and with physician education and patient activation (MD-Ed+A). Participants included 120 physicians in 5 group practices, and 712 male patients aged 50 to 75 years. The interventions comprised a Web-based educational program for all intervention physicians and MD-Ed+A patients compared with usual education (brochures from the Centers for Disease Control and Prevention). The primary outcome measure was patients' reported postvisit shared decision making regarding prostate cancer screening; secondary measures included unannounced standardized patients' reported shared decision making and the physician's recommendation for prostate cancer screening. RESULTS: Patients' ratings of shared decision making were moderate and did not differ between groups. MD-Ed+A patients reported that physicians had higher prostate cancer screening discussion rates (MD-Ed+A = 65%, MD-Ed = 41%, control=38%; P <.01). Standardized patients reported that physicians seeing MD-Ed+A patients were more neutral during prostate cancer screening recommendations (MD-Ed+A=50%, MD-Ed=33%, control=15%; P <.05). Of the male patients, 80% had had previous PSA tests. CONCLUSIONS: Although activating physicians and patients did not lead to significant changes in all aspects of physician attitudes and behaviors that we studied, interventions that involved physicians did have a large effect on their attitudes toward screening and in the discussions they had with patients, including their being more likely than control physicians to engage in prostate cancer screening discussions and more likely to be neutral in their final recommendations.

Implementation outcomes of a multiinstitutional web-based ethical, legal, and social implications genetics curriculum for primary care residents in three specialties.

PURPOSE: Medical genetics lends itself to disseminated teaching methods because of mismatches between numbers of physicians having patients with genetic disorders and availability of genetic specialists. METHOD: During 3 years, we implemented an interactive, web-based curriculum on ethical, legal, and social implications in medical genetics for primary care residents in three specialties at three institutions. Residents took five (of 10) cases and three (of five) tutorials that varied by specialty. We assessed changes in self-efficacy (primary outcome), knowledge, application, and viewpoints. RESULTS: Overall enrollment was 69% (279/403). One institution did not complete implementation and was dropped from pre-post comparisons. We developed a six-factor ethical, legal, and social implications self-efficacy scale (Cronbach α = 0.95). Baseline self-efficacy was moderate (71/115; range: 23-115) and increased 15% after participation. Pre-post knowledge scores were high and unchanged. Residents reported that this curriculum covered ethical, legal, and social implications/genetics better than their usual curricula. Most (68-91%) identified advantages, especially in providing flexibility and stimulating self-directed learning. After participation, residents reported creating learning goals (66%) and acting on those goals (62%). CONCLUSIONS: Ethical, legal, and social implications genetics curricular participation led to modest self-efficacy gains. Residents reported that the curriculum covered unique content areas, had advantages over traditional curriculum, and that they applied ethical, legal, and social implications content clinically. We share lessons from developing and implementing this complex web-based curriculum across multiple institutions.

'Annals of Emergency Medicine Journal Club: the measurement of reliability.

Editor's capsule summary for Cruz et al: (1) WHAT IS ALREADY KNOWN ON THIS TOPIC: Valid clinical research requires high-quality data collection. Physicians are commonly considered the standard by which valid prospective data are obtained. WHAT QUESTION THIS STUDY ADDRESSED: This study determined whether non-medically trained research assistants could reliably collect subjective historical data from emergency department patients with chest pain. WHAT THIS STUDY ADDS TO OUR KNOWLEDGE: This prospective comparative study included 33 research assistants, 39 physicians, and 143 patients. Research assistants demonstrated fair to excellent reliability (as defined by crude agreement and kappa) when obtaining cardiac histories and cardiac risk factors. HOW THIS MIGHT CHANGE CLINICAL PRACTICE: The results of this study will not change clinical practice. They do, however, provide evidence to support the use of trained research assistants for the collection of certain types of clinical data.

A test of knowledge about prostate cancer screening. Online pilot evaluation among Southern California Physicians.

BACKGROUND: Although the benefits of prostate cancer screening are uncertain and guidelines recommend that physicians share the screening decision with their patients, most U.S. men over age 50 are routinely screened, often without counseling. OBJECTIVE: To develop an instrument for assessing physicians' knowledge related to the U.S. Preventive Services Task Force recommendations on prostate cancer screening. PARTICIPANTS: Seventy internists, family physicians, and general practitioners in the Los Angeles area who deliver primary care to adult men. MEASUREMENTS: We assessed knowledge related to prostate cancer screening (natural history, test characteristics, treatment effects, and guideline recommendations), beliefs about the net benefits of screening, and prostate cancer screening practices for men in different age groups, using an online survey. We constructed a knowledge scale having 15 multiple-choice items. RESULTS: Participants' mean knowledge score was 7.4 (range 3 to 12) of 15 (Cronbach's alpha=0.71). Higher knowledge scores were associated with less belief in a mortality benefit from prostate-specific antigen (PSA) testing (r=-.49, P<.001). Participants could be categorized as low, age-selective, and high users of routine PSA screening. High users had lower knowledge scores than age-selective or low users, and they believed much more in mortality benefits from PSA screening. CONCLUSIONS: Based on its internal consistency and its correlations with measures of physicians' net beliefs and self-reported practices, the knowledge scale developed in this study holds promise for measuring the effects of professional education on prostate cancer screening. The scale deserves further evaluation in broader populations.

The stadium diagram, a web-based tool for visualizing the expected outcomes of alternative clinical management strategies.

OBJECTIVE: To explore the use of graphical animation for helping clinicians to understand the evidence about expected risks and benefits associated with multi-step clinical management strategies. METHODS: We used Flash and XML to create a tool capable of displaying the sequence of health state changes that may result from a specific management strategy, as applied to a simulated population. We evaluated this tool in 6 focus groups involving a total of 44 community internists and family physicians. We successively revised the tool based on grounded theory analysis of the focus group transcripts. RESULTS: The process of responding to design issues raised in focus groups resulted in a final tool that presents a group of person icons arranged in rows to give the illusion of people in a stadium or theater. Each action in the management strategy causes persons to change color and move among rows to reflect changes in health state. The tool can play audio narration to explain each step and links are provided to the supporting evidence. Most physicians found these visualizations to be attractive and clear. Some were interested in using the tool with patients. Others rejected the specific decision model used to demonstrate the tool and a few rejected the notion of applying quantitative risks to individual patients. CONCLUSIONS: A visual approach to demonstrating the possible benefits and harms of a given management strategy holds interest for many clinicians. However, visualizations may fail to influence clinicians who do not believe the available evidence.

Automated linking of free-text complaints to reason-for-visit categories and International Classification of Diseases diagnoses in emergency department patient record databases.

STUDY OBJECTIVE: The use of the International Classification of Diseases system to describe emergency department (ED) case mix has disadvantages. We therefore developed computer algorithms that recognize a combination of words, word fragments, and word patterns to link free-text complaint fields to 20 reason-for-visit categories. We examine the feasibility and reliability of applying these reason-for-visit categories to ED patient-visit databases. METHODS: We analyzed a database (containing complaints and International Classification of Diseases diagnoses for 1 year's visits to a single ED) using a 3-step process (create initial terms, maximize sensitivity, maximize specificity) to define inclusion and exclusion terms for 20 reason-for-visit categories. To assess the reliability of the reason-for-visit assignment algorithm, we repeated the final 2 steps on a second database, composed of visits sampled from 21 EDs. For each database, we determined the prevalence of complaints that link to each reason-for-visit category and the distributions of International Classification of Diseases, Ninth Revision diagnoses that resulted for all patients and patients stratified by age. RESULTS: The 20 reason-for-visit categories capture 77% of all patients in database 1 (mean age 33.5 years) and 67% of all patients in database 2 (mean age 38.9 years). The percentage of visits captured by the 20 reason-for-visit categories, by age range, for databases 1 and 2 are (respectively) 0 to 2 years (84% and 76%), 3 to 10 years (82% and 74%), 11 to 65 years (76% and 68%), and 66 years or older (69% and 60%). The proportions of all complaints that link to each reason-for-visit category are largely similar between databases. Every complaint field that is linked to each reason-for-visit category includes at least 1 term that relates it to the category title, and the most frequently assigned diagnoses in each reason-for-visit category are those that one would expect to be associated with the reason-for-visit category complaints. CONCLUSION: The method by which free-text complaint fields are parsed into reason-for-visit categories is feasible and reasonably reliable; the finalized database 1 reason-for-visit category inclusion/exclusion terms lists required only modest changes to work well in database 2. The reason-for-visit categories used here are broadly defined to maximize the proportion of visits that they capture; more narrowly defined reason-for-visit categories will require more extensive revision of their inclusion/exclusion terms lists when used in different databases. A prospective, reason-for-visit-based ED classification system could have several useful applications (including syndromic surveillance), although content validity analysis will be necessary to investigate this hypothesis.

The use of dedicated methodology and statistical reviewers for peer review: a content analysis of comments to authors made by methodology and regular reviewers.

STUDY OBJECTIVES: In 1997, Annals of Emergency Medicine initiated a protocol by which every original research article, in addition to each regular review, was concurrently evaluated by 1 of 2 methodology and statistical reviewers. We characterized and contrasted comments made by the methodology and regular peer reviewers. METHODS: After pilot testing, interrater reliability assessment, and revision, we finalized a 99-item taxonomy of reviewer comments organized in 8 categories. Two authors, uninvolved in the writing of reviews, classified each comment from a random sample of methodology reviews from 1999. For 30 of these reviews (15 for each methodology reviewer), the 2 authors also scored all (range 2 to 5) regular reviews. RESULTS: Sixty-five reviews by methodologist A, 60 by methodologist B, and 68 by regular reviewers were analyzed. Comments by methodologist A most frequently concerned the presentation of results (33% of all comments) and methods (17%). Methodologist B commented most frequently on presentation of results (28%) and statistical methods (16%). Regular reviewers most frequently made non-methodology/statistical comments (45%) and comments on presentation of results (18%). Of note, comments made by methodology and regular reviewers about methods issues were often contradictory. CONCLUSION: The distributions of comments made by the 2 methodology and statistical reviewers were similar, although reviewer A emphasized presentation and reviewer B stressed statistical issues. The regular reviewers (most of whom were unaware that a dedicated methodology and statistical reviewer would be reviewing the article) paid much less attention to methodology issues. The 2 dedicated methodology and statistical reviewers created reviews that were similarly focused and emphasized methodology issues that were distinct from the issues raised by regular reviewers.