Frameworks for Integrating Learning Analytics With the Electronic Health Record.

ABSTRACT: The information systems designed to support clinical care have evolved separately from those that support health professions education. This has resulted in a considerable digital divide between patient care and education, one that poorly serves practitioners and organizations, even as learning becomes ever more important to both. In this perspective, we advocate for the enhancement of existing health information systems so that they intentionally facilitate learning. We describe three well-regarded frameworks for learning that can point toward how health care information systems can best evolve to support learning. The Master Adaptive Learner model suggests ways that the individual practitioner can best organize their activities to ensure continual self-improvement. The PDSA cycle similarly proposes actions for improvement but at a health care organization's workflow level. Senge's Five Disciplines of the Learning Organization, a more general framework from the business literature, serves to further inform how disparate information and knowledge flows can be managed for continual improvement. Our main thesis holds that these types of learning frameworks should inform the design and integration of information systems serving the health professions. An underutilized mediator of educational improvement is the ubiquitous electronic health record. The authors list learning analytic opportunities, including potential modifications of learning management systems and the electronic health record, that would enhance health professions education and support the shared goal of delivering high-quality evidence-based health care.

Electronic Health Records That Support Health Professional Reflective Practice: a Missed Opportunity in Digital Health.

A foundational component of digital health involves collecting and leveraging electronic health data to improve health and wellbeing. One of the central technologies for collecting these data are electronic health records (EHRs). In this commentary, the authors explore intersection between digital health and data-driven reflective practice that is described, including an overview of the role of EHRs underpinning technology innovation in healthcare. Subsequently, they argue that EHRs are a rich but under-utilised source of information on the performance of health professionals and healthcare teams that could be harnessed to support reflective practice and behaviour change. EHRs currently act as systems of data collection, not systems of data engagement and reflection by end users such as health professionals and healthcare organisations. Further consideration should be given to supporting reflective practice by health professionals in the design of EHRs and other clinical information systems.

Gaps in preparedness of clergy and healthcare providers to address mental health needs of returning service members.

To elucidate gaps in the preparedness of clergy and healthcare providers to care for service members (SM) with deployment-related mental health needs. Participants identified clinically relevant symptoms in a standardized video role play of a veteran with deployment-related mental health needs and discussed their preparedness to deal with SM. Clergy members identified suicide and depression most often, while providers identified difficulty sleeping, low energy, nightmares and irritability. Neither clergy nor providers felt prepared to minister to or treat SM with traumatic brain injury. Through a mixed methods approach, we identified gaps in preparedness of clergy and healthcare providers in dealing with the mental health needs of SM.

Supporting the returning veteran: building linkages between clergy and health professionals.

OBJECTIVE: Rather than seeking psychiatric services, veterans often turn to clergy members as first responders to cope with exposure to traumatic events. The goal of this study was to evaluate clergy preparedness to assist with these issues and to determine if an educational symposium geared toward this population would increase preparedness and collaboration with psychiatrists. METHODS: A pre- and post-conference survey was administered to clergy members who attended an educational symposium on the benefits of collaboration between psychiatry and spirituality for service members. RESULTS: Analyses found that clergy frequently self-reported difficulties recognizing symptoms of posttraumatic stress disorder (PTSD) and traumatic brain injury (TBI) and identifying resources. After attending a symposium on the topic, follow-up data indicated significant increases in the clergy's preparedness to address traumatic events. CONCLUSIONS: Educational programs may assist clergy in filling knowledge gaps related to recognizing symptoms of PTSD and TBI and providing resources to veterans and their family members.

Autonomic arousal and learning in Web-based simulation: a feasibility study.

INTRODUCTION: Autonomic arousal is an important component of understanding learning as it is related to cognitive effort, attention, and emotional arousal. Currently, however, little is known about its relationship to online education. We conducted a study to determine the feasibility of measuring autonomic arousal and engagement in online continuing medical education (CME). METHOD: Using the Computer Simulation Assessment Tool (CSAT) platform, health care providers were randomly assigned to either high- or low-valence versions of a Web-based simulation on risk assessment for a returning veteran. Data were collected on participants' actions within the simulation, self-reported cognitive engagement, knowledge retention, and autonomic arousal measured using galvanic skin response (GSR). RESULTS: Participants in the high-valence condition (n = 7) chose a lower percentage of critical actions (M = 79.2, SD = 4.2) than participants in the low valence (n = 8) condition (M = 83.9, SD = 3.6, t(1,14) = 2.44, p = .03). While not statistically significant, high-valence participants reported higher cognitive engagement. Participants in the high-valence condition showed a larger increase in physiologic arousal when comparing mean tonic GSR during the orientation simulation to the study simulation (high-valence mean difference = 4.21 µS, SD = 1.23 vs low-valence mean difference = 1.64 µS, SD = 2.32, t(1,13) = -2.62, p = .01). DISCUSSION: In addition to being consistent with previous engagement research, this experiment functioned as a feasibility study for measuring autonomic arousal in online CME. The current study provides a framework for future studies, which may use neurophysiology to identify the critical autonomic and engagement components associated with effective online learning.

Interprofessional differences in disposition decisions: results from a standardized web-based patient assessment.

OBJECTIVE: This study examined differences in disposition decisions among mental health professionals using a standardized Web-based simulation. METHODS: Using a Web-based simulation that described, across users, the same complex psychiatric patient, credentialed clinicians in a psychiatry department conducted a violence risk assessment and selected a level of follow-up care. RESULTS: Of 410 clinicians who completed the simulation, 60% of psychiatrists were more likely than other types of clinicians to select higher levels of care (inpatient or emergency services) for the standardized virtual patient (odds ratio=2.67, 95% confidence interval=1.67-4.25), even after adjustment for other factors. Virtual actions taken, such as contracting with the patient for safety and discussing hospitalization, elucidated these training differences. CONCLUSIONS: Training backgrounds were important determinants of clinicians' actions and the dispositions they recommended for a psychiatric patient at high risk of self-harm and harm to others in the educational setting and may suggest the need for further training to standardize and optimize care.

Simulation-based ongoing professional practice evaluation in psychiatry: a novel tool for performance assessment.

BACKGROUND: Ongoing professional practice evaluation (OPPE) activities consist of a quantitative, competency-based evaluation of clinical performance. Hospitals must design assessments that measure clinical competencies, are scalable, and minimize impact on the clinician's daily routines. A psychiatry department at a large academic medical center designed and implemented an interactive Web-based psychiatric simulation focusing on violence risk assessment as a tool for a departmentwide OPPE. METHODS: Of 412 invited clinicians in a large psychiatry department, 410 completed an online simulation in April-May 2012. Participants received scheduled e-mail reminders with instructions describing how to access the simulation. Using the Computer Simulation Assessment Tool, participants viewed an introductory video and were then asked to conduct a risk assessment, acting as a clinician in the encounter by selecting actions from a series of drop-down menus. Each action was paired with a corresponding video segment of a clinical encounter with a standardized patient. Participants were scored on the basis of their actions within the simulation (Measure 1) and by their responses to the open-ended questions in which they were asked to integrate the information from the simulation in a summative manner (Measure 2). RESULTS: Of the 410 clinicians, 381 (92.9%) passed Measure 1,359 (87.6%) passed Measure 2, and 5 (1.2%) failed both measures. Seventy-five (18.3%) participants were referred for focused professional practice evaluation (FPPE) after failing either Measure 1, Measure 2, or both. CONCLUSIONS: Overall, Web-based simulation and e-mail engagement tools were a scalable and efficient way to assess a large number of clinicians in OPPE and to identify those who required FPPE.

Attendees' perceptions of commercial influence in noncommercially funded CME programs.

INTRODUCTION: The controversy surrounding commercial support for continuing medical education (CME) programs has led to policy changes, but data show no significant difference in perceived bias between commercial and noncommercial CME. Indeed, what attendees perceive as commercial influence is not fully understood. We sought to clarify what sources contribute to attendees' perceptions of commercial influence in non-industry-supported CME programs, and how attendees perceive that this influence manifests itself on both speaker and program levels. METHODS: Evaluation forms were received from 1 544 attendees at 14 live noncommercially supported CME programs in 2006, 2007, and 2010. Attendees rated perceived commercial influence for each lecture and the entire program. Using open-ended and "check all that apply'' questions, participants specified perceived sources and manifestations of commercial influence. RESULTS: Attendees rating individual lectures but not the entire program as commercially influenced accounted for 59.9% of those who identified bias. The most frequently endorsed source of commercial influence was individual speakers' funding, and the most listed manifestations were speakers' mentions of pharmaceuticals and expressions of personal opinions. Rating the entire program commercially influenced correlated with whether attendees considered the funding of referenced research a source of influence. DISCUSSION: CME attendees consider a broad spectrum of factors when reporting commercial influence. Evaluation forms should include bias questions per lecture as well as items to clarify perceived sources and manifestations of commercial influence.

Enhancing informed consent best practices: gaining patient, family and provider perspectives using reverse simulation.

BACKGROUND: Obtaining informed consent in the clinical setting is an important yet challenging aspect of providing safe and collaborative care to patients. While the medical profession has defined best practices for obtaining informed consent, it is unclear whether these standards meet the expressed needs of patients, their families as well as healthcare providers. The authors sought to address this gap by comparing the responses of these three groups with a standardised informed consent paradigm. METHODS: Piloting a web-based 'reverse' simulation paradigm, participants viewed a video showing a standardised doctor engaging in an informed consent discussion. The scenario depicted a simulated patient with psychotic symptoms who is prescribed an atypical antipsychotic medication. 107 participants accessed the simulation online and completed a web-based debriefing survey. RESULTS: Survey responses from patients, family members and healthcare providers indicated disparities in information retention, perception of the doctor's performance and priorities for required elements of the consent process. CONCLUSIONS: To enhance existing informed consent best practices, steps should be taken to improve patient retention of critical information. Adverse events should be described in the short-term and long-term along with preventative measures, and alternative psychosocial and pharmacological treatment options should be reviewed. Information about treatment should include when the medication takes therapeutic effect and how to safely maintain the treatment. The reverse simulation design is a model that can discern gaps in clinical practice, which can be used to improve patient care.

Attention to body mass index by child psychiatry providers when prescribing second-generation antipsychotic medication to children: a survey study using a clinical vignette.

AIM: We sought to determine whether weight and body mass index measurement were taken into consideration when prescribing second-generation antipsychotic (SGA) medication to a child. METHODS: Two hundred clinicians were surveyed using a hypothetical clinical case vignette at a child psychopharmacology, postgraduate medical education course. The vignette described an overweight 10-year-old boy who was about to be prescribed an SGA medication to control psychotic symptoms. The reference to the patient's being 'overweight' was purposefully included to determine if providers would assess the patient's risk of morbidity from the metabolic side effects of the SGAs at the time of prescribing. RESULTS: Only 7.0% of prescribers listed either 'body mass index' or a combination of 'height' and 'weight' as part of their next treatment steps for an overweight child before prescribing an SGA. CONCLUSIONS: These results suggest the need for education as to the importance of body mass index monitoring when prescribing second-generation antipsychotic medications to children.

Web-based simulation in psychiatry residency training: a pilot study.

BACKGROUND: Medical specialties, including surgery, obstetrics, anesthesia, critical care, and trauma, have adopted simulation technology for measuring clinical competency as a routine part of their residency training programs; yet, simulation technologies have rarely been adapted or used for psychiatry training. OBJECTIVE: The authors describe the development of a web-based computer simulation tool intended to assess physician competence in obtaining informed consent before prescribing antipsychotic medication to a simulated patient with symptoms of psychosis. METHOD: Eighteen residents participated in a pilot study of the Computer Simulation Assessment Tool (CSAT). Outcome measures included physician performance on required elements, pre- and post-test measures of physician confidence in obtaining informed consent, and levels of system usability. RESULTS: Data suggested that the CSAT increased physician confidence in obtaining informed consent and that it was easy to use. CONCLUSIONS: The CSAT was an effective educational tool in simulating patient-physician interactions, and it may serve as a model for use of other web-based simulations to augment traditional teaching methods in residency education.

Physicians' perceptions, preparedness for reporting, and experiences related to impaired and incompetent colleagues.

CONTEXT: Peer monitoring and reporting are the primary mechanisms for identifying physicians who are impaired or otherwise incompetent to practice, but data suggest that the rate of such reporting is lower than it should be. OBJECTIVE: To understand physicians' beliefs, preparedness, and actual experiences related to colleagues who are impaired or incompetent to practice medicine. DESIGN, SETTING, AND PARTICIPANTS: Nationally representative survey of 2938 eligible physicians practicing in the United States in 2009 in anesthesiology, cardiology, family practice, general surgery, internal medicine, pediatrics, and psychiatry. Overall, 1891 physicians (64.4%) responded. MAIN OUTCOME MEASURES: Beliefs about and preparedness for reporting and experiences with colleagues who practice medicine while impaired or who are incompetent in their medical practice. RESULTS: Sixty-four percent (n = 1120) of surveyed physicians agreed with the professional commitment to report physicians who are significantly impaired or otherwise incompetent to practice. Nonetheless, only 69% (n = 1208) of physicians reported being prepared to effectively deal with impaired colleagues in their medical practice, and 64% (n = 1126) reported being so prepared to deal with incompetent colleagues. Seventeen percent (n = 309) of physicians had direct personal knowledge of a physician colleague who was incompetent to practice medicine in their hospital, group, or practice. Of those with this knowledge, 67% (n = 204) reported this colleague to the relevant authority. Underrepresented minorities and graduates of non-US medical schools were less likely than their counterparts to report, and physicians working in hospitals or medical schools were most likely to report. The most frequently cited reason for taking no action was the belief that someone else was taking care of the problem (19% [n = 58]), followed by the belief that nothing would happen as a result of the report (15% [n = 46]) and fear of retribution (12% [n = 36]). CONCLUSION: Overall, physicians support the professional commitment to report all instances of impaired or incompetent colleagues in their medical practice to a relevant authority; however, when faced with these situations, many do not report.

Assessing clinicians' management of first episode schizophrenia using clinical case vignettes.

BACKGROUND: Patients with first episode schizophrenia may present in a variety of clinical settings to providers who have a range of knowledge and skills. A thoughtful workup of patients with new-onset psychosis is critical, and the treatment of first episode schizophrenia differs from that of chronic psychotic disorders. Clinical case vignettes with free-form responses can be used to carefully assess whether front line practitioners provide guideline-adherent management of first episode psychosis. METHODS: A clinical case vignette, presenting a patient with first episode schizophrenia, was created and administered to the attendees of a continuing medical education programme. Free-form responses to questions regarding differential diagnosis, workup, treatment and treatment duration were scored based on published practice guidelines. Response frequencies were tabulated and performance was compared among professional disciplines. RESULTS: Sixty-two attendees completed the vignette. Though the attendees typically considered a broad differential diagnosis and appropriately initiated treatment with antipsychotics, the respondents' proposed medical workup was limited, and they prescribed antipsychotics at higher doses and for a shorter duration than recommended in the literature. The prescribers outperformed the non-prescribers on treatment questions (P = 0.006), but the two groups' performance did not significantly differ on the assessment questions (P = 0.08). CONCLUSIONS: The front line clinicians who encounter patients with first episode schizophrenia may have significant practice gaps in the initial and follow-up care of these patients. Given the preliminary nature of this study and the debate about the optimal care for first episode psychosis, further study is needed. If such gaps are confirmed, additional educational interventions are required to align clinical management with published practice guidelines.

Reach of benchmark psychiatric trial results to community-based providers: a case study of CATIE.

OBJECTIVE: To evaluate the familiarity of front-line clinicians with findings from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE), the influence of didactic continuing medical education on provider knowledge about key details of CATIE, and how location-related factors affect rates of pre-event knowledge and intraevent learning about CATIE. METHOD: Data derived from the Massachusetts General Hospital Psychiatry Academy (MGH-PA) semester II live symposia provided in different cities nationally between September and December 2006 were analyzed to evaluate providers' self-assessment of their knowledge about CATIE. In addition, participants were also asked a preactivity and postactivity question to assess learning of material presented during the live event psychosis lecture. Descriptive statistics were utilized to characterize participants' self-assessment of knowledge about CATIE, while parametric and nonparametric statistical tests were used to evaluate the degree of observed learning and the effect of lecture location on the results. RESULTS: 3333 participants (mean attendance: N = 278 per event) attended 1 of the 12 MGH-PA live symposia. Of the subsample of providers who treat schizophrenia, 51% indicated that either they had never heard of CATIE or they were not familiar enough with its results to change their practice. Overall, the proportion of correct answers on the postactivity question was 65%, compared with 24% prior to the lecture (chi(2) = 48.68, df = 1, p < .001). Degree of learning did not differ among symposium locations. CONCLUSION: In this sample, the CATIE study had very limited dissemination to, and impact on, a geographically and occupationally diverse sample of mental health practitioners. Robust learning of a key methodologic detail of this trial was evidenced across symposium locations.

A survey of five antidepressant properties influencing clinician's treatment choices in MDD.

INTRODUCTION: The goal of the present work was to examine how clinicians' perceptions of the properties of antidepressants may influence their choice of antidepressants when treating major depressive disorder (MDD). METHODS: 273 of 682 (40%) clinicians attending a psychopharmacology review course responded to a questionnaire designed to explore their practices and perceptions with regards to antidepressant pharmacotherapy. RESULTS: Most clinicians ranked efficacy (57.3%) as the most important factor when selecting antidepressants, followed by safety (23.0%), tolerability (9.4%), rapidity of action (5.2%), and cost (4.9%). However, when presented with hypothetical scenarios in which there was a difference in efficacy between two antidepressant agents, the relative safety, tolerability, and cost of the two agents significantly influenced the likelihood of choosing one antidepressant over another. In fact, clinicians required progressively greater differences in efficacy between two agents in order to select one antidepressant over another given a difference in terms of their safety than tolerability, or their tolerability than cost (p < 0.0001 all comparisons). CONCLUSIONS: When selecting an antidepressant, clinicians appear to be most influenced by efficacy, followed by safety. Rapidity of action and cost may be less salient considerations in clinical practice. Further research is necessary to elucidate factors that influence clinicians' choice of antidepressants.