Evidence reversals in primary care research: a study of randomized controlled trials.

BACKGROUND: Evidence-Based Medicine is built on the premise that clinicians can be more confident when their decisions are grounded in high-quality evidence. Furthermore, evidence from studies involving patient-oriented outcomes is preferred when making decisions about tests or treatments. Ideally, the findings of relevant and valid trials should be stable over time, that is, unlikely to be reversed in subsequent research. OBJECTIVE: To evaluate the stability of evidence from trials relevant to primary healthcare and to identify study characteristics associated with their reversal. METHODS: We studied synopses of randomized controlled trials (RCTs) published from 2002 to 2005 as "Daily POEMs" (Patient Oriented Evidence that Matters). The initial evidence (E1) from these POEMs (2002-2005) was compared with the updated evidence (E2) on that same topic in a summary resource (DynaMed 2019). Two physician-raters independently categorized each POEM-RCT as (i) reversed when E1 ≠ E2, or as (ii) not reversed, when E1 = E2. For all "Evidence Reversals" (E1 ≠ E2), we assessed the direction of change in the evidence. RESULTS: We evaluated 408 POEMs on RCTs. Of those, 35 (9%; 95% confidence interval [6-12]) were identified as reversed, 359 (88%) were identified as not reversed, and 14 (3%) were indeterminate. On average, this represents about 2 evidence reversals per annum for POEMs about RCTs. CONCLUSIONS: Over 12-17 years, 9% of RCTs summarized as POEMs are reversed. Information alerting services that apply strict criteria for relevance and validity of clinical information are likely to identify RCTs whose findings are stable over time.

We studied the extent to which evidence from randomized controlled trials (RCTs) relevant to primary care is contradicted in subsequent research. When it was, we identified this event as an evidence reversal. In addition, we sought to identify characteristics of RCTs associated with their reversal. From 408 RCTs published during the period 2002­2005, study characteristics such as sample size were identified and extracted. Subsequently, we compared the evidence reported in each of these RCTs with the evidence on that same topic in an online summary resource in 2019. This allowed us to classify each RCT in one of the following 3 categories: evidence confirmed, reversed, or uncertain if this evidence is confirmed or reversed. Over 12­17 years of follow-up time, the findings of about 9 in 10 RCTs summarized as POEMs are stable. We found no statistically significant associations between trial characteristics and their subsequent reversal. This low rate of evidence reversal is good news for the RCTs that are used to inform decision-making.

Conflicts of Interest in Psychopharmacology Textbooks.

While most medical journals require disclosures of industry payments to authors and editors, there is no requirement for textbooks. In this study we evaluated nine well-known psychopharmacology textbooks to identify payments to their writers and editors. Two-thirds of the textbooks had at least one editor or author who received personal payments from one or more pharmaceutical companies, for a total of 11,021,409 USD paid to 11 of 21 editors/authors over a seven-year period. Much of this money was paid to a single author but 24% of the writers received over 75,000 USD each over this time period. There are several psychopharmacology textbooks authored by writers without apparent financial conflicts of interest. Just as with medical journals, medical textbooks should be transparent about payments made to their authors and editors.

The science of education: using learning theory to solve teaching problems in medical education.

Although theories do not play a major role in clinical medicine, the application of established theories of education can improve medical teaching. By understanding and harnessing learning theories, clinician educators can solve common teaching problems, thoughtfully craft their teaching sessions, determine whether learners or residents have achieved the intended learning objectives, and even earn higher course ratings. Key theories to be aware of include cognitive load theory, social learning theory, experiential learning, constructivism, humanism, behaviourism and connectivism. Various teaching problems are presented, along with explanations rooted in educational theory and possible solutions for use in the clinical educational setting.

Top 20 POEMs of the Past 20 Years: A Survey of Practice-Changing Research for Family Physicians.

POEMs (patient-oriented evidence that matters) are studies that address a relevant clinical question, demonstrate improved patient-oriented outcomes, and have the potential to change practice. For 20 years the authors of this article have reviewed more than 100 English language clinical journals monthly to identify POEMs in the medical literature relevant to primary care practice. This article identifies the POEMs in each of the last 20 years that were highest ranked for having recommended a major and persistent change in practice that year. They include POEMs that recommend a novel, effective intervention, a second group that recommends abandoning an ineffective practice, and a third group that recommends abandoning a potentially harmful practice. The top POEMs of the past 20 years illustrate the breadth of practice change in primary care and the need for family physicians to have a systematic approach to keeping up with the medical literature, such as that in POEMs, especially because many of these important articles did not appear in the primary care literature.

Cultivating the Inner Life of a Physician Through Written Reflection.

All of us have an "inner life" that forms the core of who we are. It shapes and is shaped by our actions and experiences. During physician training, attention to residents' inner life requires a focus on their beliefs and emotions as well as their ethical and spiritual development, topics often considered to be outside the realm of clinical training and practice. We suggest that written reflections, as part of medical residency curriculum, can allow residents to explore their inner lives. The depth and range of residents' explorations show the value of adding brief, protected time for residents to explore their hopes, joys, struggles, and feelings, and to develop meaning from their experiences with patients.

Developing a Clinician Friendly Tool to Identify Useful Clinical Practice Guidelines: G-TRUST.

BACKGROUND: Clinicians are faced with a plethora of guidelines. To rate guidelines, they can select from a number of evaluation tools, most of which are long and difficult to apply. The goal of this project was to develop a simple, easy-to-use checklist for clinicians to use to identify trustworthy, relevant, and useful practice guidelines, the Guideline Trustworthiness, Relevance, and Utility Scoring Tool (G-TRUST). METHODS: A modified Delphi process was used to obtain consensus of experts and guideline developers regarding a checklist of items and their relative impact on guideline quality. We conducted 4 rounds of sampling to refine wording, add and subtract items, and develop a scoring system. Multiple attribute utility analysis was used to develop a weighted utility score for each item to determine scoring. RESULTS: Twenty-two experts in evidence-based medicine, 17 developers of high-quality guidelines, and 1 consumer representative participated. In rounds 1 and 2, items were rewritten or dropped, and 2 items were added. In round 3, weighted scores were calculated from rankings and relative weights assigned by the expert panel. In the last round, more than 75% of experts indicated 3 of the 8 checklist items to be major indicators of guideline usefulness and, using the AGREE tool as a reference standard, a scoring system was developed to identify guidelines as useful, may not be useful, and not useful. CONCLUSION: The 8-item G-TRUST is potentially helpful as a tool for clinicians to identify useful guidelines. Further research will focus on its reliability when used by clinicians.