Effect of Interpolated Questions on Podcast Knowledge Acquisition and Retention: A Double-Blind, Multicenter, Randomized Controlled Trial.

STUDY OBJECTIVE: We determine whether interpolated questions in a podcast improve knowledge acquisition and retention. METHODS: This double-blinded controlled trial randomized trainees from 6 emergency medicine programs to listen to 1 of 2 versions of a podcast, produced de novo on the history of hypertension. The versions were identical except that 1 included 5 interpolated questions to highlight educational points (intervention). There were 2 postpodcast assessments, 1 within 48 hours of listening (immediate posttest) and a second 2 to 3 weeks later (retention test), consisting of the same 15 multiple-choice questions, 5 directly related to the interpolated questions and 10 unrelated. The primary outcome was the difference in intergroup mean assessment scores at each point. A sample size was calculated a priori to identify a significant interarm difference. RESULTS: One hundred thirty-seven emergency medicine trainees were randomized to listen to a podcast, 69 without interpolated questions (control) and 68 with them (intervention). Although no significant difference was detected at the immediate posttest, trainees randomized to the interpolated arm had a significantly higher mean score on the retention test, with a 5.6% absolute difference (95% confidence interval 0.2% to 10.9%). For the material covered by the interpolated questions, the intervention arm had significantly higher mean scores on both the immediate posttest and retention tests, with absolute differences of 8.7% (95% confidence interval 1.6% to 15.8%) and 10.1% (95% confidence interval 2.8% to 17.4%), respectively. There was no significant intergroup difference for noninterpolated questions. CONCLUSION: Podcasts containing interpolated questions improved knowledge retention at 2 to 3 weeks, driven by the retention of material covered by the interpolated questions. Although the gains may be modest, educational podcasters may improve knowledge retention by using interpolating questions.

Quality Evaluation Scores are no more Reliable than Gestalt in Evaluating the Quality of Emergency Medicine Blogs: A METRIQ Study.

Construct: We investigated the quality of emergency medicine (EM) blogs as educational resources. PURPOSE: Online medical education resources such as blogs are increasingly used by EM trainees and clinicians. However, quality evaluations of these resources using gestalt are unreliable. We investigated the reliability of two previously derived quality evaluation instruments for blogs. APPROACH: Sixty English-language EM websites that published clinically oriented blog posts between January 1 and February 24, 2016, were identified. A random number generator selected 10 websites, and the 2 most recent clinically oriented blog posts from each site were evaluated using gestalt, the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) score, and the Medical Education Translational Resources: Impact and Quality (METRIQ-8) score, by a sample of medical students, EM residents, and EM attendings. Each rater evaluated all 20 blog posts with gestalt and 15 of the 20 blog posts with the ALiEM AIR and METRIQ-8 scores. Pearson's correlations were calculated between the average scores for each metric. Single-measure intraclass correlation coefficients (ICCs) evaluated the reliability of each instrument. RESULTS: Our study included 121 medical students, 88 EM residents, and 100 EM attendings who completed ratings. The average gestalt rating of each blog post correlated strongly with the average scores for ALiEM AIR (r = .94) and METRIQ-8 (r = .91). Single-measure ICCs were fair for gestalt (0.37, IQR 0.25-0.56), ALiEM AIR (0.41, IQR 0.29-0.60) and METRIQ-8 (0.40, IQR 0.28-0.59). CONCLUSION: The average scores of each blog post correlated strongly with gestalt ratings. However, neither ALiEM AIR nor METRIQ-8 showed higher reliability than gestalt. Improved reliability may be possible through rater training and instrument refinement.

Examining Reliability and Validity of an Online Score (ALiEM AIR) for Rating Free Open Access Medical Education Resources.

STUDY OBJECTIVE: Since 2014, Academic Life in Emergency Medicine (ALiEM) has used the Approved Instructional Resources (AIR) score to critically appraise online content. The primary goals of this study are to determine the interrater reliability (IRR) of the ALiEM AIR rating score and determine its correlation with expert educator gestalt. We also determine the minimum number of educator-raters needed to achieve acceptable reliability. METHODS: Eight educators each rated 83 online educational posts with the ALiEM AIR scale. Items include accuracy, usage of evidence-based medicine, referencing, utility, and the Best Evidence in Emergency Medicine rating score. A generalizability study was conducted to determine IRR and rating variance contributions of facets such as rater, blogs, posts, and topic. A randomized selection of 40 blog posts previously rated through ALiEM AIR was then rated again by a blinded group of expert medical educators according to their gestalt. Their gestalt impression was subsequently correlated with the ALiEM AIR score. RESULTS: The IRR for the ALiEM AIR rating scale was 0.81 during the 6-month pilot period. Decision studies showed that at least 9 raters were required to achieve this reliability. Spearman correlations between mean AIR score and the mean expert gestalt ratings were 0.40 for recommendation for learners and 0.35 for their colleagues. CONCLUSION: The ALiEM AIR scale is a moderately to highly reliable, 5-question tool when used by medical educators for rating online resources. The score displays a fair correlation with expert educator gestalt in regard to the quality of the resources. The score displays a fair correlation with educator gestalt.

FOAM authorship: Who's teaching our learners?

Background: Free open-access medical education (FOAM) is extremely popular among learners and educators despite lacking the traditional peer review process. Despite the potential for inaccurate, low-quality, or biased content, little has been published describing FOAM authors. Methods: We performed a cross-sectional analysis of 12 months of content from the top 25 blogs in the 2020 Social Media Index from August 2020-2021. We recorded the number of posts per site and descriptive characteristics of authors, including gender affiliation, conflicts of interest (COI) statements, and type of practice (academic, community, or hybrid). Results: We identified 2141 posts by 1001 authors. More than half were produced by six websites: EM Docs (266), Life in the Fast Lane (232), EMCrit (188), ALiEM (185), Don't Forget the Bubbles (181), and Rebel EM (174). Most content (1680 posts, 78.5%) lacked a COI statement. Authors were mostly academic (89%), mostly held MD degrees (67.4%), and were mostly men (59.7%). Geographically, most FOAM authors reside in the United States (59.5%), Canada (22.42%), or the United Kingdom (9.4%). Conclusions: Of all the posts in the top 25 sites in 2020, more than half came from six sites, and authors were largely North American men in academics with MD degrees. Learners, content creators, and educators should consider the ways in which a more diverse authorship pool might bring value to the FOAM educational experience.

Systematic online academic resource (SOAR) review: Pediatric respiratory infectious disease.

Background: Free open access medical education (FOAM) resources have become increasingly popular in graduate medical education. Despite their accessibility, the assessment of FOAM resources' quality is challenging due to their decentralized nature and the diverse qualifications of their authors and distribution platforms. In this first pediatric systematic online academic resource (SOAR) review, we utilized a systematic methodology to aggregate and assess the quality of FOAM resources on pediatric respiratory infectious disease topics. Methods: We searched 177 keywords using FOAMSearch, the top 50 FOAM websites on the Social Media Index, and seven additional pediatric emergency medicine-focused blogs. Following a basic initial screen, resources then underwent full-text quality assessment utilizing the revised Medical Education Translational Resources: Impact and Quality (rMETRIQ) tool. Results: The search yielded 44,897 resources. After 44,456 were excluded, 441 underwent quality assessment. A total of 36/441 posts (8% of posts) reached the high-quality threshold score (rMETRIQ ≥ 16). The most frequent topics overall were pneumonia and bronchiolitis. A total of 67/441 posts (15% of posts) were found to have a rMETRIQ score of less than or equal to 7, which may indicate poor quality. Conclusions: We systematically identified, described, and performed quality assessment on FOAM resources pertaining to the topic of pediatric respiratory infectious disease. We found that there is a paucity of high-quality posts on this topic. Despite this, the curated list of high-quality resources can help guide trainees and educators toward relevant educational information and suggest unmet needs for future FOAM resources.

A broad-spectrum antiviral targeting entry of enveloped viruses.

We describe an antiviral small molecule, LJ001, effective against numerous enveloped viruses including Influenza A, filoviruses, poxviruses, arenaviruses, bunyaviruses, paramyxoviruses, flaviviruses, and HIV-1. In sharp contrast, the compound had no effect on the infection of nonenveloped viruses. In vitro and in vivo assays showed no overt toxicity. LJ001 specifically intercalated into viral membranes, irreversibly inactivated virions while leaving functionally intact envelope proteins, and inhibited viral entry at a step after virus binding but before virus-cell fusion. LJ001 pretreatment also prevented virus-induced mortality from Ebola and Rift Valley fever viruses. Structure-activity relationship analyses of LJ001, a rhodanine derivative, implicated both the polar and nonpolar ends of LJ001 in its antiviral activity. LJ001 specifically inhibited virus-cell but not cell-cell fusion, and further studies with lipid biosynthesis inhibitors indicated that LJ001 exploits the therapeutic window that exists between static viral membranes and biogenic cellular membranes with reparative capacity. In sum, our data reveal a class of broad-spectrum antivirals effective against enveloped viruses that target the viral lipid membrane and compromises its ability to mediate virus-cell fusion.

Emergency Medicine Procedures: A Review of Approved Instructional Resources from the World of Free Open Access Medical Education.

The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series was created in 2014 to address the Free Open Access Medical Education (FOAM) movement's decentralized nature and lack of inherent peer review. The AIR series provides a topic-based, curated list of online educational content vetted by academic emergency medicine (EM) faculty that meets individualized interactive instruction criteria for EM trainees. Relevant FOAM resources were identified from the top 50 FOAM websites using the Social Media Index and then scored by EM faculty using a validated instrument to identify the highest quality posts related to a topic. This article reviews FOAM resources pertaining to EM procedures that were labeled as an "Approved Instructional Resource" or "Honorable Mention" using the AIR series methodology.