Assessment of management reasoning: Design considerations drawn from analysis of simulated outpatient encounters.

PURPOSE: Management reasoning is a distinct subset of clinical reasoning. We sought to explore features to be considered when designing assessments of management reasoning. METHODS: This is a hybrid empirical research study, narrative review, and expert perspective. In 2021, we reviewed and discussed 10 videos of simulated (staged) physician-patient encounters, actively seeking actions that offered insights into assessment of management reasoning. We analyzed our own observations in conjunction with literature on clinical reasoning assessment, using a constant comparative qualitative approach. RESULTS: Distinguishing features of management reasoning that will influence its assessment include management scripts, shared decision-making, process knowledge, illness-specific knowledge, and tailoring of the encounter and management plan. Performance domains that merit special consideration include communication, integration of patient preferences, adherence to the management script, and prognostication. Additional facets of encounter variation include the clinical problem, clinical and nonclinical patient characteristics (including preferences, values, and resources), team/system characteristics, and encounter features. We cataloged several relevant assessment approaches including written/computer-based, simulation-based, and workplace-based modalities, and a variety of novel response formats. CONCLUSIONS: Assessment of management reasoning could be improved with attention to the performance domains, facets of variation, and variety of approaches herein identified.

Validation of the Learner Engagement Instrument for Continuing Professional Development.

PURPOSE: Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables. METHOD: Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or online) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. All statistical methods accounted for repeated measures by participants. RESULTS: 415 unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with Comparative Fit Index ≥0.99, Standardized Root Mean Square Residual ≤0.031, and Root Mean Square Error of Approximation ≤0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive-in-class domain (50-rater G-coefficient 0.55 to 0.66). All findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho 0.58), and a small correlation was found with participant age (rho 0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form. CONCLUSIONS: This study confirms a 4-domain model of learner engagement, and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.

Management reasoning and patient-clinician interactions: Insights from shared decision-making and simulated outpatient encounters.

PURPOSE: To expand understanding of patient-clinician interactions in management reasoning. METHODS: We reviewed 10 videos of simulated patient-clinician encounters to identify instances of problematic and successful communication, then reviewed the videos again through the lens of two models of shared decision-making (SDM): an 'involvement-focused' model and a 'problem-focused' model. Using constant comparative qualitative analysis we explored the connections between these patient-clinician interactions and management reasoning. RESULTS: Problems in patient-clinician interactions included failures to: encourage patient autonomy; invite the patient's involvement in decision-making; convey the health impact of the problem; explore and address concerns and questions; explore the context of decision-making (including patient preferences); meet the patient where they are; integrate situational preferences and priorities; offer >1 viable option; work with the patient to solve a problem of mutual concern; explicitly agree to a final care plan; and build the patient-clinician relationship. Clinicians' 'management scripts' varied along a continuum of prioritizing clinician vs patient needs. Patients also have their own cognitive scripts that guide their interactions with clinicians. The involvement-focused and problem-focused SDM models illuminated distinct, complementary issues. CONCLUSIONS: Management reasoning is a deliberative interaction occurring in the space between individuals. Juxtaposing management reasoning alongside SDM generated numerous insights.

Learner engagement and teaching effectiveness in livestreamed versus in-person CME.

INTRODUCTION: Engaging learners in continuing medical education (CME) is challenging. Recently, CME courses have transitioned to livestreamed CME, with learners viewing live, in-person courses online. The authors aimed to (1) compare learner engagement and teaching effectiveness in livestreamed with in-person CME and (2) determine how livestream engagement and teaching effectiveness is associated with (A) interactivity metrics, (B) presentation characteristics and (C) medical knowledge. METHODS: A 3-year, non-randomised study of in-person and livestream CME was performed. The course was in-person for 2018 but transitioned to livestream for 2020 and 2021. Learners completed the Learner Engagement Inventory and Teaching Effectiveness Instrument after each presentation. Both instruments were supported by content, internal structure and relations to other variables' validity evidence. Interactivity metrics included learner use of audience response, questions asked by learners and presentation views. Presentation characteristics included presentations using audience response, using pre/post-test format, time of day and words per slide. Medical knowledge was assessed by audience response. A repeated measures analysis of variance (anova) was used for comparisons and a mixed model approach for correlations. RESULTS: A total of 159 learners (response rate 27%) completed questionnaires. Engagement did not significantly differ between in-person or livestream CME. (4.56 versus 4.53, p = 0.64, maximum 5 = highly engaged). However, teacher effectiveness scores were higher for in-person compared with livestream (4.77 versus 4.71 p = 0.01, maximum 5 = highly effective). For livestreamed courses, learner engagement was associated with presentation characteristics, including presentation using of audience response (yes = 4.57, no = 4.45, p < .0001), use of a pre/post-test (yes = 4.62, no = 4.54, p < .0001) and time of presentation (morning = 4.58, afternoon = 4.53, p = .0002). Significant associations were not seen for interactivity metrics or medical knowledge. DISCUSSION: Livestreaming may be as engaging as in-person CME. Although teaching effectiveness in livestreaming was lower, this difference was small. CME course planners should consider offering livestream CME while exploring strategies to enhance teaching effectiveness in livestreamed settings.

Management Reasoning: Empirical Determination of Key Features and a Conceptual Model.

PURPOSE: Management reasoning is a critical yet understudied phenomenon in clinical practice and medical education. The authors sought to empirically identify key features of management reasoning and construct a model describing the management reasoning process. METHOD: In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters and used a coding form to document key features and insights related to management reasoning. The team used a constant comparative approach to distill 120 pages of raw observations into an 18-page list of management tasks, processes, and insights. The team then had a series of discussions to iteratively refine these findings into a parsimonious model of management reasoning. RESULTS: The investigators empirically identified 12 distinct features of management reasoning: contrasting and selection among multiple solutions; prioritization of patient, clinician, and system preferences and constraints; communication and shared decision making; ongoing monitoring and adjustment of the management plan; dynamic interplay among people, systems, and competing priorities; illness-specific knowledge; process knowledge; management scripts; clinician roles as patient teacher and salesperson; clinician-patient relationship; prognostication; and organization of the clinical encounter (sequencing and time management). Management scripts seemed to play a prominent and critical role. The model of management reasoning comprised 4 steps: instantiation of a management script, identifying (multiple) options and beginning to teach the patient, shared decision making, and ongoing monitoring and adjustment. This model also conceives 2 overarching features: that management reasoning is personalized to the patient and that it occurs between individuals rather than exclusively within the clinician's mind. CONCLUSIONS: Management scripts constitute a key feature of management reasoning, along with teaching patients about viable options, shared decision making, ongoing monitoring and adjustment, and personalization. Management reasoning seems to be constructed and negotiated between individuals rather than exclusively within the clinician.

Management reasoning scripts: Qualitative exploration using simulated physician-patient encounters.

INTRODUCTION: Management reasoning is distinct from diagnostic reasoning and remains incompletely understood. The authors sought to empirically investigate the concept of management scripts. METHODS: In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters, and used a coding form to document observations about management reasoning. The team used constant comparative analysis to integrate empirically-grounded insights with theories related to cognitive scripts and Type 1/Type 2 thinking. RESULTS: Management scripts are precompiled conceptual knowledge structures that represent and connect management options and clinician tasks in a temporal or logical sequence. Management scripts appear to differ substantially from illness scripts. Management scripts varied in quality (in content, sequence, flexibility, and fluency) and generality. The authors empirically identified six key features (components) of management scripts: the problem (diagnosis); management options; preferences, values, and constraints; education needs; interactions; and encounter flow. The authors propose a heuristic framework describing script activation, selection, instantiation with case-specific details, and application to guide development of the management plan. They further propose that management reasoning reflects iterative, back-and-forth involvement of both Type 1 (non-analytic, effortless) and Type 2 (analytic, effortful) thinking. Type 1 thinking likely influences initial script activation, selection, and initial instantiation. Type 2 increasingly influences subsequent script revisions, as activation, selection, and instantiation become more deliberate (effortful) and more hypothetical (involving mental simulation). DISCUSSION: Management scripts constitute a key feature of management reasoning, and could represent a new target for training in clinical reasoning (distinct from illness scripts).

Associations Between Physician Continuous Professional Development and Referral Patterns: A Systematic Review and Meta-Analysis.

PURPOSE: Both overuse and underuse of clinician referrals can compromise high-value health care. The authors sought to systematically identify and synthesize published research examining associations between physician continuous professional development (CPD) and referral patterns. METHOD: The authors searched MEDLINE, Embase, PsycInfo, and the Cochrane Database on April 23, 2020, for comparative studies evaluating CPD for practicing physicians and reporting physician referral outcomes. Two reviewers, working independently, screened all articles for inclusion. Two reviewers reviewed all included articles to extract information, including data on participants, educational interventions, study design, and outcomes (referral rate, intended direction of change, appropriateness of referral). Quantitative results were pooled using meta-analysis. RESULTS: Of 3,338 articles screened, 31 were included. These studies enrolled at least 14,458 physicians and reported 381,165 referral events. Among studies comparing CPD with no intervention, 17 studies with intent to increase referrals had a pooled risk ratio of 1.91 (95% confidence interval: 1.50, 2.44; P < .001), and 7 studies with intent to decrease referrals had a pooled risk ratio of 0.68 (95% confidence interval: 0.55, 0.83; P < .001). Five studies did not indicate the intended direction of change. Subgroup analyses revealed similarly favorable effects for specific instructional approaches (including lectures, small groups, Internet-based instruction, and audit/feedback) and for activities of varying duration. Four studies reported head-to-head comparisons of alternate CPD approaches, revealing no clear superiority for any approach. Seven studies adjudicated the appropriateness of referral, and 9 studies counted referrals that were actually completed (versus merely requested). CONCLUSIONS: Although between-study differences are large, CPD is associated with statistically significant changes in patient referral rates in the intended direction of impact. There are few head-to-head comparisons of alternate CPD interventions using referrals as outcomes.

Cost-effectiveness and Economic Benefit of Continuous Professional Development for Drug Prescribing: A Systematic Review.

Importance: The economic impact of continuous professional development (CPD) education is incompletely understood. Objective: To systematically identify and synthesize published research examining the costs associated with physician CPD for drug prescribing. Evidence Review: MEDLINE, Embase, PsycInfo, and the Cochrane Database were searched from inception to April 23, 2020, for comparative studies that evaluated the cost of CPD focused on drug prescribing. Two reviewers independently screened all articles for inclusion and reviewed all included articles to extract data on participants, educational interventions, study designs, and outcomes (costs and effectiveness). Results were synthesized for educational costs, health care costs, and cost-effectiveness. Findings: Of 3338 articles screened, 38 were included in this analysis. These studies included at least 15 659 health care professionals and 1 963 197 patients. Twelve studies reported on educational costs, ranging from $281 to $183 554 (median, $15 664). When economic outcomes were evaluated, 31 of 33 studies (94%) comparing CPD with no intervention found that CPD was associated with reduced health care costs (drug costs), ranging from $4731 to $6 912 000 (median, $79 373). Four studies found reduced drug costs for 1-on-1 outreach compared with other CPD approaches. Regarding cost-effectiveness, among 5 studies that compared CPD with no intervention, the incremental cost-effectiveness ratio for a 10% improvement in prescribing ranged from $15 390 to $437 027 to train all program participants. Four comparisons of alternative CPD approaches found that 1-on-1 educational outreach was more effective but more expensive than group education or mailed materials (incremental cost-effectiveness ratio, $18-$4105 per physician trained). Conclusions and Relevance: In this systematic review, CPD for drug prescribing was associated with reduced health care (drug) costs. The educational costs and cost-effectiveness of CPD varied widely. Several CPD instructional approaches (including educational outreach) were more effective but more costly than comparators.

Costs and Economic Impacts of Physician Continuous Professional Development: A Systematic Scoping Review.

PURPOSE: Nearly all health care professionals engage in continuous professional development (CPD), yet little is known about the cost and cost-effectiveness of physician CPD. Clarification of key concepts, comprehensive identification of published work, and determination of research gaps would facilitate application of existing evidence and planning for future investigations. The authors sought to systematically map study themes, methods, and outcomes in peer-reviewed literature on the cost and value of physician CPD. METHOD: The authors conducted a scoping review, systematically searching MEDLINE, Embase, PsycInfo, and Cochrane Library databases for comparative economic evaluations of CPD for practicing physicians through April 2020. Two reviewers, working independently, screened all articles for inclusion. Three reviewers iteratively reviewed all included articles to inductively identify key features including participants, educational interventions, study designs, cost ingredients, and cost analyses. Two reviewers then independently reexamined all included articles to code these features. RESULTS: Of 3,338 potentially eligible studies, 111 were included. Physician specialties included internal, family, or general medicine (80 studies [72%]), surgery (14 studies [13%]), and medicine subspecialties (7 studies [6%]). Topics most often addressed general medicine (45 studies [41%]) or appropriate drug use (37 studies [33%]). Eighty-seven studies (78%) compared CPD with no intervention. Sixty-three studies (57%) reported the cost of training, and 79 (71%) evaluated the economic impact (money saved/lost following CPD). Training cost ingredients (median 3 itemized per study) and economic impact ingredients (median 1 per study) were infrequently and incompletely identified, quantified, or priced. Twenty-seven studies (24%) reported cost-impact expressions such as cost-effectiveness ratio or net value. Nineteen studies (17%) reported sensitivity analyses. CONCLUSIONS: Studies evaluating the costs and economic impact of physician CPD are few. Gaps exist in identification, quantification, pricing, and analysis of cost outcomes. The authors propose a comprehensive framework for appraising ingredients and a preliminary reference case for economic evaluations.

Perioperative Venous Thromboembolism Prophylaxis.

Venous thromboembolism (VTE) is a preventable cause of postoperative morbidity and mortality; however, audits suggest that the use of thromboprophylaxis is underused. In this review, we describe our approach to prevention of postoperative VTE and provide guidance on how to formulate an optimal VTE prophylaxis plan. We recommend that all patients undergo thrombosis- and bleeding-risk assessment as part of their preoperative evaluation. The risk of thrombosis can be estimated based on patient- and procedure-specific factors, using validated risk-assessment models such as the Caprini score. There are no validated models to predict perioperative bleeding; however, several risk factors have been proposed. Patients should ambulate early and frequently after surgery. We recommend no additional prophylaxis in patients at very low risk of VTE (Caprini score 0). Patients at low risk of VTE (Caprini 1 to 2) are recommended to receive either mechanical or pharmacological prophylaxis. Patients at moderate (Caprini 3 to 4) to high risk of VTE (Caprini ≥5) are recommended pharmacological prophylaxis either alone or combined with mechanical prophylaxis. Patients at high risk of bleeding should receive mechanical prophylaxis until their risk of bleeding is reduced and pharmacological prophylaxis can be reconsidered. Populations for which the Caprini score has not been validated (such as orthopedic surgery) are recommended prophylaxis based on individual and procedure-specific risk factors. Prophylaxis is typically continued until the patient is ambulatory or until hospital dismissal; however, longer durations can be considered in certain circumstances (high-risk patients undergoing malignant abdominopelvic operations, bariatric operations, and certain orthopedic operations).

The relationship between learner engagement and teaching effectiveness: a novel assessment of student engagement in continuing medical education.

BACKGROUND: Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants' engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME. METHODS: We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE. RESULTS: 51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach's alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain. CONCLUSION: Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.

Preoperative Cardiac Risk Assessment.

Major adverse cardiac events are common causes of perioperative mortality and major morbidity. Preventing these complications requires thorough preoperative risk assessment and postoperative monitoring of at-risk patients. Major guidelines recommend assessment based on a validated risk calculator that incorporates patient- and procedure-specific factors. American and European guidelines define when stress testing is needed on the basis of functional capacity assessment. Favoring cost-effectiveness, Canadian guidelines instead recommend obtaining brain natriuretic peptide or N-terminal prohormone of brain natriuretic peptide levels to guide postoperative screening for myocardial injury or infarction. When conditions such as acute coronary syndrome, severe pulmonary hypertension, and decompensated heart failure are identified, nonemergent surgery should be postponed until the condition is appropriately managed. There is an evolving role of biomarkers and myocardial injury after noncardiac surgery to enhance risk stratification, but the effect of interventions guided by these strategies is unclear.

Characteristics and quality of rotation-specific resident learning goals: a prospective study.

Background: Residents are expected to develop the skills to set learning goals. Setting learning goals is part of self-regulated learning, setting the foundation for creating a learning plan, deploying learning strategies, and assessing their progress to those goals. While effective goal setting is essential to resident self-regulated learning, residents struggle with setting learning goals and desire faculty assistance with goal setting.Objective: We aimed to characterize the topics and quality of residents' rotation-specific learning goals.Design: We conducted a prospective study of 153 internal medicine residents, assessing 455 learning goals for general medicine inpatient rotations. We coded learning goal themes, competencies, and learning domains, and assessed quality using the validated Learning Goal Scoring Rubric. We compared topic categories, competencies, learning domains, and quality between the first and second months of postgraduate (PGY)-1 residents and between PGY-1 and PGY-3 residents. We assessed factors associated with learning goal completion.Results: The overall response rate was 80%. The top three learning goal categories were patient management, specific diseases related to general medicine, and teaching skills. There were no changes in learning goal characteristics between PGY-1 months (p ≥ 0.04). There were differences between PGY-1 and PGY-3 residents' learning goals in patient management (28% vs 6%; p < .001), specific disease conditions (19% vs 3%; p < .001), and teaching skills (2% vs 56%; p < .001). There was no difference in learning goal quality between PGY-1 months (1.63 vs. 1.67; p = 0.82). The PGY-3 learning goals were of higher quality than PGY-1 learning goals for the 'specific goal' item (1.38 vs. 0.98, p = 0.005), but not for other items or overall (all p ≥ 0.02). Residents reported 85% (297/347) learning goal completion.Conclusions: Resident rotation-specific learning goals reflect a broad array of topics. Residents' learning goal quality was low and residents may benefit from guidance to support residents' learning goals.

Chinese physician perceptions regarding industry support of continuing medical education programs: a cross-sectional survey.

Background: Industry funding in continuing medical education has been extensively studied in the USA. Although continuing medical education is also a requirement for Chinese physicians, little is known about Chinese physician perceptions of industry support in continuing medical education.Objective: We aim to determine perceptions regarding industry support for CME among Chinese physicians at a large CME course, examine potential associations between Chinese physicians' perceptions and their demographic characteristics, and compare Chinese and US physicians' perceptions of industry support for CME.Design: We performed a cross-sectional survey of physicians at a nephrology continuing medical education conference in China. All participants received a previously published, anonymous survey consisting of 4 items, with questions asked in English and Mandarin Chinese. Responses were compared with those of a previous cohort in the USA.Results: The response rate was 24% (128/541). Most respondents were nephrologists (112/126, 89%), women (91/128, 71%), and aged 20 to 40 years (79/127, 62%). Most respondents preferred industry-supported continuing medical education (84/123, 68%) or had no preference (33/123, 27%). More clinicians than clinical researchers supported industry offsetting costs (76.9% vs 58.3%; P = .03). Almost half of participants (58/125, 46%) stated that industry-supported continuing medical education was biased in support of industry. Compared with US physicians, Chinese physicians were more likely to believe, or had no opinion, that industry-supported courses were biased (67.2% vs 47.0%; P < .001).Conclusions: Chinese continuing medical education participants preferred industry-sponsored continuing medical education and were strongly in favor of industry offsetting costs, but almost half believed that such education was biased in favor of supporting companies. Concern for bias was higher among Chinese than US physicians. Given participants' concerns, further study examining industry bias in Chinese continuing medical education is recommended.Abbreviations: CME: Continuing medical education; US: USA.

Primary care physicians' perceptions of practice improvement as a professional responsibility: a cross-sectional study.

Continuous quality improvement is a component of professionalism. Maintenance of Certification (MOC) is a mechanism in the USA for physicians to keep current with medical knowledge and contribute to practice improvement. Little is known about primary care physicians' perceptions of the practice improvement (Part IV) components of MOC. We aimed to determine primary care physicians' perceptions of their professional responsibility to participate in Part IV MOC. This was a cross-sectional study of primary care physicians using the American Medical Association Masterfile. We developed a nine-item survey, designed from expert consensus and literature to determine views on Part IV MOC as a professional responsibility. We surveyed 1500 randomly selected primary care physicians via mail from November 2014 to May 2015. The response rate was 42% (627 of 1,500): 47% (273 of 585) were family practitioners and 49% (289 of 585) were internists. Factor analysis revealed a two-factor survey, with five items pertaining to positive views of MOC Part IV and four items pertaining to negative views. Internists were more likely to view MOC Part IV as time consuming (82.0% vs. 70.3%, P = .001), expensive (50.9% vs. 38.8%, P = .004), and not relevant to practice (39.1% vs. 23.8%, P < .001). Family medicine practitioners were more likely to view MOC Part IV as improving patient care (64.5% vs. 48.8%, P < .001) and maintaining professional responsibility (48.7% vs. 32.5%, P < .001). Regardless of specialty, most physicians viewed MOC Part IV as time intensive, not beneficial for career advancement, and not a professional responsibility. Family medicine practitioners demonstrated more positive views of MOC Part IV. The difference between family medicine practitioners and internists could be related to the ABIM MOC controversy. Future changes to practice improvement requirements could focus on limiting time requirements and on clinical relevance. ABBREVIATIONS: ABIM: American Board of Internal Medicine; AMA: American Medical Association; CQI: continuous quality improvement; IRB: institutional review board; MOC: Maintenance of Certification; QI: quality improvement.

Conference presentation to publication: a retrospective study evaluating quality of abstracts and journal articles in medical education research.

BACKGROUND: There is little evidence regarding the comparative quality of abstracts and articles in medical education research. The Medical Education Research Study Quality Instrument (MERSQI), which was developed to evaluate the quality of reporting in medical education, has strong validity evidence for content, internal structure, and relationships to other variables. We used the MERSQI to compare the quality of reporting for conference abstracts, journal abstracts, and published articles. METHODS: This is a retrospective study of all 46 medical education research abstracts submitted to the Society of General Internal Medicine 2009 Annual Meeting that were subsequently published in a peer-reviewed journal. We compared MERSQI scores of the abstracts with scores for their corresponding published journal abstracts and articles. Comparisons were performed using the signed rank test. RESULTS: Overall MERSQI scores increased significantly for published articles compared with conference abstracts (11.33 vs 9.67; P < .001) and journal abstracts (11.33 vs 9.96; P < .001). Regarding MERSQI subscales, published articles had higher MERSQI scores than conference abstracts in the domains of sampling (1.59 vs 1.34; P = .006), data analysis (3.00 vs 2.43; P < .001), and validity of evaluation instrument (1.04 vs 0.28; P < .001). Published articles also had higher MERSQI scores than journal abstracts in the domains of data analysis (3.00 vs 2.70; P = .004) and validity of evaluation instrument (1.04 vs 0.26; P < .001). CONCLUSIONS: To our knowledge, this is the first study to compare the quality of medical education abstracts and journal articles using the MERSQI. Overall, the quality of articles was greater than that of abstracts. However, there were no significant differences between abstracts and articles for the domains of study design and outcomes, which indicates that these MERSQI elements may be applicable to abstracts. Findings also suggest that abstract quality is generally preserved from original presentation to publication.

Flipping the Continuing Medical Education Classroom: Validating a Measure of Attendees' Perceptions.

INTRODUCTION: New teaching approaches for CME are needed. In flipped classrooms, coursework is completed beforehand and applied during class time. Studies of flipped classrooms and their potential benefits in CME have not been published. We sought to develop and validate an instrument measuring flipped classroom perceptions, identify whether participation changed perceptions, and determine which flipped classroom components were perceived as most effective. METHODS: In this cross-sectional validation study, 167 participants in the Mayo Clinic's 2015 Internal Medicine Board Review course received surveys. Online modules were developed to deliver content before flipped classroom courses on acid-base disorders and electrolyte disorders. A flipped classroom perception instrument (FCPI) was developed and validated. The FCPI, with eight items structured on 5-point Likert scales, was given to participants before and after their flipped classroom experiences. RESULTS: Of the 167 participants, 111 returned surveys. Flipped classroom perceptions improved, with mean (SD) FCPI scores increasing from 3.74 (0.75) to 3.94 (0.76) (P < .001). The percentage of participants who preferred flipped classrooms increased from 38% before the course to 53% after (P = .002). Positive changes in FCPI scores were unrelated to module completion. Most participants thought knowledge was enhanced by in-class sessions and online modules equally. DISCUSSION: The FCPI, the first validated measure of participants' perceptions of a CME flipped classroom, has strong validity evidence. Participants' perceptions of and preference for the flipped classroom improved after experiencing the flipped CME classroom. These findings support the need to further explore flipped classroom models in CME.