Quality Indicators for Major Depressive Disorder in Adults: A Review of Performance Measures by the American College of Physicians.

Major depressive disorder (MDD) is a severe mood disorder that affects at least 8.4% of the adult population in the United States. Characteristics of MDD include persistent sadness, diminished interest in daily activities, and a state of hopelessness. The illness may progress quickly and have devastating consequences if left untreated. Eight performance measures are available to evaluate screening, diagnosis, and successful management of MDD. However, many performance measures do not meet the criteria for validity, reliability, evidence, and meaningfulness.The American College of Physicians (ACP) embraces performance measurement as a means to externally validate the quality of care of practices, medical groups, and health plans and to drive reimbursement processes. However, a plethora of performance measures that provide low or no value to patient care have inundated physicians, practices, and systems and burdened them with collecting and reporting of data. The ACP's Performance Measurement Committee (PMC) reviews performance measures using a validated process to inform regulatory and accreditation bodies in an effort to recognize high-quality performance measures, address gaps and areas for improvement in performance measures, and help reduce reporting burden. Out of 8 performance measures, the PMC found only 1 measure (suicide risk assessment) that was valid at all levels of attribution. This paper presents a review of MDD performance measures and highlights opportunities to improve performance measures addressing MDD management.

Quality Indicators for Osteoporosis in Adults: A Review of Performance Measures by the American College of Physicians.

Primary osteoporosis is characterized by decreasing bone mass and density and reduced bone strength that leads to a higher risk for fracture, especially hip and spine fractures. The prevalence of osteoporosis in the United States is estimated at 12.6% for adults older than 50 years. Although it is most frequently diagnosed in White and Asian females, it still affects males and females of all ethnicities. Osteoporosis is considered a major health issue, which has prompted the development and use of several performance measures to assess and improve the effectiveness of screening, diagnosis, and treatment. These performance measures are often used in accountability, public reporting, and/or payment programs. However, the reliability, validity, evidence, attribution, and meaningfulness of performance measures have been questioned. The purpose of this paper is to present a review of current performance measures on osteoporosis and inform physicians, payers, and policymakers in their selection of performance measures for this condition. The Performance Measurement Committee identified 6 osteoporosis performance measures relevant to internal medicine physicians, only 1 of which was found valid at all levels of attribution. This paper also proposes a performance measure concept to address a performance gap for the initial approach to therapy for patients with a new diagnosis of osteoporosis based on the current American College of Physicians guideline.

Effects of the Million Hearts Model on Myocardial Infarctions, Strokes, and Medicare Spending: A Randomized Clinical Trial.

Importance: The Million Hearts Model paid health care organizations to assess and reduce cardiovascular disease (CVD) risk. Model effects on long-term outcomes are unknown. Objective: To estimate model effects on first-time myocardial infarctions (MIs) and strokes and Medicare spending over a period up to 5 years. Design, Setting, and Participants: This pragmatic cluster-randomized trial ran from 2017 to 2021, with organizations assigned to a model intervention group or standard care control group. Randomized organizations included 516 US-based primary care and specialty practices, health centers, and hospital-based outpatient clinics participating voluntarily. Of these organizations, 342 entered patients into the study population, which included Medicare fee-for-service beneficiaries aged 40 to 79 years with no previous MI or stroke and with high or medium CVD risk (a 10-year predicted probability of MI or stroke [ie, CVD risk score] ≥15%) in 2017-2018. Intervention: Organizations agreed to perform guideline-concordant care, including routine CVD risk assessment and cardiovascular care management for high-risk patients. The Centers for Medicare & Medicaid Services paid organizations to calculate CVD risk scores for Medicare fee-for-service beneficiaries. CMS further rewarded organizations for reducing risk among high-risk beneficiaries (CVD risk score ≥30%). Main Outcomes and Measures: Outcomes included first-time CVD events (MIs, strokes, and transient ischemic attacks) identified in Medicare claims, combined first-time CVD events from claims and CVD deaths (coronary heart disease or cerebrovascular disease deaths) identified using the National Death Index, and Medicare Parts A and B spending for CVD events and overall. Outcomes were measured through 2021. Results: High- and medium-risk model intervention beneficiaries (n = 130 578) and standard care control beneficiaries (n = 88 286) were similar in age (median age, 72-73 y), sex (58%-59% men), race (7%-8% Black), and baseline CVD risk score (median, 24%). The probability of a first-time CVD event within 5 years was 0.3 percentage points lower for intervention beneficiaries than control beneficiaries (3.3% relative effect; adjusted hazard ratio [HR], 0.97 [90% CI, 0.93-1.00]; P = .09). The 5-year probability of combined first-time CVD events and CVD deaths was 0.4 percentage points lower in the intervention group (4.2% relative effect; HR, 0.96 [90% CI, 0.93-0.99]; P = .02). Medicare spending for CVD events was similar between the groups (effect estimate, -$1.83 per beneficiary per month [90% CI, -$3.97 to -$0.30]; P = .16), as was overall Medicare spending including model payments (effect estimate, $2.11 per beneficiary per month [90% CI, -$16.66 to $20.89]; P = .85). Conclusions and Relevance: The Million Hearts Model, which encouraged and paid for CVD risk assessment and reduction, reduced first-time MIs and strokes. Results support guidelines to use risk scores for CVD primary prevention. Trial Registration: ClinicalTrials.gov Identifier: NCT04047147.

Reducing Acute Hospitalizations at High-Performing CPC+ Primary Care Practice Sites: Strategies, Activities, and Facilitators.

PURPOSE: Despite evidence suggesting that high-quality primary care can prevent unnecessary hospitalizations, many primary care practices face challenges in achieving this goal, and there is little guidance identifying effective strategies for reducing hospitalization rates. We aimed to understand how practices in the Comprehensive Primary Care Plus (CPC+) program substantially reduced their acute hospitalization rate (AHR) over 2 years. METHODS: We used Bayesian analyses to identify the CPC+ practice sites having the highest probability of achieving a substantial reduction in the adjusted Medicare AHR between 2016 and 2018 (referred to here as AHR high performers). We then conducted telephone interviews with 64 respondents at 14 AHR high-performer sites and undertook within- and cross-case comparative analysis. RESULTS: The 14 AHR high performers experienced a 6% average decrease (range, 4% to 11%) in their Medicare AHR over the 2-year period. They credited various care delivery activities aligned with 3 strategies for reducing AHR: (1) improving and promoting prompt access to primary care, (2) identifying patients at high risk for hospitalization and addressing their needs with enhanced care management, and (3) expanding the breadth and depth of services offered at the practice site. They also identified facilitators of these strategies: enhanced payments through CPC+, prior primary care practice transformation experience, use of data to identify high-value activities for patient subgroups, teamwork, and organizational support for innovation. CONCLUSIONS: The AHR high performers observed that strengthening the local primary care infrastructure through practice-driven, targeted changes in access, care management, and comprehensiveness of care can meaningfully reduce acute hospitalizations. Other primary care practices taking on the challenging work of reducing hospitalizations can learn from CPC+ practices and may consider similar strategies, selecting activities that fit their context, personnel, patient population, and available resources.

Performance Measures for Physicians Providing Clinical Care Using Telemedicine: A Position Paper From the American College of Physicians.

There has been an exponential growth in the use of telemedicine services to provide clinical care, accelerated by the COVID-19 pandemic. Clinical care delivered via telemedicine has become a major and accepted method of health care delivery for many patients. There is an urgent need to understand quality of care in the telemedicine environment. This American College of Physicians position paper presents 6 recommendations to ensure the appropriate use of performance measures to evaluate quality of clinical care provided in the telemedicine environment.

Optimizing clinical guidelines to address antimicrobial-resistant infections: A conceptual framework reflecting stakeholder perspectives.

Objectives: Clinical guidelines or guidance is an important tool for preventing and treating antimicrobial-resistant (AMR) infections. We sought to understand and support the effective use of guidelines and guidance for AMR infections. Methods: Key informant interviews and a stakeholder meeting on the development and use of guidelines and guidance for management of AMR infections; the interview findings and meeting discussion informed a conceptual framework for AMR infection clinical guidelines. Participants: Interview participants included experts with experience in guidelines development and physician and pharmacist hospital leaders and antibiotic stewardship program leaders. Stakeholder meeting participants included federal and nonfederal participants involved in research, policy, and practice related to prevention and management of AMR infections. Results: Participants described challenges related to timeliness of guidelines, methodologic limitations of the development process, and issues with usability across a range of clinical settings. These findings, and participants' suggestions for mitigating the challenges identified, informed a conceptual framework for AMR infection clinical guidelines. The framework components include (1) science and evidence, (2) guideline and guidance development and dissemination, and (3) implementation and real-world practice. These components are supported by engaged stakeholders whose leadership and resources help to improve patient and population AMR infection prevention and management. Conclusions: Use of guidelines and guidance documents for management of AMR infections can be supported through (1) a robust body of scientific evidence to inform guidelines and guidance; (2) approaches and tools to support timely, transparent guidelines that are relevant and actionable for all clinical audiences; and (3) tools to implement guidelines and guidance effectively.

A Cautionary Tale: Exploring Stakeholder Emotions During Curricular Change Using Group Concept Mapping.

PhenomenonCurricular change is essential but challenging. Change efforts often struggle and some fail due to well-articulated challenges as well as other barriers less understood. The curricular change literature characterizes the organizational and operational features of successful change yet virtually ignores stakeholder emotions. A deeper understanding of these emotional responses during the change process could enhance participant and organizational well-being and support change success. Approach In 2019, data were collected at one large North American medical school using group concept mapping methodology, an asynchronous mixed methods approach. We sought to generate themes characterizing the emotional responses of faculty, administrative staff, and students across multiple stakeholder groups participating in the new and traditional (legacy) curricula 27 months after curricular change initiation. Participants brainstormed, sorted, and rated statements on emotional responses. Rating participants rated each statement using a Likert scale from 1 (does not resonate) to 4 (very strongly resonates) according to this prompt: "This emotional response resonates with my emotional experience during the curricular change." Multidimensional scaling and hierarchical cluster analyses were used to generate emotional profiles and compare them across stakeholders. Findings Of 335 stakeholders invited, 123 contributed to brainstorming (36.7%), 153 completed rating (45.7%) and 33 completed sorting (9.9%). Participants generated six themes of emotional responses to curricular change: enthusiastic, apprehensive, overwhelmed, missed opportunities, uncertain, and abandoned. The enthusiastic theme overall had the highest mean statement ratings while the abandoned theme had the lowest. Demographic subgroup analysis revealed new curriculum students (Class of 2021) were most enthusiastic while legacy curriculum students (Class of 2020) were more likely to feel abandoned. Overall, faculty and administrative staff were more enthusiastic than students whereas students rated the five other themes higher than faculty and administrative staff. InsightsCurricular change is emotionally taxing. Students in both curricula experienced greater uncertainty, apprehension, sense of missed opportunities, and feeling overwhelmed than did faculty and administrative staff. Legacy curriculum students rated statements in the abandoned cluster highest while new curriculum students rated statements in the enthusiastic cluster highest. Given the ubiquity of curricular change which often includes a legacy cohort, medical schools embarking on this journey must carefully attend to the varied emotional responses of their different stakeholder groups. The very activities recommended by organizational change models used in medical education, such as communicating wins early and often, could alienate legacy students, creating emotional polarization. These findings suggest that tailored communication strategies are necessary during change implementation to optimize success.

Communicating Diagnostic Uncertainty at Emergency Department Discharge: A Simulation-Based Mastery Learning Randomized Trial.

PURPOSE: There are no standardized approaches for communicating with patients discharged from the emergency department with diagnostic uncertainty. This trial tested efficacy of the Uncertainty Communication Education Module, a simulation-based mastery learning curriculum designed to establish competency in communicating diagnostic uncertainty. METHOD: Resident physicians at 2 sites participated in a 2-arm waitlist randomized controlled trial from September 2019 to June 2020. After baseline (T1) assessment of all participants via a standardized patient encounter using the Uncertainty Communication Checklist (UCC), immediate access physicians received training in the Uncertainty Communication Education Module, which included immediate feedback, online educational modules, a smartphone-based application, and telehealth deliberate practice with standardized patients. All physicians were retested 16-19 weeks later (T2) via in-person standardized patient encounters; delayed access physicians then received the intervention. A final test of all physicians occurred 11-15 weeks after T2 (T3). The primary outcome measured the percentage of physicians in the immediate versus delayed access groups meeting or exceeding the UCC minimum passing standard at T2. RESULTS: Overall, 109 physicians were randomized, with mean age 29 years (range 25-46). The majority were male (n = 69, 63%), non-Hispanic/Latino (n = 99, 91%), and White (n = 78, 72%). At T2, when only immediate access participants had received the curriculum, immediate access physicians demonstrated increased mastery (n = 29, 52.7%) compared with delayed access physicians (n = 2, 3.7%, P < .001; estimated adjusted odds ratio of mastery for the immediate access participants, 31.1 [95% CI, 6.8-143.1]). There were no significant differences when adjusting for training site or stage of training. CONCLUSIONS: The Uncertainty Communication Education Module significantly increased mastery in communicating diagnostic uncertainty at the first postintervention test among emergency physicians in standardized patient encounters. Further work should assess the impact of clinical implementation of these communication skills.

Assessing patient, physician, and practice characteristics predicting the use of low-value services.

OBJECTIVE: To examine characteristics of beneficiaries, physicians, and their practice sites associated with greater use of low-value services (LVS) using LVS measures that reflect current care practices. DATA SOURCES: This study was conducted in the context of a large, nationwide primary care redesign initiative (Comprehensive Primary Care Plus), using Medicare claims data in 2018. STUDY DESIGN: We examined beneficiary-level total counts of LVS based on the existing 31 claims-based measures updated by excluding three services provided with diminishing frequency to Medicare beneficiaries and by replacing these with more recently identified LVS. We estimated hierarchical linear models with an extensive list of beneficiary, physician, and practice site characteristics to examine the contribution of characteristics at each level in predicting greater use of LVS. We also examined the proportion of variation in LVS use attributable to the set of characteristics at each level. DATA COLLECTION/EXTRACTION METHODS: The study included 5,074,642 Medicare fee-for-service beneficiaries attributed to 32,406 primary care physicians in 11,009 primary care practice sites. PRINCIPAL FINDINGS: Patients with disabilities, end-stage renal disease, and those in regions with higher poverty rates receive 10 (standard error [SE] = 3.0), 80 (SE = 14.0), and 10 (SE = 1.0) more LVS per 1000 beneficiaries across all 31 measures combined than patients without such attributes, respectively. Greater physician comprehensiveness and an increase in the number of primary care practitioners at a practice were associated with 40 (SE = 20.0) and 20 (SE = 6.0) fewer LVS per 1000 beneficiaries, respectively. Yet, the explanatory variables we examined only account for 11 percent of the variation in LVS use, with most of the variation (87 percent) being due to unobserved differences at the beneficiary level. CONCLUSIONS: Unexplained residual variation, from underlying patient preferences and behavior of non-primary care providers, could be important determinants of LVS use.

Preparing women experiencing symptoms of menopause for shared decision making about treatment.

OBJECTIVE: To develop a decision support intervention that can be used with women experiencing menopausal symptoms to facilitate treatment shared decision making. METHODS: Our research team contacted patients with reported menopausal symptoms by telephone to obtain consent and administer a baseline survey. Subsequently, we sent participants a booklet on the treatment of menopausal symptoms. A nurse educator then contacted participants by telephone to review the booklet and guide them through a structured decision counseling exercise designed to help clarify treatment preference. A 60-day endpoint telephone survey was completed. RESULTS: Forty-eight consenting participants completed the baseline survey and 37 (77%) also completed a decision counseling session. At baseline, 19 of the women who had decision counseling were not being treated for menopausal symptoms and 18 were being treated. After decision counseling, 13 (68%) participants who were not being treated and 14 (78%) who were being treated identified a preferred treatment. Comparison of baseline and endpoint survey data showed that participant treatment knowledge increased (P = 0.007) and treatment decisional conflict decreased (P < 0.001). Furthermore, 71% of participants reported that they had received new information about treatment and 94% said they believed better prepared to discuss treatment with their healthcare provider. CONCLUSIONS: Nurse-led decision counseling increased participant treatment knowledge, reduced treatment decisional conflict, and helped to clarify treatment preference. Implementation of this strategy could help to facilitate provider-patient shared decision making about the treatment of menopausal symptoms.

Assessing the impact of medically tailored meals and medical nutrition therapy on type 2 diabetes: Protocol for Project MiNT.

BACKGROUND: Research has shown that among people with type 2 diabetes mellitus, reduction in hemoglobin A1c (HbA1c) prevents long term complications. Medically tailored meals (MTM) and telehealth-delivered medical nutrition therapy (tele-MNT) are promising strategies for patient-centered diabetes care. OBJECTIVES: Project MiNT will determine whether provision of MTM with and without the addition of telehealth-delivered medical nutrition therapy improves HbA1c and is cost effective for patients with type 2 diabetes mellitus. METHODS: Patients with poorly controlled type 2 diabetes mellitus (HbA1c >8%) will be recruited from Jefferson Health. Eligible patients will be randomized to one of three arms: 1) usual care, 2) 12 weeks of home-delivered MTM, or 3) MTM + 12 months of tele-MNT. All participants (n = 600) will complete three follow-up assessments at 3, 6, and 12 months. The primary outcome is change in HbA1c at 6 months. Secondary outcomes include change in HbA1c at 3 and 12 months and cost-effectiveness of the intervention at 6 and 12 months. Conclusion Findings from Project MiNT will inform MTM coverage and financing decisions, how to structure services for scalability and system-wide integration, and the role of these services in reducing health disparities.

Strategies for overcoming therapeutic inertia in type 2 diabetes: A systematic review and meta-analysis.

AIMS: To systematically investigate the effect of interventions to overcome therapeutic inertia on glycaemic control in individuals with type 2 diabetes. MATERIALS AND METHODS: We electronically searched for randomized controlled trials or quasi-experimental studies published between January 1, 2004 and December 31, 2019 evaluating the effect of interventions on glycated haemoglobin (HbA1c) control. Characteristics of included studies and HbA1c difference between intervention and control arms (main outcome) were extracted. Interventions were grouped as: care management and patient education; nurse or certified diabetes educator (CDE); pharmacist; or physician-based. RESULTS: Thirty-six studies including 22 243 individuals were combined in nonlinear random-effects meta-regressions; the median (range) duration of intervention was 1 year (0.9 to 36 months). Compared to the control arm, HbA1c reduction ranged from: -17.7 mmol/mol (-1.62%) to -4.4 mmol/mol (-0.40%) for nurse- or CDE-based interventions; -13.1 mmol/mol (-1.20%) to 3.3 mmol/mol (0.30%) for care management and patient education interventions; -9.8 mmol/mol (-0.90%) to -6.6 mmol/mol (-0.60%) for pharmacist-based interventions; and -4.4 mmol/mol (-0.40%) to 2.8 mmol/mol (0.26%) for physician-based interventions. Across the included studies, a reduction in HbA1c was observed only during the first year (6 months: -4.2 mmol/mol, 95% confidence interval [CI] -6.2, -2.2 [-0.38%, 95% CI -0.56, -0.20]; 1 year: -1.6 mmol/mol, 95% CI -3.3, 0.1 [-0.15%, 95% CI -0.30, 0.01]) and in individuals with preintervention HbA1c >75 mmol/mol (9%). CONCLUSIONS: The most effective approaches to mitigating therapeutic inertia and improving HbA1c were those that empower nonphysician providers such as pharmacists, nurses and diabetes educators to initiate and intensify treatment independently, supported by appropriate guidelines.

A National Research Strategy to Reduce Firearm-Related Injury and Death: Recommendations from the Health Policy Research Subcommittee of the Society of General Internal Medicine (SGIM).

Injury due to firearms is a serious health issue in the USA, leading to nearly 40,000 deaths annually and many more non-fatal injuries. Despite the significant impact on morbidity and mortality, relatively little research funding is dedicated to understanding the impact of firearm-related injury and to developing strategies to mitigate harm. In part, research has been stymied by decades-old language in federal legislation that was interpreted as prohibiting federal funding for firearm injury-related research. This paper, prepared by members of the Society of General Internal Medicine (SGIM), calls for support for research that seeks to understand the nature of firearm-related injury and resources to develop effective approaches to prevent it. We outline recommendations to develop evidence to inform policymakers and the medical and public health communities. These recommendations include (1) development of a shared national research agenda to address firearm-related injury and death; (2) allocation of federal funds specifically for research related to firearm injury; (3) support for the career development of researchers studying firearm-related injury; and (4) facilitating access to comprehensive data sources needed for developing evidence.

Simulation-based mastery learning compared to standard education for discussing diagnostic uncertainty with patients in the emergency department: a randomized controlled trial.

BACKGROUND: Diagnostic uncertainty occurs frequently in emergency medical care, with more than one-third of patients leaving the emergency department (ED) without a clear diagnosis. Despite this frequency, ED providers are not adequately trained on how to discuss diagnostic uncertainty with these patients, who often leave the ED confused and concerned. To address this training need, we developed the Uncertainty Communication Education Module (UCEM) to teach physicians how to discuss diagnostic uncertainty. The purpose of the study is to evaluate the effectiveness of the UCEM in improving physician communications. METHODS: The trial is a multicenter, two-arm randomized controlled trial designed to teach communication skills using simulation-based mastery learning (SBML). Resident emergency physicians from two training programs will be randomly assigned to immediate or delayed receipt of the two-part UCEM intervention after completing a baseline standardized patient encounter. The two UCEM components are: 1) a web-based interactive module, and 2) a smart-phone-based game. Both formats teach and reinforce communication skills for patient cases involving diagnostic uncertainty. Following baseline testing, participants in the immediate intervention arm will complete a remote deliberate practice session via a video platform and subsequently return for a second study visit to assess if they have achieved mastery. Participants in the delayed intervention arm will receive access to UCEM and remote deliberate practice after the second study visit. The primary outcome of interest is the proportion of residents in the immediate intervention arm who achieve mastery at the second study visit. DISCUSSION: Patients' understanding of the care they received has implications for care quality, safety, and patient satisfaction, especially when they are discharged without a definitive diagnosis. Developing a patient-centered diagnostic uncertainty communication strategy will improve safety of acute care discharges. Although use of SBML is a resource intensive educational approach, this trial has been deliberately designed to have a low-resource, scalable intervention that would allow for widespread dissemination and uptake. TRIAL REGISTRATION: The trial was registered at clinicaltrials.gov (NCT04021771). Registration date: July 16, 2019.

Development of the Uncertainty Communication Checklist: A Patient-Centered Approach to Patient Discharge From the Emergency Department.

Clear communication with patients upon emergency department (ED) discharge is important for patient safety during the transition to outpatient care. Over one-third of patients are discharged from the ED with diagnostic uncertainty, yet there is no established approach for effective discharge communication in this scenario. From 2017 to 2019, the authors developed the Uncertainty Communication Checklist for use in simulation-based training and assessment of emergency physician communication skills when discharging patients with diagnostic uncertainty. This development process followed the established 12-step Checklist Development Checklist framework and integrated patient feedback into 6 of the 12 steps. Patient input was included as it has potential to improve patient-centeredness of checklists related to assessment of clinical performance. Focus group patient participants from 2 clinical sites were included: Thomas Jefferson University Hospital, Philadelphia, PA, and Northwestern University Hospital, Chicago, Illinois.The authors developed a preliminary instrument based on existing checklists, clinical experience, literature review, and input from an expert panel comprising health care professionals and patient advocates. They then refined the instrument based on feedback from 2 waves of patient focus groups, resulting in a final 21-item checklist. The checklist items assess if uncertainty was addressed in each step of the discharge communication, including the following major categories: introduction, test results/ED summary, no/uncertain diagnosis, next steps/follow-up, home care, reasons to return, and general communication skills. Patient input influenced both what items were included and the wording of items in the final checklist. This patient-centered, systematic approach to checklist development is built upon the rigor of the Checklist Development Checklist and provides an illustration of how to integrate patient feedback into the design of assessment tools when appropriate.

Setting a Minimum Passing Standard for the Uncertainty Communication Checklist Through Patient and Physician Engagement.

BACKGROUND: Historically, medically trained experts have served as judges to establish a minimum passing standard (MPS) for mastery learning. As mastery learning expands from procedure-based skills to patient-centered domains, such as communication, there is an opportunity to incorporate patients as judges in setting the MPS. OBJECTIVE: We described our process of incorporating patients as judges to set the MPS and compared the MPS set by patients and emergency medicine residency program directors (PDs). METHODS: Patient and physician panels were convened to determine an MPS for a 21-item Uncertainty Communication Checklist. The MPS for both panels were independently calculated using the Mastery Angoff method. Mean scores on individual checklist items with corresponding 95% confidence intervals were also calculated for both panels and differences analyzed using a t test. RESULTS: Of 240 eligible patients and 42 eligible PDs, 25 patients and 13 PDs (26% and 65% cooperation rates, respectively) completed MPS-setting procedures. The patient-generated MPS was 84.0% (range 45.2-96.2, SD 10.2) and the physician-generated MPS was 88.2% (range 79.7-98.1, SD 5.5). The overall MPS, calculated as an average of these 2 results, was 86.1% (range 45.2-98.1, SD 9.0), or 19 of 21 checklist items. CONCLUSIONS: Patients are able to serve as judges to establish an MPS using the Mastery Angoff method for a task performed by resident physicians. The patient-established MPS was nearly identical to that generated by a panel of residency PDs, indicating similar expectations of proficiency for residents to achieve skill "mastery."

Development and preliminary validation of a scale to measure patient uncertainty: The "Uncertainty Scale".

Research suggests that patient uncertainty related to experiencing symptoms may drive decisions to seek care. The only validated measure of patient uncertainty assesses uncertainty related to defined illness. In prior work, we engaged patients to describe uncertainty related to symptoms and used findings to develop the 'U-Scale' scale. In this work, we present results from preliminary scale reliability and validity testing. Psychometric testing demonstrated content validity, high internal consistency, and evidence for concurrent validity. Next steps include administration in diverse populations for continued refinement and validation, and exploration of the potential contribution of uncertainty to healthcare utilization.

Use of Group Concept Mapping to Identify Patient Domains of Uncertainty That Contribute to Emergency Department Use.

INTRODUCTION: Prior research suggests that uncertainty related to symptoms is a driver of emergency department (ED) use, and that patients often leave the ED with uncertainty not being addressed. Our objective was to engage patients to identify domains that contribute to feelings of uncertainty and decisions to use the ED. METHODS: We used Group Concept Mapping, a quasi-qualitative/quasi-quantitative method, to elicit patients' views on how uncertainty related to experiencing symptoms contributes to decisions to access the ED. Purposive sampling was used to recruit participants who either sought treatment at the ED twice within a 30-day period, or visited both the ED and a primary care provider at least once within the past year. RESULTS: Thirty-four participants engaged in two rounds of Group Concept Mapping during which participants participated in structured brainstorming of ideas, followed by ranking and clustering of ideas into domains. The first round generated 47 idea statements reflecting uncertainty about consequences, severity, emergency room services, primary care options, finances, and psychologic concerns. The second round generated 52 idea statements reflecting uncertainty about self-management, causation, diagnosis and treatment plan, trust in the provider and institution, accessibility, and alternative care options. DISCUSSION: Factors that contribute to uncertainty and decision-making about ED use are both intrinsic (ie, cause, symptom severity) and extrinsic (ie, finances, accessibility). These domains can inform approaches to measure the uncertainty that patients experience, and to design and test interventions for nurses and other providers to help manage patient uncertainty during acute illness.