Flares and their impact among male urologic chronic pelvic pain syndrome patients: An in-depth qualitative analysis in the Multidisciplinary Approach to the Study of Chronic Pelvic Pain (MAPP) Research Network.

INTRODUCTION: There has been a sparse exploration of the lived experience of men with urologic chronic pelvic pain syndrome (UCPPS), and none with the goal of Investigating the experience of "flares" as part of this chronic pain syndrome in men. METHODS: We conducted three focus groups of male UCPPS patients at two sites of the MAPP Research Network (n = 16 total participants) to explore the full spectrum of flares and their impact on men's lives. RESULTS: Flare experiences were common and specific symptom components varied widely. Men reported nonpelvic symptoms (e.g., diarrhea), and variability in symptom intensity (mild to severe), duration (minutes to days), and frequency of flares. Flares episodes, and the threat of flares, were disruptive to their lives, social roles, and relationships. Distinct long-term impacts were reported, such as decreased sexual activity, decreased travel, and potential loss of employment or career. The themes included social isolation and the need for a sense of control and understanding over their unpredictable symptoms. CONCLUSIONS: Given their negative impact, future research with men and UCPPS should focus on approaches to prevent flares, and should consider a multimodal approach to reducing the frequency, severity, and/or duration. Quality of life may be improved by providing men with a sense of control over their symptoms and offering them multimodal treatment options, consistent with the recommendations for further research for women with UCPPS.

Understanding Waste in Health Care: Perceptions of Frontline Physicians Regarding Time Use and Appropriateness of Care They and Others Provide.

BACKGROUND: Approximately 30% of total US health care spending is thought to be "wasted" on activities like unnecessary and inefficiently delivered services. OBJECTIVES: To assess the perceptions of clinic-based physicians regarding their use of time and appropriateness of care provided. DESIGN: Cross-sectional online survey of all Southern California Permanente Medical Group partner and associate physicians (N = 1034) who were primarily providing clinic-based care in 1 of 4 geographically and operationally distinct Kaiser Permanente Southern California Medical Centers. MAIN OUTCOME MEASURES: The proportion of time spent on direct patient care tasks perceived to require the respondent's clinical/specialty training as a physician or another physician who has similar years of clinical training (vs physicians with fewer years of clinical training, nonphysicians, or automated or computerized systems), and the proportion of care provided by the respondent and by other physicians with whom they are familiar that is perceived to be appropriate (vs equivocal or inappropriate). RESULTS: More than 61% of respondents indicated that 15% of their time spent on direct patient care could be shifted to nonphysicians, and between 10% and 16% of care provided was equivocal or inappropriate. DISCUSSION: The low proportion of care perceived as equivocal or inappropriate indicates there is little room for reducing such care or that physicians have difficulty assessing care appropriateness. The latter suggests that attempts to reduce or to eliminate inappropriate care may be unsuccessful until physician beliefs, knowledge, or behaviors are better understood and addressed. CONCLUSION: On the basis of these findings, it is apparent that within at least one health care system, the opportunity to increase value through task shifting and avoiding inappropriate care is more narrow than commonly perceived on a national level.

Physician Professional Satisfaction and Area of Clinical Practice: Evidence from an Integrated Health Care Delivery System.

CONTEXT: For health care reform to succeed, health care systems need a professionally satisfied primary care workforce. Evidence suggests that primary care physicians are less satisfied than those in other medical specialties. OBJECTIVE: To assess three domains of physician satisfaction by area of clinical practice among physicians practicing in an established integrated health system. DESIGN: Cross-sectional online survey of all Southern California Permanente Medical Group (SCPMG) partner and associate physicians (N = 1034) who were primarily providing clinic-based care in 1 of 4 geographically and operationally distinct Kaiser Permanente Southern California Medical Centers. MAIN OUTCOME MEASURES: Primary measure was satisfaction with one's day-to-day professional life as a physician. Secondary measures were satisfaction with quality of care and income. RESULTS: Of the 636 physicians responding to the survey (61.5% response rate), on average, 8 in 10 SCPMG physicians reported satisfaction with their day-to-day professional life as a physician. Primary care physicians were only minimally less likely to report being satisfied (difference of 8.2-9.5 percentage points; p < 0.05) than were other physicians. Nearly all physicians (98.2%) were satisfied with the quality of care they are able to provide. Roughly 8 in 10 physicians reported satisfaction with their income. No differences were found between primary care physicians and those in other clinical practice areas regarding satisfaction with quality of care or income. CONCLUSION: It is possible to create practice settings, such as SCPMG, in which most physicians, including those in primary care, experience high levels of professional satisfaction.

A randomized, controlled trial of team-based competition to increase learner participation in quality-improvement education.

OBJECTIVE: Several barriers challenge resident engagement in learning quality improvement (QI). We investigated whether the incorporation of team-based game mechanics into an evidence-based online learning platform could increase resident participation in a QI curriculum. DESIGN: Randomized, controlled trial. SETTING: Tertiary-care medical center residency training programs. PARTICIPANTS: Resident physicians (n = 422) from nine training programs (anesthesia, emergency medicine, family medicine, internal medicine, ophthalmology, orthopedics, pediatrics, psychiatry and general surgery) randomly allocated to a team competition environment (n = 200) or the control group (n = 222). INTERVENTION: Specialty-based team assignment with leaderboards to foster competition, and alias assignment to de-identify individual participants. MAIN OUTCOME MEASURES: Participation in online learning, as measured by percentage of questions attempted (primary outcome) and additional secondary measures of engagement (i.e. response time). Changes in participation measures over time between groups were assessed with a repeated measures ANOVA framework. RESULTS: Residents in the intervention arm demonstrated greater participation than the control group. The percentage of questions attempted at least once was greater in the competition group (79% [SD ± 32] versus control, 68% [SD ± 37], P= 0.03). Median response time was faster in the competition group (P= 0.006). Differences in participation continued to increase over the duration of the intervention, as measured by average response time and cumulative percent of questions attempted (each P< 0.001). CONCLUSIONS: Team competition increases resident participation in an online course delivering QI content. Medical educators should consider game mechanics to optimize participation when designing learning experiences.

A Quasi-experimental Evaluation of Performance Improvement Teams in the Safety-Net: A Labor-Management Partnership Model for Engaging Frontline Staff.

BACKGROUND: Unit-based teams (UBTs), initially developed by Kaiser Permanente and affiliated unions, are natural work groups of clinicians, managers, and frontline staff who work collaboratively to identify areas for improvement and implement solutions. OBJECTIVE: We evaluated the UBT model implemented by the Los Angeles County Department of Health Services in partnership with its union to engage frontline staff in improving patient care. DESIGN: We conducted a quasi-experimental study, comparing surveys at baseline and 6 months, among personnel in 10 clinics who received UBT training to personnel in 5 control clinics. We also interviewed staff from 5 clinics that received UBT training and 3 control clinics. PARTICIPANTS: We conducted 330 surveys and 38 individual, semi-structured interviews with staff at an outpatient facility in South Los Angeles. INTERVENTIONS: Each UBT leader received an 8-hour training in basic performance improvement methods, and each UBT was assigned a team "coach." MAIN MEASURES: Our outcome measure was 6-month change in the "adaptive reserve" score, the units' self-reported ability to make and sustain change. We analyzed transcripts of the interviews to find common themes regarding the UBT intervention. KEY RESULTS: The survey response rate was 63% (158/252) at baseline and 75% (172/231) at 6 months. There was a significant difference-in-change in adaptive reserve between UBTs and non-UBTs at 6 months (+0.11 vs -0.13; P = .02). Nine of the 10 UBTs reported increases in adaptive reserve and 8 UBTs reported decreased no-show rates or patient length of stay in clinic. Staff overwhelmingly felt the UBTs were a positive intervention because it allowed all levels of staff to have a voice in improvement. CONCLUSIONS: Our results indicate that partnership between management and unions to engage frontline staff in teams may be a useful tool to improve delivery of health care in a safety-net setting.

Emergency physician perceptions of medically unnecessary advanced diagnostic imaging.

OBJECTIVES: The objective was to determine emergency physician (EP) perceptions regarding 1) the extent to which they order medically unnecessary advanced diagnostic imaging, 2) factors that contribute to this behavior, and 3) proposed solutions for curbing this practice. METHODS: As part of a larger study to engage physicians in the delivery of high-value health care, two multispecialty focus groups were conducted to explore the topic of decision-making around resource utilization, after which qualitative analysis was used to generate survey questions. The survey was extensively pilot-tested and refined for emergency medicine (EM) to focus on advanced diagnostic imaging (i.e., computed tomography [CT] or magnetic resonance imaging [MRI]). The survey was then administered to a national, purposive sample of EPs and EM trainees. Simple descriptive statistics to summarize physician responses are presented. RESULTS: In this study, 478 EPs were approached, of whom 435 (91%) completed the survey; 68% of respondents were board-certified, and roughly half worked in academic emergency departments (EDs). Over 85% of respondents believe too many diagnostic tests are ordered in their own EDs, and 97% said at least some (mean = 22%) of the advanced imaging studies they personally order are medically unnecessary. The main perceived contributors were fear of missing a low-probability diagnosis and fear of litigation. Solutions most commonly felt to be "extremely" or "very" helpful for reducing unnecessary imaging included malpractice reform (79%), increased patient involvement through education (70%) and shared decision-making (56%), feedback to physicians on test-ordering metrics (55%), and improved education of physicians on diagnostic testing (50%). CONCLUSIONS: Overordering of advanced imaging may be a systemic problem, as many EPs believe a substantial proportion of such studies, including some they personally order, are medically unnecessary. Respondents cited multiple complex factors with several potential high-yield solutions that must be addressed simultaneously to curb overimaging.

Emergency physician perceptions of shared decision-making.

OBJECTIVES: Despite the potential benefits of shared decision-making (SDM), its integration into emergency care is challenging. Emergency physician (EP) perceptions about the frequency with which they use SDM, its potential to reduce medically unnecessary diagnostic testing, and the barriers to employing SDM in the emergency department (ED) were investigated. METHODS: As part of a larger project examining beliefs on overtesting, questions were posed to EPs about SDM. Qualitative analysis of two multispecialty focus groups was done exploring decision-making around resource use to generate survey items. The survey was then pilot-tested and revised to focus on advanced diagnostic imaging and SDM. The final survey was administered to EPs recruited at four emergency medicine (EM) conferences and 15 ED group meetings. This report addresses responses regarding SDM. RESULTS: A purposive sample of 478 EPs from 29 states were approached, of whom 435 (91%) completed the survey. EPs estimated that, on average, multiple reasonable management options exist in over 50% of their patients and reported employing SDM with 58% of such patients. Respondents perceived SDM as a promising solution to reduce overtesting. However, despite existing research to the contrary, respondents also commonly cited beliefs that 1) "many patients prefer that the physician decides," 2) "when offered a choice, many patients opt for more aggressive care than they need," and 3) "it is too complicated for patients to know how to choose." CONCLUSIONS: Most surveyed EPs believe SDM is a potential high-yield solution to overtesting, but many perceive patient-related barriers to its successful implementation.

Urological chronic pelvic pain syndrome flares and their impact: qualitative analysis in the MAPP network.

INTRODUCTION AND HYPOTHESIS: Although in-depth qualitative information is critical to understanding patients' symptom experiences and to developing patient-centered outcome measures, only one previous qualitative study has assessed urological chronic pelvic pain syndrome (UCPPS) symptom exacerbations ("flares"). METHODS: We conducted eight focus groups of female UCPPS (interstitial cystitis/bladder pain syndrome) patients at four sites from the MAPP Research Network (n = 57, mean = 7/group) to explore the full spectrum of flares and their impact on patients' lives. RESULTS: Flare experiences were common and varied widely in terms of UCPPS symptoms involved, concurrent nonpelvic symptoms (e.g., diarrhea), symptom intensity (mild to severe), duration (minutes to years), and frequency (daily to < once/year), although the most commonly described flares were painful flares lasting days. These latter flares were also most disruptive to participants' lives, causing some to cancel social events, miss work or school, and in the worst cases, go to the emergency room or on disability leave. Participants also reported a longer-term impact of flares, including negative effects on their sexual functioning and marital, family, and social relationships; and the loss of employment or limited career or educational advancement. Emerging themes included the need for a sense of control over unpredictable symptoms and reduced social engagement. CONCLUSIONS: Given their negative impact, future research should focus on approaches to prevent flares, and to reduce their frequency, severity, and/or duration. Patients' quality of life may also be improved by providing them with a sense of control over their symptoms through ready access to flare medications/therapy, and by engaging them socially.

Evaluation of California's Statewide Mental Health Prevention and Early Intervention Programs: Summary of Key Year 2 Findings.

This article summarizes key findings from the second year of an evaluation of the California Mental Health Services Authority's statewide prevention and early intervention programs.

Sleep disturbances and nocturnal symptoms: relationships with quality of life in a population-based sample of women with interstitial cystitis/bladder pain syndrome.

STUDY OBJECTIVES: To characterize the nature and impact of sleep disturbances on quality of life (QOL) in women with interstitial cystitis/bladder pain syndrome (IC/BPS). METHODS: Participants were 3,397 women from a telephone probability survey who met IC/BPS symptom criteria. Sleep quality, duration, and IC/BPS nocturnal symptoms (i.e., trouble sleeping due to bladder pain, urgency, or needing to use the bathroom), general QOL (mental and physical health and sexual functioning), and IC/BPS QOL impairment were assessed via self-report during telephone interview. RESULTS: Over half of the sample reported poor sleep quality, sleep duration ≤ 6 hours, or trouble sleeping due to IC/BPS symptoms. After covariate adjustment, short sleep duration was significantly associated with greater IC/BPS QOL impairment (ß = -0.04; p < 0.001) and poorer self-reported physical health (ß = 1.86; p < 0.001). Poor sleep quality was significantly associated with greater IC/BPS QOL impairment (ß = 0.06; p < 0.001), poorer self-reported physical health (ß = -2.86; p < 0.001), and greater sexual dysfunction (ß = -0.04; p < 0.05). IC/BPS nocturnal symptoms were significantly associated with greater IC/BPS impairment (ß = 0.14; p < 0.001), poorer physical health (ß = -2.76; p < 0.001) and mental health (ß = 0.52; p < 0.01), and greater sexual dysfunction (ß = -0.10; p < 0.001), after covariate adjustment. After further adjustment for IC/BPS nocturnal symptoms, we found that poor sleep quality and short sleep duration were independent correlates of poor self-reported physical health. CONCLUSIONS: Poor sleep quality and short sleep duration, as well as disorder-specific sleep disturbances, are highly prevalent in women with IC/BPS and are associated with poorer disease-specific and general QOL.

Symptom persistence in a community cohort of women with interstitial cystitis/bladder pain syndrome (IC/BPS): 3-, 6-, 9-, and 12-month follow-up from the RICE cohort.

INTRODUCTION AND HYPOTHESIS: The persistence of interstitial cystitis/bladder pain syndrome (IC/BPS) symptoms has been described in women seeking medical care. The purpose of this study was to determine whether symptoms persist among a population-based sample of women. METHODS: A probability sample of US women was identified through a two-stage telephone screening process using the Research and Development (RAND) Interstitial Cystitis Epidemiology (RICE) high-sensitivity case definition. A randomly selected subgroup (n = 508) was enrolled in a longitudinal study and interviewed about their symptoms at baseline, 3, 6, 9, and 12 months. Bivariate and multivariate linear regression analyses determined predictors of persistence of symptoms over the four waves. RESULTS: A total of 436 women with a mean age of 47.5 years responding to all waves were included in the analysis. Forty-one percent met the RICE high-sensitivity case definition at baseline and in all four waves; an additional 21 % met the definition at baseline and in three waves. Women with a college degree (+12 % vs. no college, p = 0.02) and who were younger (-5 % per decade of age, p < 0.01) had higher chances of symptom persistence at each wave. Scoring one standard deviation higher on the continuity of symptoms and the O'Leary-Sant Interstitial Cystitis Symptom index increased the chances of symptom persistence by 4 % and 2 %, respectively (both p < 0.01). CONCLUSIONS: The majority of women demonstrated symptom persistence across at least three of four waves over 12 months. These women tended to be younger, college-educated, and to have reported a history of greater continuity of symptoms and higher severity of symptoms at baseline.

Consequences of interstitial cystitis/bladder pain symptoms on women's work participation and income: results from a national household sample.

PURPOSE: We describe differences in work participation and income by bladder symptom impact and comorbidities among women with interstitial cystitis/bladder pain syndrome. MATERIALS AND METHODS: Cross-sectional data from 2,767 respondents younger than 65 years identified with interstitial cystitis/bladder pain syndrome symptoms were analyzed. The data were taken from the RAND Interstitial Cystitis Epidemiology (RICE) survey, and included retrospective self-reports of interstitial cystitis/bladder pain syndrome impact, severity, years since onset, related comorbidities (depressive symptomatology, number of conditions), work participation and income, and personal characteristics. Multiple regressions predicted 5 current work outcomes of works now, kept from working by pain, missed work days, days worked when bothered by symptoms and real income change since symptom onset. RESULTS: Controlling for work status at symptom onset and personal characteristics, greater bladder symptom impact predicted a greater likelihood of not now working, kept more days from working by pain, missed more work days and working more days with symptoms. More depressive symptomatology and greater number of comorbidities predicted reduced work participation. Women experienced no growth in real income since symptom onset. Measures of symptom severity were not associated with any of the economic outcomes. CONCLUSIONS: Greater interstitial cystitis/bladder pain syndrome symptom impact, depressive symptomatology and count of comorbidities (but not symptom severity) were each associated with less work participation and leveling of women's long-term earnings. Management of bladder symptom impact on nonwork related activities and depressive symptomatology may improve women's work outcomes.

Evaluation of the California Mental Health Services Authority's Prevention and Early Intervention Initiatives: Executive Summary and Commentary.

In 2004, California voters passed Proposition 63, the Mental Health Services Act (MHSA), which includes a mandate that the state provide prevention and early intervention (PEI) services and education for people who experience mental illness in the state of California. The California Mental Health Services Authority (CalMHSA), a coalition of California counties formed to provide economic and administrative support to mental health service delivery, formed the Statewide PEI Implementation Program based on extensive recommendations from a large number of stakeholders statewide. The Statewide PEI program is made up of three strategic initiatives: (1) reduction of stigma and discrimination towards those with mental illness, (2) prevention of suicide, and (3) improvement in student mental health. This article provides a summary and commentary on a more detailed interim RAND evaluation of the CalMHSA Statewide PEI Program.

Evaluation of the California Mental Health Services Authority's Prevention and Early Intervention Initiatives: Progress and Preliminary Findings.

The California Mental Health Services Authority (CalMHSA) statewide Prevention and Early Intervention (PEI) program comprises three strategic initiatives: (1) reduction of stigma and discrimination toward those with mental illness, (2) prevention of suicide, and (3) improvement in student mental health. Community agencies serve as PEI program partners to perform activities intended to meet the goals of the initiatives. This article evaluates the progress of the PEI program partners in achieving their goals and establishes baseline population tracking of key risk factors and long-term outcomes targeted by the initiatives. Based on a model to assess the program partners' capacities and resources and a recent survey of California adults, this article shows that the partners have greatly expanded their abilities to launch numerous PEI activities and programs.

Recommendations for Sustaining California's Statewide Mental Health Prevention and Early Intervention Programs.

Provides preliminary advice about which of the California Mental Health Services Authority's Prevention and Early Intervention activities seem most valuable to sustain or, in some cases, enhance.

California's historic effort to reduce the stigma of mental illness: the Mental Health Services Act.

In a historic effort to reduce the stigma of mental illness, California voters approved the Mental Health Services Act in 2004. The law funds a comprehensive statewide prevention initiative that places stigma and discrimination reduction at its center, with 25 projects providing interventions at the institutional, societal, and individual levels. Stakeholders selected specific strategies from the research-based California Strategic Plan on Reducing Stigma and Discrimination. Strategies range from social marketing to increase public knowledge to capacity building at the local level, including training that emphasizes participation by consumers of mental health services and cultural competence. Collectively, these strategies aim to foster permanent change in the public perception of mental illness and in the individual experience of stigma. We examined the context, planning, programming, and evaluation of this effort.

Health-related quality of life in patients with interstitial cystitis/bladder pain syndrome and frequently associated comorbidities.

PURPOSE: To estimate the association of chronic non-urologic conditions [i.e., fibromyalgia (FM), chronic fatigue syndrome (CFS), and irritable bowel syndrome (IBS)] with health-related quality of life (HRQOL) in patients with interstitial cystitis/bladder pain syndrome (IC/BPS). METHODS: A total of 276 women with established diagnoses of IC/BPS completed a telephone interview which included demographics, self-reported medical conditions, the SF-36 health survey, and the interstitial cystitis symptom index (ICSI). Multivariate linear regression analysis was used to identify correlates of SF-36 physical and mental component summary scores. RESULTS: Mean patient age was 45.1 (SD 15.9) years, and 83% of the subjects were white. Mean values for the SF-36 Physical Component Score (PCS) and Mental Component Score (MCS) means were 39 (SD 14) and 45 (SD 12), respectively, indicating significant HRQOL reductions. Mean ICSI score was 11.27 (SD = 4.86). FM and IBS were significantly associated with worse SF-36 scores: -8 points on the PCS (p < 0.001) and -6 points on the MCS (p < 0.001). CFS and the presence of other pelvic conditions (overactive bladder, vulvodynia, endometriosis) were not significantly associated with SF-36 PCS and MCS scores. CONCLUSIONS: In patients with IC/BPS, the presence of FM, CFS, and IBS has a significant association with HRQOL, equivalent in impact to the bladder symptoms themselves. These results emphasize the importance of a multidisciplinary approach to treating patients with IC/BPS and other conditions.

The prevalence and overlap of interstitial cystitis/bladder pain syndrome and chronic prostatitis/chronic pelvic pain syndrome in men: results of the RAND Interstitial Cystitis Epidemiology male study.

PURPOSE: As part of the RICE (RAND Interstitial Cystitis Epidemiology) study, we developed validated case definitions to identify interstitial cystitis/bladder pain syndrome in women and chronic prostatitis/chronic pelvic pain syndrome in men. Using population based screening methods, we applied these case definitions to determine the prevalence of these conditions in men. MATERIALS AND METHODS: A total of 6,072 households were contacted by telephone to screen for men who had symptoms of interstitial cystitis/bladder pain syndrome or chronic prostatitis/chronic pelvic pain syndrome. An initial 296 men screened positive, of whom 149 met the inclusionary criteria and completed the telephone interview. For interstitial cystitis/bladder pain syndrome 2 case definitions were applied (1 with high sensitivity and 1 with high specificity), while for chronic prostatitis/chronic pelvic pain syndrome a single case definition (with high sensitivity and specificity) was used. These case definitions were used to classify subjects into groups based on diagnosis. RESULTS: The interstitial cystitis/bladder pain syndrome weighted prevalence estimates for the high sensitivity and high specificity definitions were 4.2% (3.1-5.3) and 1.9% (1.1-2.7), respectively. The chronic prostatitis/chronic pelvic pain syndrome weighted prevalence estimate was 1.8% (0.9-2.7). These values equate to 1,986,972 (95% CI 966,042-2,996,924) men with chronic prostatitis/chronic pelvic pain syndrome and 2,107,727 (95% CI 1,240,485-2,974,969) men with the high specificity definition of interstitial cystitis/bladder pain syndrome in the United States. The overlap between men who met the high specificity interstitial cystitis/bladder pain syndrome case definition or the chronic prostatitis/chronic pelvic pain syndrome case definition was 17%. CONCLUSIONS: Symptoms of interstitial cystitis/bladder pain syndrome and chronic prostatitis/chronic pelvic pain syndrome are widespread among men in the United States. The prevalence of interstitial cystitis/bladder pain syndrome symptoms in men approaches that in women, suggesting that this condition may be underdiagnosed in the male population.

Exploring the Addition of Physician Identifiers to the California Hospital Discharge Data Set.

To advance consideration of whether California should collect and release physician-identified data, RAND conducted a study to explore issues associated with requiring the inclusion of physician identifiers in the California hospital discharge data set and the potential use of physician-identified data by the state and/or release to others. RAND researchers conducted interviews with a broad set of California stakeholders, reviewed the legal and regulatory authority of the Office of Statewide Health Planning and Development to collect and release physician identifiers, and interviewed representatives from other states to understand any issues encountered by the states in their collection and use of physician-identified data. The authors found that physician-identified data could be useful to a variety of stakeholders. Of the 48 states that have hospital discharge reporting programs, all but California collect physician identifiers and do so without substantial burden to hospitals. States vary in their release policies, but those who do release the data have not reported problems. California stakeholders expressed concerns related to who would have access to the data, how the data would be analyzed, and how consumers would interpret the information, which should be carefully considered in efforts to advance the collection of physician identifiers in the California hospital discharge data.

CMS Innovation Center Health Care Innovation Awards: Evaluation Plan.

The Center for Medicare and Medicaid Innovation within the Centers for Medicare & Medicaid Services (CMS) has funded 108 Health Care Innovation Awards, funded through the Affordable Care Act, for applicants who proposed compelling new models of service delivery or payment improvements that promise to deliver better health, better health care, and lower costs through improved quality of care for Medicare, Medicaid, and Children's Health Insurance Program enrollees. CMS is also interested in learning how new models would affect subpopulations of beneficiaries (e.g., those eligible for Medicare and Medicaid and complex patients) who have unique characteristics or health care needs that could be related to poor outcomes. In addition, the initiative seeks to identify new models of workforce development and deployment, as well as models that can be rapidly deployed and have the promise of sustainability. This article describes a strategy for evaluating the results. The goal for the evaluation design process is to create standardized approaches for answering key questions that can be customized to similar groups of awardees and that allow for rapid and comparable assessment across awardees. The evaluation plan envisions that data collection and analysis will be carried out on three levels: at the level of the individual awardee, at the level of the awardee grouping, and as a summary evaluation that includes all awardees. Key dimensions for the evaluation framework include implementation effectiveness, program effectiveness, workforce issues, impact on priority populations, and context. The ultimate goal is to identify strategies that can be employed widely to lower cost while improving care.