The Pennsylvania Rural Health Model: Hospitals' Early Experiences With Global Payment for Rural Communities.

GOAL: In January 2019, the first cohort of rural hospitals began to operate under the Pennsylvania Rural Health Model for all-payer prospective global budget reimbursement as part of a demonstration funded by the Center for Medicare and Medicaid Innovation. Using information from primary source documents and interviews with key stakeholders, we sought to identify challenges and lessons learned throughout the design, development, and early implementation stages of the model. METHODS: We relied on two qualitative research approaches: (1) review of primary source documents such as peer-reviewed publications and news accounts related to the model and (2) semistructured interviews with key staff and stakeholders, including current and former members of the Pennsylvania Department of Health, first-year applicant hospitals, technical assistance providers, and members of state and federal organizations and agencies familiar with the Pennsylvania and Maryland payment reform efforts for rural health and rural hospitals (N = 20). PRINCIPAL FINDINGS: We identified four primary attributes that innovative projects such as the model need: (1) a champion at the state and hospital level, significant cooperation across state agencies and between federal and state agencies, and support from nongovernment stakeholders; (2) ongoing engagement and education of all stakeholders, particularly related to rural health disparities, the challenges faced by rural hospitals (especially resource limitations), and the differences between rural and urban health and health service delivery; (3) realistic time lines, noting that stakeholder relationships with hospital leadership develop over many months; and (4) multistakeholder collaboration, because participating hospitals must have ongoing engagement with community members (i.e., consumers of healthcare), nonacute community partners, and other rural hospitals to foster a "rural health movement." APPLICATIONS TO PRACTICE: A successful Pennsylvania model holds promise for other states seeking to address the needs of rural populations and the hospitals that are vital to those communities. The lessons in this article can assist others in making the transition from volume to value in rural healthcare.

Member engagement in multi-sector health care alliances.

Many communities are developing innovative forms of collaborative organizations such as multi-sector health care alliances (MHCAs) to address problems of misaligned incentives among providers, payers, and community stakeholders and improve health and health care. Member engagement is essential to the success of these organizations due to their dependence on volunteer members to develop and implement strategy and provide material and in-kind support for alliance efforts, yet relatively little research has examined how alliances can foster engagement. This study examined behavioral indicators of member engagement (e.g., recruitment and retention of organizational and individual members) and how they are related to two foundational dimensions of alliance functioning - alliance leadership and community centrality. Using three rounds of an internet-based survey of alliance members from 14 alliances, the study found that organizational recruitment and retention increased over time, from 26.6% to 41.5% and 56.0% to 65.2%, respectively. Recruitment of individuals increased over the study period (38.3% to 47.2%, while retention of individual members declined over the study period (61.0% to 53.2%). Alliance leadership was associated with lower levels of recruitment (both organizational and individual members) but higher levels of organizational retention (both organizational and individual members). Collectively, our findings suggest that behavioral aspects of alliances are more effective at retaining members than relatively stable characteristics such as size and positioning in the community. Contrasting relationships between recruitment and retention, however, suggest that different forms of leadership may be required to simultaneously attract new members while retaining existing ones.

RNAi-mediated gene knockdown of progesterone 5ß-reductases in Digitalis lanata reduces 5ß-cardenolide content.

KEY MESSAGE: Studying RNAi-mediated DlP5ßR1 and DlP5ßR2 knockdown shoot culture lines of Digitalis lanata, we here provide direct evidence for the participation of PRISEs (progesterone 5ß-reductase/iridoid synthase-like enzymes) in 5ß-cardenolide formation. Progesterone 5ß-reductases (P5ßR) are assumed to catalyze the reduction of progesterone to 5ß-pregnane-3,20-dione, which is a crucial step in the biosynthesis of the 5ß-cardenolides. P5ßRs are encoded by VEP1-like genes occurring ubiquitously in embryophytes. P5ßRs are substrate-promiscuous enone-1,4-reductases recently termed PRISEs (progesterone 5ß-reductase/iridoid synthase-like enzymes). Two PRISE genes, termed DlP5ßR1 (AY585867.1) and DlP5ßR2 (HM210089.1) were isolated from Digitalis lanata. To give experimental evidence for the participation of PRISEs in 5ß-cardenolide formation, we here established several RNAi-mediated DlP5ßR1 and DlP5ßR2 knockdown shoot culture lines of D. lanata. Cardenolide contents were lower in D. lanata P5ßR-RNAi lines than in wild-type shoots. We considered that the gene knockdowns may have had pleiotropic effects such as an increase in glutathione (GSH) which is known to inhibit cardenolide formation. GSH levels and expression of glutathione reductase (GR) were measured. Both were higher in the Dl P5ßR-RNAi lines than in the wild-type shoots. Cardenolide biosynthesis was restored by buthionine sulfoximine (BSO) treatment in Dl P5ßR2-RNAi lines but not in Dl P5ßR1-RNAi lines. Since progesterone is a precursor of cardenolides but can also act as a reactive electrophile species (RES), we here discriminated between these by comparing the effects of progesterone and methyl vinyl ketone, a small RES but not a precursor of cardenolides. To the best of our knowledge, we here demonstrated for the first time that P5ßR1 is involved in cardenolide formation. We also provide further evidence that PRISEs are also important for plants dealing with stress by detoxifying reactive electrophile species (RES).

21-Hydroxypregnane 21-O-malonylation, a crucial step in cardenolide biosynthesis, can be achieved by substrate-promiscuous BAHD-type phenolic glucoside malonyltransferases from Arabidopsis thaliana and homolog proteins from Digitalis lanata.

Three putative 21-hydroxypregnane 21-O-malonyltransferases (21MaT) from Digitalis lanata were partially purified. Two of them were supposed to be BAHD-type enzymes. We were unable to purify them in quantities necessary for reliable sequencing. We identified two genes in A. thaliana coding for substrate-promiscuous BAHD-type phenolic glucoside malonyltransferases (AtPMaT1, AtPMaT2) and docked various 21-hydroxypregnanes into the substrate-binding site of a homology model built on the BAHD template 2XR7 (NtMaT1 from N. tabacum). Recombinant forms of Atpmat1 and Atpmat2 were expressed in E. coli and the recombinant enzymes characterized with regard to their substrate preferences. They were shown to malonylate various 21-hydroxypregnanes. The Atpmat1 sequence was used to identify candidate genes in Digitalis lanata (Dlmat1 to Dlmat4). Dlmat1 and Dlmat2 were also expressed in E. coli and shown to possess 21-hydroxypregnane 21-O-malonyltransferase activity.

Understanding Nutritional Problems of Metastatic Breast Cancer Patients: Opportunities for Supportive Care Through eHealth.

BACKGROUND: Metastatic breast cancer patients are now living longer but cope with potential symptoms of metastatic disease and prolonged cancer treatment. Nutrition can play a vital role in managing these sequelae, and eHealth tools are emerging as promising delivery options for nutrition interventions. OBJECTIVE: To qualitatively assess nutritional problems and concerns of women with metastatic breast cancer and to explore how to address these problems within an existing eHealth platform. METHODS: Semistructured interviews were conducted with 21 women with metastatic breast cancer participating in an ongoing eHealth study. Interviews were audiotaped, transcribed verbatim, and analyzed using a team-based content analysis approach. RESULTS: Most respondents reported currently or previously experiencing nutritional problems due to adverse effects of cancer treatment or the disease itself; these were rarely addressed during routine clinical care. Five major themes emerged: (1) knowledge about nutrition, (2) nutrition information-seeking, (3) social aspects of nutrition, (4) nutrition interest, and (5) how to address nutrition with an eHealth platform. The respondents reported diverse experiences and perspectives regarding nutrition and diet in the context of metastatic breast cancer within each theme. CONCLUSION: Findings indicate the importance of providing women with metastatic breast cancer with information about nutrition that is tailored to their specific disease, as these patients report both nutritional problems and uncertainty regarding nutritional strategies. IMPLICATIONS FOR PRACTICE: These findings are a first step toward designing supportive care interventions that could fill the gap related to nutritional concerns not addressed during routine clinical care.

Quality improvement education in surgical specialty training: A comparison of Vascular Surgery resident and Urology Resident experiences and attitudes.

BACKGROUND: Academic institutions have increasingly focused on educating physicians and surgeons in concepts of value-based care, including quality improvement (QI). The extent to which QI curricular competencies are addressed in specialty surgical residency training is unclear. METHODS: A survey instrument was developed by content experts and sent to Vascular Surgery and Urology residents electronically. Descriptive statistics and bivariate associations were calculated using StataMP 13.1. RESULTS: Vascular Surgery and Urology residents reported exposure to similar types of QI curriculum. Fewer than half of residents reported achieving targets for graduation (Vascular 31%, Urology 42%) related to QI, and few residents in either group felt very well-prepared to lead a QI initiative (Vascular 13%, Urology 8%). CONCLUSION: QI education in surgical specialty training amongst Vascular Surgery and Urology residencies is similar and insufficient. Surgical specialties may benefit from collaborative efforts to improve the quality of QI education.

The exercise in all chemotherapy trial.

BACKGROUND: Multiple international organizations have called for exercise to become standard practice in the setting of oncology care. The feasibility of integrating exercise within systemic chemotherapy has not been investigated. METHODS: Patients slated to receive infusion therapy between April 2017 and October 2018 were screened for possible inclusion. The study goal was to establish the acceptability and feasibility of embedding an exercise professional into the chemotherapy infusion suite as a method of making exercise a standard part of cancer care. The exercise prescriptions provided to patients were individualized according to results of brief baseline functional testing. RESULTS: In all, 544 patients were screened, and their respective treating oncologists deemed 83% of them to be medically eligible to participate. After further eligibility screening, 226 patients were approached. Nearly 71% of these patients (n = 160) accepted the invitation to participate in the Exercise in All Chemotherapy trial. Feasibility was established because 71%, 55%, 69%, and 63% of the aerobic, resistance, balance, and flexibility exercises prescribed to patients were completed. Qualitative data also supported the acceptability and feasibility of the intervention from the perspective of patients and clinicians. The per-patient cost of the intervention was $190.68 to $382.40. CONCLUSIONS: Embedding an exercise professional into the chemotherapy infusion suite is an acceptable and feasible approach to making exercise standard practice. Moreover, the cost of the intervention is lower than the cost of other common community programs. Future studies should test whether colocating an exercise professional with infusion therapy could reach more patients in comparison with not colocating. LAY SUMMARY: Few studies have tested the implementation of exercise for patients with cancer by embedding an exercise professional directly into the chemotherapy infusion suite. The Exercise in All Chemotherapy trial shows that this approach is both acceptable and feasible from the perspective of clinicians and patients.

Are health systems redesigning how health care is delivered?

OBJECTIVE: To explore why and how health systems are engaging in care delivery redesign (CDR)-defined as the variety of tools and organizational change processes health systems use to pursue the Triple Aim. STUDY SETTING: A purposive sample of 24 health systems across 4 states as part of the Agency for Healthcare Research and Quality's Comparative Health System Performance Initiative. STUDY DESIGN: An exploratory qualitative study design to gain an "on the ground" understanding of health systems' motivations for, and approaches to, CDR, with the goals of identifying key dimensions of CDR, and gauging the depth of change that is possible based on the particular approaches to redesign care being adopted by the health systems. DATA COLLECTION: Semi-structured telephone interviews with health system executives and physician organization leaders from 24 health systems (n = 162). PRINCIPAL FINDINGS: We identify and define 13 CDR activities and find that the health systems' efforts are varied in terms of both the combination of activities they are engaging in and the depth of innovation within each activity. Health system executives who report strong internal motivation for their CDR efforts describe more confidence in their approach to CDR than those who report strong external motivation. Health system leaders face uncertainty when implementing CDR due to a limited evidence base and because of the slower than expected pace of payment change. CONCLUSIONS: The ability to validly and reliably measure CDR activities-particularly across varying organizational contexts and markets-is currently limited but is key to better understanding CDR's impact on intended outcomes, which is important for guiding both health system decision making and policy making.

Understanding how health systems facilitate primary care redesign.

OBJECTIVE: To understand how health systems are facilitating primary care redesign (PCR), examine the PCR initiatives taking place within systems, and identify barriers to this work. STUDY SETTING: A purposive sample of 24 health systems in 4 states. STUDY DESIGN: Data were systematically reviewed to identify how system leaders define and implement initiatives to redesign primary care delivery and identify challenges. Researchers applied codes which were based on the theoretical PCR literature and created new codes to capture emerging themes. Investigators analyzed coded data then produced and applied a thematic analysis to examine how health systems facilitate PCR. DATA COLLECTION: Semi-structured telephone interviews with 162 system executives and physician organization leaders from 24 systems. PRINCIPAL FINDINGS: Leaders at all 24 health systems described initiatives to redesign the delivery of primary care, but many were in the early stages. Respondents described the use of centralized health system resources to facilitate PCR initiatives, such as regionalized care coordinators, and integrated electronic health records. Team-based care, population management, and care coordination were the most commonly described initiatives to transform primary care delivery. Respondents most often cited improving efficiency and enhancing clinician job satisfaction, as motivating factors for team-based care. Changes in payment and risk assumption as well as community needs were commonly cited motivators for population health management and care coordination. Return on investment and the slower than anticipated rate in moving from fee-for-service to value-based payment were noted by multiple respondents as challenges health systems face in redesigning primary care. CONCLUSIONS: Given their expanding role in health care and the potential to leverage resources, health systems are promising entities to promote the advancement of PCR. Systems demonstrate interest and engagement in this work but face significant challenges in getting to scale until payment models are in alignment with these efforts.

The importance of understanding and measuring health system structural, functional, and clinical integration.

OBJECTIVE: We explore if there are ways to characterize health systems-not already revealed by secondary data-that could provide new insights into differences in health system performance. We sought to collect rich qualitative data to reveal whether and to what extent health systems vary in important ways across dimensions of structural, functional, and clinical integration. DATA SOURCES: Interviews with 162 c-suite executives of 24 health systems in four states conducted through "virtual" site visits between 2017 and 2019. STUDY DESIGN: Exploratory study using thematic comparative analysis to describe factors that may lead to high performance. DATA COLLECTION: We used maximum variation sampling to achieve diversity in size and performance. We conducted, transcribed, coded, and analyzed in-depth, semi-structured interviews with system executives, covering such topics as market context, health system origin, organizational structure, governance features, and relationship of health system to affiliated hospitals and POs. PRINCIPAL FINDINGS: Health systems vary widely in size and ownership type, complexity of organization and governance arrangements, and ability to take on risk. Structural, functional, and clinical integration vary across systems, with considerable activity around centralizing business functions, aligning financial incentives with physicians, establishing enterprise-wide EHR, and moving toward single signatory contracting. Executives describe clinical integration as more difficult to achieve, but essential. Studies that treat "health system" as a binary variable may be inappropriately aggregating for analysis health systems of very different types, at different degrees of maturity, and at different stages of structural, functional, and clinical integration. As a result, a "signal" indicating performance may be distorted by the "noise." CONCLUSIONS: Developing ways to account for the complex structures of today's health systems can enhance future efforts to study systems as complex organizations, to assess their performance, and to better understand the effects of payment innovation, care redesign, and other reforms.

A National Survey of Integrated Vascular Surgery Residents' Experiences With and Attitudes About Quality Improvement During Residency.

BACKGROUND: Integrated vascular surgery residency, or "0+5," programs provide education in the Accreditation Council for Graduate Medical Education (ACGME) competencies of Systems-Based Practice (SBP) and Practice-Based Learning and Improvement (PBLI), which include milestones related to quality improvement (QI). It is unclear what QI curricula are in place in 0+5 programs nationally or how 0+5 residents perceive the importance of QI. OBJECTIVE: The purpose of this study is to assess current 0+5 residents' knowledge, experiences with, and attitudes about QI. DESIGN: A survey was developed using the ACGME Common Program Requirements and Milestones pertaining to QI. All 0+5 residents from 2017 to 2018 academic year were emailed an electronic link to the survey. Descriptive statistics and cross-tabulations were calculated using Stata/MP version 13.1. SETTING: All 0+5 vascular surgery residency programs in the United State (n = 52). PARTICIPANTS: The survey was completed by 35% (n = 90/257) of 0+5 residents, representing 75% of 0+5 programs in the United States (n = 39/52). RESULTS: Forty-one percent of respondents felt that applying QI methods is very important and 33% felt that QI education is very important for their future work, however, just 13% felt very prepared to lead a QI initiative. Residents' perceptions of preparedness to lead QI projects and the importance they attached to QI education were significantly influenced by their participation in a QI project (p = 0.003 and p = 0.038 respectively). Finally, just 8% (n = 6) of residents responded correctly to all 13 knowledge-based questions and these residents felt better prepared to lead a QI initiative compared to those who answered incorrectly (p = 0.002). CONCLUSIONS: Most 0+5 residents report participation in a QI project during residency, however, few feel prepared to lead a QI initiative in practice. Furthermore, only half of PGY5 0+5 residents report achieving specific ACGME targets for graduation pertaining to QI. Current QI curricula in 0+5 programs may be inadequate in teaching fundamental QI concepts and achieving ACGME competency targets for graduation.

Understanding U.S. Health Systems: Using Mixed Methods to Unpack Organizational Complexity.

INTRODUCTION: As hospitals and physician organizations increasingly vertically integrate, there is an important opportunity to use health systems to improve performance. Prior research has largely relied on secondary data sources, but little is known about how health systems are organized "on the ground" and what mechanisms are available to influence physician practice at the front line of care. METHODS: We collected in-depth information on eight health systems through key informant interviews, descriptive surveys, and document review. Qualitative data were systematically coded. We conducted analyses to identify organizational structures and mechanisms through which health systems influence practice. RESULTS: As expected, we found that health systems vary on multiple dimensions related to organizational structure (e.g., size, complexity) which reflects history, market and mission. With regard to levers of influence, we observed within-system variation both in mechanisms (e.g., employment of physicians, system-wide EHR, standardization of service lines) and level of influence. Concepts such as "core" versus "peripheral" were more salient than "ownership" versus "contract." DISCUSSION: Data from secondary sources can help identify and map health systems, but they do not adequately describe them or the variation that exists within and across systems. To examine the degree to which health systems can influence performance, more detailed and nuanced information on health system characteristics is necessary. CONCLUSION: The mixed-methods data accrual approach used in this study provides granular qualitative data that enables researchers to describe multi-layered health systems, grasp the context in which they operate, and identify the key drivers of performance.

Dissemination of quality improvement innovations by multisector health care alliances.

PURPOSE: Multisector health care alliances (alliances) are increasingly viewed as playing an important role in improving the health and health care of local populations, in part by disseminating innovative practices, yet alliances face a number of challenges to disseminating these practices beyond a limited set of initial participants. The purpose of this paper is to examine how alliances attempt to disseminate innovative practices and the facilitating and inhibiting factors that alliances confront when trying to do so. DESIGN/METHODOLOGY/APPROACH: The authors adopted multiple holistic case study design of eight alliances with a maximum variation case selection strategy to reflect a range of structural and geographic characteristics. Semi-structured interviews with staff, leaders and board members were used. FINDINGS: The findings show that dissemination is a multidirectional process that is closely if not inextricably intertwined with capacity- and context-related factors (of the alliance, partnering organizations and target organizations). Thus, standardized approaches to dissemination are likely the exception and not the rule, and highlight the value of existing frameworks as a starting point for conceptualizing the important aspects of dissemination, but they are incomplete in their description of the "on-the-ground" dissemination processes that occur in the context of collaborative organizational forms such as alliances. ORIGINALITY/VALUE: Despite a rapidly expanding evidence base to guide clinical and managerial decision making, this knowledge often fails to make its way into routine practice. Consequently, the search for effective strategies to reduce this gap has accelerated in the past decade. This study sheds light on those strategies and the challenges to implementing them.

CFTR Expression Analysis for Subtyping of Human Pancreatic Cancer Organoids.

BACKGROUND: Organoid cultures of human pancreatic ductal adenocarcinoma (PDAC) have become a promising tool for tumor subtyping and individualized chemosensitivity testing. PDACs have recently been grouped into different molecular subtypes with clinical impact based on cytokeratin-81 (KRT81) and hepatocyte nuclear factor 1A (HNF1A). However, a suitable antibody for HNF1A is currently unavailable. The present study is aimed at establishing subtyping in PDAC organoids using an alternative marker. METHODS: A PDAC organoid biobank was generated from human primary tumor samples containing 22 lines. Immunofluorescence staining was established and done for 10 organoid lines for cystic fibrosis transmembrane conductance regulator (CFTR) and KRT81. Quantitative real-time PCR (qPCR) was performed for CFTR and HNF1A. A chemotherapeutic drug response analysis was done using gemcitabine, 5-FU, oxaliplatin, and irinotecan. RESULTS: A biobank of patient-derived PDAC organoids was established. The efficiency was 71% (22/31) with 68% for surgical resections and 83% for fine needle aspirations. Organoids could be categorized into the established quasimesenchymal, exocrine-like, and classical subtypes based on KRT81 and CFTR immunoreactivity. CFTR protein expression was confirmed on the transcript level. CFTR and HNF1A transcript expression levels positively correlated (n = 10; r = 0.927; p = 0.001). PDAC subtypes of the primary tumors and the corresponding organoid lines were identical for most of the cases analyzed (6/7). Treatment with chemotherapeutic drugs revealed tendencies but no significant differences regarding drug responses. CONCLUSIONS: Human PDAC organoids can be classified into known subtypes based on KRT81 and CFTR immunoreactivity. CFTR and HNF1A mRNA levels correlated well. Furthermore, subtype-specific immunoreactivity matched well between PDAC organoids and the respective primary tumor tissue. Subtyping of human PDACs using CFTR might constitute an alternative to HNF1A and should be further investigated.

The perceived importance of intersectoral collaboration by health care alliances.

There is growing interest in the use of intersectoral collaboration (e.g., alliances, coalitions, partnerships) to address complex, health-related issues in local communities. Relatively little empirical research, however, has examined how to foster and sustain collaboration across sectors during later stages of development, despite a recognition that the needs and goals for collaboration may change over time. The purpose of this study was to address this gap by examining the perceptions of alliance participants regarding the importance of collaborating with different industry sectors as alliances transitioned from stable, prescriptive foundation support to a more uncertain future. Our findings suggest that, in addition to the contextual characteristics highlighted in previous research, the perceived importance of intersectoral collaboration varies for different types of alliances and participants. Moreover, the salience of these characteristics varied for different types of collaboration, in our case, collaboration with nonmedical health care sectors and nonhealth care sectors. Collectively, our findings point to the importance of thinking more comprehensively, across multiple levels of influence, when considering ways to foster or sustain intersectoral collaboration.

Erratum to "CFTR Expression Analysis for Subtyping of Human Pancreatic Cancer Organoids".

[This corrects the article DOI: 10.1155/2019/1024614.].

Funding profiles of multisector health care alliances and their positioning for sustainability.

Purpose The purpose of this paper is to examine the relationship between different aspects of alliance funding profiles (e.g. range of sources, dependence on specific sources) and participant' perceptions of how well the organization is positioned for the future. Design/methodology/approach A mixed method study in the context of eight alliances participating in the Robert Wood Johnson Foundation's Aligning Forces for Quality program. Data collection approaches included surveys of alliance participants and semi-structured interviews with alliance leaders. Findings The findings indicate that dependence on grant revenues, in particular, may be problematic for how well alliances are positioned for sustainability. While a number of approaches were identified to reduce dependence on grants, implementing these strategies presented more of a challenge for alliances due to the contextual demands of their external environment and a need to strike a balance between pursuing alternative revenue sources and fidelity to the mission and identity of the organization. Practical implications Alliance leaders need to have not only a broad and accurate understanding of their external environment, but also an appreciation of the alliance's identity in that environment. Collectively, the findings can help organizational leaders be more informed about their funding choices and the implications those choices have for the future of their organization. Originality/value Collaborative forms of organizations (e.g. alliances, coalitions, networks) are increasingly viewed as an effective means of addressing complex, multifaceted health, and social challenges. For collaborative organizations that depend on the coordinated efforts of volunteers, addressing such complex issues is predicated on sustaining programmatic activities as well as the interest and participation of stakeholders over extended periods of time. This study sheds light on how leaders of these organizations may improve their prospects for sustainability.

Evaluating a complex, multi-site, community-based program to improve healthcare quality: the summative research design for the Aligning Forces for Quality initiative.

OBJECTIVE: The Aligning Forces for Quality (AF4Q) initiative was the Robert Wood Johnson Foundation's (RWJF's) signature effort to increase the overall quality of healthcare in targeted communities throughout the country. In addition to sponsoring this 16-site complex program, RWJF funded an independent scientific evaluation to support objective research on the initiative's effectiveness and contributions to basic knowledge in 5 core programmatic areas. The research design, data, and challenges faced during the summative evaluation phase of this near decade-long program are discussed. STUDY DESIGN: A descriptive overview of the summative research design and its development for a multi-site, community-based, healthcare quality improvement initiative is provided. METHODS: The summative research design employed by the evaluation team is discussed. RESULTS: The evaluation team's summative research design involved a data-driven assessment of the effectiveness of the AF4Q program at large, assessments of the impact of AF4Q in the specific programmatic areas, and an assessment of how the AF4Q alliances were positioned for the future at the end of the program. CONCLUSION: The AF4Q initiative was the largest privately funded community-based healthcare improvement initiative in the United States to date and was implemented at a time of rapid change in national healthcare policy. The implementation of large-scale, multi-site initiatives is becoming an increasingly common approach for addressing problems in healthcare. The summative evaluation research design for the AF4Q initiative, and the lessons learned from its approach, may be valuable to others tasked with evaluating similarly complex community-based initiatives.