Defining sustainability in practice: views from implementing real-world innovations in health care.

BACKGROUND: One of the key conceptual challenges in advancing our understanding of how to more effectively sustain innovations in health care is the lack of clarity and agreement on what sustainability actually means. Several reviews have helped synthesize and clarify how researchers conceptualize and operationalize sustainability. In this study, we sought to identify how individuals who implement and/or sustain evidence-informed innovations in health care define sustainability. METHODS: We conducted in-depth, semi-structured interviews with implementation leaders and relevant staff involved in the implementation of evidence-based innovations relevant to cancer survivorship care (n = 27). An inductive approach, using constant comparative analysis, was used for analysis of interview transcripts and field notes. RESULTS: Participants described sustainability as an ongoing and dynamic process that incorporates three key concepts and four important conditions. The key concepts were: (1) continued capacity to deliver the innovation, (2) continued delivery of the innovation, and (3) continued receipt of benefits. The key conditions related to (2) and (3), and included: (2a) innovations must continue in the absence of the champion or person/team who introduced it and (3a) adaptation is critical to ensuring relevancy and fit, and thus to delivering the intended benefits. CONCLUSIONS: Participants provided a nuanced view of sustainability, with both continued delivery and continued benefits only relevant under certain conditions. The findings reveal the interconnected elements of what sustainability means in practice, providing a unique and important perspective to the academic literature.

Knowledge translation for realist reviews: a participatory approach for a review on scaling up complex interventions.

BACKGROUND: Knowledge syntheses that use a realist methodology are gaining popularity. Yet, there are few reports in the literature that describe how results are summarised, shared and used. This paper aims to inform knowledge translation (KT) for realist reviews by describing the process of developing a KT strategy for a review on pathways for scaling up complex public health interventions. METHODS: The participatory approach used for the realist review was also used to develop the KT strategy. The approach included three main steps, namely (1) an international meeting focused on interpreting preliminary findings from the realist review and seeking input on KT activities; (2) a targeted literature review on KT for realist reviews; and (3) consultations with primary knowledge users of the review. RESULTS: The international meeting identified a general preference among knowledge users for findings from the review that are action oriented. A need was also identified for understanding how to tailor findings for specific knowledge user groups in relation to their needs. The literature review identified four papers that included brief descriptions of planned or actual KT activities for specific research studies; however, information was minimal on what KT activities or products work for whom, under what conditions and why. The consultations revealed that KT for realist reviews should consider the following: (1) activities closely aligned with the preferences of specific knowledge user groups; (2) key findings that are sensitive to factors within the knowledge user's context; and (3) actionable statements that can advance KT goals, activities or products. The KT strategy derived from the three activities includes a planning framework and tailored KT activities that address preferences of knowledge users for findings that are action oriented and context relevant. CONCLUSIONS: This paper provides an example of a KT strategy for realist reviews that blends theoretical and practical insights. Evaluation of the strategy's implementation will provide useful insights on its effectiveness and potential for broader application.

Setting strategy for system change: using concept mapping to prioritise national action for chronic disease prevention.

BACKGROUND: Chronic diseases are a serious and urgent problem, requiring at-scale, multi-component, multi-stakeholder action and cooperation. Despite numerous national frameworks and agenda-setting documents to coordinate prevention efforts, Australia, like many countries internationally, is yet to substantively impact the burden from chronic disease. Improved evidence on effective strategies for the prevention of chronic disease is required. This research sought to articulate a priority set of important and feasible action domains to inform future discussion and debate regarding priority areas for chronic disease prevention policy and strategy. METHODS: Using concept mapping, a mixed-methods approach to making use of the best available tacit knowledge of recognised, diverse and well-experienced actors, and national actions to improve the prevention of chronic disease in Australia were identified and then mapped. Participants (ranging from 58 to 78 in the various stages of the research) included a national sample of academics, policymakers and practitioners. Data collection involved the generation and sorting of statements by participants. A series of visual representations of the data were then developed. RESULTS: A total of 95 statements were distilled into 12 clusters for action, namely Inter-Sectoral Partnerships; Systems Perspective/Action; Governance; Roles and Responsibilities; Evidence, Feedback and Learning; Funding and Incentive; Creating Demand; Primary Prevention; Social Determinants and Equity; Healthy Environments; Food and Nutrition; and Regulation and Policy. Specific areas for more immediate national action included refocusing the health system to prevention over cure, raising the profile of public health with health decision-makers, funding policy- and practice-relevant research, improving communication about prevention, learning from both global best-practice and domestic successes and failures, increasing the focus on primary prevention, and developing a long-term prevention strategy with an explicit funding commitment. CONCLUSIONS: Preventing chronic diseases and their risk factors will require at-scale, multi-component, multi-stakeholder action and cooperation. The concept mapping procedures used in this research have enabled the synthesis of views across different stakeholders, bringing both divergent and convergent perspectives to light, and collectively creating signals for where to prioritise national action. Previous national strategies for chronic disease prevention have not collated the tacit knowledge of diverse actors in the prevention of chronic disease in this structured way.

Scaling up complex interventions: insights from a realist synthesis.

Preventing chronic diseases, such as cancer, cardiovascular disease and diabetes, requires complex interventions, involving multi-component and multi-level efforts that are tailored to the contexts in which they are delivered. Despite an increasing number of complex interventions in public health, many fail to be 'scaled up'. This study aimed to increase understanding of how and under what conditions complex public health interventions may be scaled up to benefit more people and populations.A realist synthesis was conducted and discussed at an in-person workshop involving practitioners responsible for scaling up activities. Realist approaches view causality through the linkages between changes in contexts (C) that activate mechanisms (M), leading to specific outcomes (O) (CMO configurations). To focus this review, three cases of complex interventions that had been successfully scaled up were included: Vibrant Communities, Youth Build USA and Pathways to Education. A search strategy of published and grey literature related to each case was developed, involving searches of relevant databases and nominations from experts. Data extracted from included documents were classified according to CMO configurations within strategic themes. Findings were compared and contrasted with guidance from diffusion theory, and interpreted with knowledge users to identify practical implications and potential directions for future research.Four core mechanisms were identified, namely awareness, commitment, confidence and trust. These mechanisms were activated within two broad scaling up strategies, those of renewing and regenerating, and documenting success. Within each strategy, specific actions to change contexts included building partnerships, conducting evaluations, engaging political support and adapting funding models. These modified contexts triggered the identified mechanisms, leading to a range of scaling up outcomes, such as commitment of new communities, changes in relevant legislation, or agreements with new funding partners.This synthesis applies and advances theory, realist methods and the practice of scaling up complex interventions. Practitioners may benefit from a number of coordinated efforts, including conducting or commissioning evaluations at strategic moments, mobilising local and political support through relevant partnerships, and promoting ongoing knowledge exchange in peer learning networks. Action research studies guided by these findings, and studies on knowledge translation for realist syntheses are promising future directions.

Outcomes of Interorganizational Networks in Canada for Chronic Disease Prevention: Insights From a Concept Mapping Study, 2015.

INTRODUCTION: We conducted a mixed methods study from June 2014 to March 2015 to assess the perspectives of stakeholders in networks that adopt a population approach for chronic disease prevention (CDP). The purpose of the study was to identify important and feasible outcome measures for monitoring network performance. METHODS: Participants from CDP networks in Canada completed an online concept mapping exercise, which was followed by interviews with network stakeholders to further understand the findings. RESULTS: Nine concepts were considered important outcomes of CDP networks: enhanced learning, improved use of resources, enhanced or increased relationships, improved collaborative action, network cohesion, improved system outcomes, improved population health outcomes, improved practice and policy planning, and improved intersectoral engagement. Three themes emerged from participant interviews related to measurement of the identified concepts: the methodological difficulties in measuring network outcomes, the dynamic nature of network evolution and function and implications for outcome assessment, and the challenge of measuring multisectoral engagement in CDP networks. CONCLUSION: Results from this study provide initial insights into concepts that can be used to describe the outcomes of networks for CDP and may offer foundations for strengthening network outcome-monitoring strategies and methodologies.

Content analysis of Australian direct-to-consumer websites for emerging breast cancer imaging devices.

OBJECTIVE: To describe the nature and frequency of information presented on direct-to-consumer websites for emerging breast cancer imaging devices. DESIGN: Content analysis of Australian website advertisements from 2 March 2011 to 30 March 2012, for three emerging breast cancer imaging devices: digital infrared thermal imaging, electrical impedance scanning and electronic palpation imaging. MAIN OUTCOME MEASURES: Type of imaging offered, device safety, device performance, application of device, target population, supporting evidence and comparator tests. RESULTS: Thirty-nine unique Australian websites promoting a direct-to-consumer breast imaging device were identified. Despite a lack of supporting evidence, 22 websites advertised devices for diagnosis, 20 advertised devices for screening, 13 advertised devices for prevention and 13 advertised devices for identifying breast cancer risk factors. Similarly, advertised ranges of diagnostic sensitivity (78%-99%) and specificity (44%-91%) were relatively high compared with published literature. Direct comparisons with conventional screening tools that favoured the new device were highly prominent (31 websites), and one-third of websites (12) explicitly promoted their device as a suitable alternative. CONCLUSIONS: Australian websites for emerging breast imaging devices, which are also available internationally, promote the use of such devices as safe and effective solutions for breast cancer screening and diagnosis in a range of target populations. Many of these claims are not supported by peer-reviewed evidence, raising questions about the manner in which these devices and their advertising material are regulated, particularly when they are promoted as direct alternatives to established screening interventions.

Advancing the application of systems thinking in health: advice seeking behavior among primary health care physicians in Pakistan.

BACKGROUND: Using measles and tuberculosis as case examples, with a systems thinking approach, this study examines the human advice-seeking behavior of primary health care (PHC) physicians in a rural district of Pakistan. This study analyzes the degree to which the existing PHC system supports their access to human advice, and explores in what ways this system might be strengthened to better meet provider needs. METHODS: The study was conducted in a rural district of Pakistan and, with a cross-sectional study design, it employed a range of research methods, namely extensive document review for mapping existing information systems, social network analysis of physicians' advice-seeking practice, and key stakeholder interviews for an in-depth understanding of the experience of physicians. Illustrations were prepared for information flow mechanism, sociographs were generated for analyzing social networks, and content analysis of qualitative findings was carried out for in-depth interpretation of underlying meanings. RESULTS: The findings of this study reveal that non-availability of competent supervisory staff, a focus on improving performance indicators rather than clinical guidance, and a lack of a functional referral system have collectively created an environment in which PHC physicians have developed their own strategies to overcome these constraints. They are well aware of the human expertise available within and outside the district. However, their advice-seeking behavior was dependent upon existence of informal social interaction with the senior specialists. Despite the limitations of the system, the physicians proactively used their professional linkages to seek advice and also to refer patients to the referral center based on their experience and the facilities that they trusted. CONCLUSIONS: The absence of functional referral systems, limited effective linkages between PHC and higher levels of care, and a focus on programmatic targets rather than clinical care have each contributed to the isolation of physicians and reactive information seeking behavior. The study findings underscore the need for a functional information system comprising context sensitive knowledge management and translation opportunities for physicians working in PHC centers. Such an information system needs to link people and resources in ways that transcend geography and discipline, and that builds on existing expertise, interpersonal relationships, and trust.

Improving the performance of interorganizational networks for preventing chronic disease: identifying and acting on research needs.

This article describes the role of interorganizational networks in chronic disease prevention and an action research agenda for promoting understanding and improvement. Through a model of engaged scholarship, leaders with expertise and experience in chronic disease prevention networks helped shape research directions focused on network value, governance, and evolution. The guiding principles for facilitating this research include applying existing knowledge, developing network-appropriate methods and measures, creating structural change, promoting an impact orientation, and fostering cultural change.

A systematic review of elastography, electrical impedance scanning, and digital infrared thermography for breast cancer screening and diagnosis.

The objective of this study aimed to systematically identify and evaluate all the available evidence of safety, effectiveness and diagnostic accuracy for three emerging classes of technology promoted for breast cancer screening and diagnosis: Digital infrared thermal imaging (DITI), electrical impedance scanning (EIS) and elastography. A systematic search of seven biomedical databases (EMBASE, PubMed, Web of Science, CRD, CINAHL, Cochrane Library, Current Contents Connect) was conducted through March 2011, along with a manual search of reference lists from relevant studies. The principal outcome measures were safety, effectiveness, and diagnostic accuracy. Data were extracted using a standardised form, and validated for accuracy by the secondary authors. Study quality was appraised using the quality assessment of diagnostic accuracy studies tool, while heterogeneity was assessed using forest plots, Cooks' distance and standardised residual scatter plots, and I (2) statistics. From 6,808 search results, 267 full-text articles were assessed, of which 60 satisfied the inclusion criteria. No effectiveness studies were identified. Only one EIS screening accuracy study was identified, while all other studies involved symptomatic populations. Significant heterogeneity was present among all device classes, limiting the potential for meta-analyses. Sensitivity and specificity varied greatly for DITI (Sens 0.25-0.97, Spec 0.12-0.85), EIS (Sens 0.26-0.98, Spec 0.08-0.81) and ultrasound elastography (Sens 0.35-1.00, Spec 0.21-0.99). It is concluded that there is currently insufficient evidence to recommend the use of these technologies for breast cancer screening. Moreover, the high level of heterogeneity among studies of symptomatic women limits inferences that may be drawn regarding their use as diagnostic tools. Future research employing standardised imaging, research and reporting methods is required.

Networks to strengthen health systems for chronic disease prevention.

Interorganizational networks that harness the priorities, capacities, and skills of various agencies and individuals have emerged as useful approaches for strengthening preventive services in public health systems. We use examples from the Canadian Heart Health Initiative and Alberta's Primary Care Networks to illustrate characteristics of networks, describe the limitations of existing frameworks for assessing the performance of prevention-oriented networks, and propose a research agenda for guiding future efforts to improve the performance of these initiatives. Prevention-specific assessment strategies that capture relevant aspects of network performance need to be identified, and feedback mechanisms are needed that make better use of these data to drive change in network activities.

Doing the dance of culture change: complexity, evidence and leadership.

The challenge of culture change in hospitals must address three distinct but interwoven tensions: the need to shift paradigm and understand healthcare as a complex adaptive system; the challenge of knitting together the contributions of both evidence-based medicine and practice-based evidence; and the critical role of distributed, problem-focused leadership.The authors of the lead paper highlight five key issues in addressing this challenge: (1) the implementation of strategies like front-line ownership (FLO) in the context of macro-level social forces; (2) the central role of distributed leadership and its strengthening within the organization; (3) the need to attend to developing systems thinking skills at all levels; (4) the very significant challenge of how to scale up the labour-intensive change strategies within FLO, the role of "simple rules" and the potential for systems thinking tools such as concept mapping and dynamic modelling; and (5) the concurrent orchestration of not one culture change but three tensions in the challenge FLO represents to simpler versus complex adaptive systems, leadership and management and the balance between evidence-based medicine and practice-based evidence, at the clinical, organizational and macro-system levels.

Over 150 potentially low-value health care practices: an Australian study.

OBJECTIVE: To develop and apply a novel method for scanning a range of sources to identify existing health care services (excluding pharmaceuticals) that have questionable benefit, and produce a list of services that warrant further investigation. DESIGN AND SETTING: A multiplatform approach to identifying services listed on the Australian Medicare Benefits Schedule (MBS; fee-for-service) that comprised: (i) a broad search of peer-reviewed literature on the PubMed search platform; (ii) a targeted analysis of databases such as the Cochrane Library and National Institute for Health and Clinical Excellence (NICE) "do not do" recommendations; and (iii) opportunistic sampling, drawing on our previous and ongoing work in this area, and including nominations from clinical and non-clinical stakeholder groups. MAIN OUTCOME MEASURES: Non-pharmaceutical, MBS-listed health care services that were flagged as potentially unsafe, ineffective or otherwise inappropriately applied. RESULTS: A total of 5209 articles were screened for eligibility, resulting in 156 potentially ineffective and/or unsafe services being identified for consideration. The list includes examples where practice optimisation (ie, assessing relative value of a service against comparators) might be required. CONCLUSION: The list of health care services produced provides a launchpad for expert clinical detailing. Exploring the dimensions of how, and under what circumstances, the appropriateness of certain services has fallen into question, will allow prioritisation within health technology reassessment initiatives.

Engaging clinicians in evidence-based disinvestment: role and perceptions of evidence.

OBJECTIVES: The aim of this study was to determine how evidence from systematic review (SR) is perceived and negotiated by expert stakeholders in considering a technology for potential disinvestment. METHODS: An evidence-informed stakeholder engagement examined results from a diagnostic accuracy SR of vitamin B12 and folate tests. Pathologists deliberated around the SR findings to generate an informed contribution to future policy for the funding of B12 and folate tests. Deliberations were transcribed and subject to qualitative analysis. RESULTS: Pathologists did not engage with findings from the SR in depth; rather they sought to contest the terms of the problem driving the review and attempted to reframe it. Pathologists questioned the usefulness of SR outcomes given the variable definitions of B12 deficiency and deferred addressing disinvestment options specifically pertaining to B12 testing. However, folate testing was proffered as a potential disinvestment candidate, based upon pathologists' definition of "appropriate" evidence beyond the bounds of the SR. CONCLUSIONS: The value of SR to informing disinvestment deliberations by expert stakeholders may be a function of timing as well as content. Engagement of stakeholders in co-produced evidence may be required at two levels: (i) Early in the synthesis phase to help shape the SR and harmonize expert views with the available evidence (including gaps); (ii) Collaboration in primary research to fill evidence-gaps thus supporting evidence-based disinvestment. Without this, information asymmetry between clinically engaged experts and decision makers may preclude the collaborative, informed, and technical discussions required to generate productive policy change.

Identifying and improving unreliable items in registries through data auditing.

OBJECTIVE: Assessing the reliability of clinical registries is important for ensuring the availability of credible data. Therefore, this study aimed to investigate the reliability of data collected by the Australian and New Zealand Haemostasis Registry (the registry). DESIGN: Data from 5% of randomly selected registry cases were re-abstracted by an independent data auditor who was blinded to the results of the original data abstraction. Categorical data were investigated for agreement between original and re-abstracted data. The mean difference and standard deviations (SD) of differences were calculated for continuous variables. We estimated a 'prediction interval' as the mean difference ± twice the SD of differences. We computed a coefficient of variation as the SD of differences. SETTING: The registry records all cases of off-licence use of recombinant activated factor VII (rFVIIa) at participating institutions (on-licence use of rFVIIa is not recorded). RESULTS: Data on 76 registry cases (6% of registry) were re-abstracted. Various parameters demonstrated high levels of inter-rater reliability, including age, gender and intensive care unit admission (88, 99 and 99% agreement, respectively). Other variables were highly unreliable, including crystalloid infusion volumes (coefficient of variation 123.01%), red blood cell units (92.05%) and time from bleeding onset to administration of rFVIIa (153.06%). CONCLUSIONS: Registry audits are useful for identifying variables with poor reliability. Repeated audits will not improve data reliability; however, they can assist in identifying and evaluating the impact of modified data collection processes on improving data reliability.

Applying composite performance measures to trauma care.

BACKGROUND: To investigate the association between a number of hospital level composite index methodologies developed from trauma indicators with inhospital mortality. METHODS: Data from January 2001 to December 2006 were extracted from the Victorian State Trauma Registry (Australia) and the Trauma Audit and Research Network (United Kingdom). Three composite methods were explored, including two denominator-based weight approaches and a factor analysis technique. The association between the composite measures and the count of inhospital mortality was investigated using Poisson regression models adjusting for expected deaths per hospital using the Trauma Injury Severity Score methodology. RESULTS: Composite scores were calculated per hospital, per year. The composite score was entered in statistical models as a raw score, and the mortality difference across the central 50% of the composite index was ascertained. In total, 9,218 patients were included and were distributed across 14 hospitals. Composite scores demonstrated an inverse relationship with risk-adjusted inhospital mortality. From the 25th to the 75th percentile of each composite, mortality decreased by 11.99%, 13.58%, and 16.13% (p < 0.05). CONCLUSION: Trauma composite indices demonstrate construct validity when used as measures of hospital level process and represent potentially useful methods of analyzing and reporting quality indicators.

Collaborative Value Creation Framework: The Potential for Strengthening Intersectoral Partnerships in Chronic Disease Prevention.

BACKGROUND: There is considerable theoretical appeal and investment in intersectoral partnerships for chronic disease prevention, yet there is limited understanding of pathways for creating collaborative value. OBJECTIVES: The purpose of this paper was to discuss the Collaborative Value Creation Framework in terms of its potential to understand how value is created in prevention partnerships. RESULTS: The framework holds promise for a) studying the significance of all types of collaborative value and in what contexts, b) examining how sources of collaborative value can be enhanced, c) surfacing how mindsets advance or hinder collaborative efforts, and d) establishing interdependent links with and between other elements of the framework such as collaborative processes, stages, and outcomes to increase value creation potential. CONCLUSIONS: Mapping the interdependence of all elements would be an innovation to the application of the framework to discover leverage points to increase collaborative value and strengthen prevention partnerships.

Using systems thinking and the Intervention Level Framework to analyse public health planning for complex problems: Otitis media in Aboriginal and Torres Strait Islander children.

BACKGROUND: Middle ear disease (otitis media) is endemic among Aboriginal and Torres Strait Islander children in Australia and represents an important cause of hearing loss. The disease is the result of a mix of biological, environmental and host risk factors that interact in complex, non-linear ways along a dynamic continuum. As such, it is generally recognised that a holistic, systems approach is required to reverse the high rates of otitis media in Aboriginal and Torres Strait Islander children. The objective of this paper is to examine the alignment between efforts designed to address otitis media in Aboriginal and Torres Strait Islander children in Queensland, Australia and core concepts of systems thinking. This paper's overall purpose is to identify which combination of activities, and at which level, hold the potential to facilitate systems changes to better support ear health among Aboriginal and Torres Strait Islander children. METHODS: We began with a review of documents identified in consultation with stakeholders and an online search. In addition, key informants were invited to participate in an online survey and a face-to-face or phone interview. Qualitative interviews using a semi-structured interview guide were used to explore survey responses in more depth. We also undertook interviews at the community level to elicit a diverse range of views. Ideas, statements or activities reported in the documents and interviews as being performed under the Intervention Level Framework were identified using qualitative thematic and content analysis. A quantitative descriptive analysis was also undertaken, whereby data was extracted into an Excel spreadsheet and coded under the relevant strategic directions and performance indicators of the Framework. Subsequently, we coded activities against the five-level intervention framework developed by Malhi and colleagues, that is: 1) paradigm; 2) goals; 3) system structure; 4) feedback and delays; and 5) structural elements. RESULTS: Overall, twenty documents were reviewed. We examined surveys and interviews with six key informants. Twenty-four individual and 3 group interviews were conducted across central and community level informants. One hundred and four items were coded from the 20 documents and 156 items from interview data. For both data sets, the majority of activities were coded at the structural elements level. The results suggested three key areas where further work is needed to drive sustained improvements: 1) build the governance structures needed for paradigm shift to achieve a multi-sectoral approach; 2) develop shared system level goals; 3) develop system-wide feedback processes. CONCLUSIONS: Sustained progress in improving ear health within Aboriginal and Torres Strait Islander children requires a holistic, system-wide approach. To advance such work, governance structures for multi-sectoral collaboration including the development of joint goals and monitoring and feedback are required. Intervening at these higher leverage points could have a profound effect on persistent public health issues.

Measuring quality.

It is no longer sufficient for health care professionals to provide high quality health care, they must also be able to demonstrate that they are meeting and often exceeding quality targets. Quality indicators (QIs) provide a means of measuring and assessing quality, however there are advantages and disadvantages of indicator measurement. Further, the clinical perspective needs to be balanced against managerial control when developing valid, reliable, sensitive and specific QIs. While indicators do not represent a perfect measurement device, they may provide a useful tool for improving patient safety and meeting community expectations.

Exploring the unanticipated effects of multi-sectoral partnerships in chronic disease prevention.

Multi-sectoral partnerships are important parts of many public health efforts to address chronic diseases, such as cancer, diabetes, and cardiovascular disease. Despite the potential value of multi-sectoral approaches, uncertainty exists regarding their effects on individuals, organizations, communities and populations. This article reports on a study that examined the unanticipated effects (both positive and negative) of the Public Health Agency of Canada's (the Agency) Multi-sectoral Partnerships initiative, which supports more than 30 multi-sectoral partnership projects across Canada. Thirteen semi-structured interviews were conducted with staff from organizations participating in 3 diverse partnership projects as part of the Agency's multi-sectoral partnerships initiative. Multiple unanticipated effects were identified and organized into 4 themes: (1) insights about the flexibility and responsiveness of government; (2) access to new and valuable resources (people, skills, expertise); (3) opportunity to build new capacities; and (4) understanding realistic timelines for partnership activities and outcomes. While these effects were unanticipated for study participants, they resonate with insights from the literature on multi-sectoral partnerships. These results raise a number of questions for consideration as partnership initiatives continue to evolve, including the types of training that partners might need; the individual and organizational capacities required for partnership approaches; and the evaluation techniques that might be most useful to capture the non-linear effects of partnership approaches.