Impact through research in education and studies in human society: A review of Australian Research Council 'high-for-impact' case studies.

Research impact is an important measure of the effective transmission and ongoing contribution of research beyond the scope of initial research publication outputs; however, determining what constitutes 'high-for-impact' research can be difficult for specific fields of study. This review of the Australian Research Council's Engagement and Impact Assessment 2018 analyses high-for-impact case studies submitted in the fields of Education (n = 17) and Studies in Human Society (n = 11) with the aim of understanding and explicating how high impact research has been evidenced in these fields. The review was guided by three research questions that concern the identification of the key characteristics of high-for-impact case studies, their reported impacts, and the evidence researchers cite to support claims of impact. The review highlights an important limitation in how impact is defined and understood by researchers, particularly cultural and social impact. Half of the analysed case studies involved international engagement, with minimal partner collaboration in the global south and countries in the Indo-Pacific, despite the region's strategic geo-political importance for Australia. Our findings draw into question the distribution of funding to universities and where investment might best be made for the highest potential return on research impact. Another key finding is that reported impacts across the domains of economy, society, culture, national security, public service, health, environment and quality of life offer little satisfactory evidence of impact, despite affording valuable insights into the nature of impact claimed. Accordingly, we conclude that to enhance the value of research and demonstrate impact in Education and Social Sciences, improved impact literacy is required among researchers. We assert that a better understanding of what constitutes impact and how it can be evidenced will support more impactful research designs. Wider adoption of the holistic anthropological definition of culture, which integrates values, practices and products, would enhance impact case studies by expanding their focus to include the broader cultural changes that underpin sustained social change. While the ARC engagement and impact agenda is a step in the right direction, improving the value of research for society will require a radical reconceptualisation of research and its funding, well beyond the current assessment framework. The Lowitja Institute's research-for-impact framework [1] is proposed as an alternative approach to research priority-setting based on explicit evidence gap analysis.

"We are not stray leaves blowing about in the wind": exploring the impact of Family Wellbeing empowerment research, 1998-2021.

BACKGROUND: An Aboriginal-developed empowerment and social and emotional wellbeing program, known as Family Wellbeing (FWB), has been found to strengthen the protective factors that help Indigenous Australians to deal with the legacy of colonisation and intergenerational trauma. This article reviews the research that has accompanied the implementation of the program, over a 23 year period. The aim is to assess the long-term impact of FWB research and identify the key enablers of research impact and the limitations of the impact assessment exercise. This will inform more comprehensive monitoring of research impact into the future. METHODS: To assess impact, the study took an implementation science approach, incorporating theory of change and service utilisation frameworks, to create a logic model underpinned by Indigenous research principles. A research impact narrative was developed based on mixed methods analysis of publicly available data on: 1) FWB program participation; 2) research program funding; 3) program outcome evaluation (nine studies); and 4) accounts of research utilisation (seven studies). RESULTS: Starting from a need for research on empowerment identified by research users, an investment of $2.3 million in research activities over 23 years produced a range of research outputs that evidenced social and emotional wellbeing benefits arising from participation in the FWB program. Accounts of research utilisation confirmed the role of research outputs in educating participants about the program, and thus, facilitating more demand (and funding acquisition) for FWB. Overall research contributed to 5,405 recorded participants accessing the intervention. The key enablers of research impact were; 1) the research was user- and community-driven; 2) a long-term mutually beneficial partnership between research users and researchers; 3) the creation of a body of knowledge that demonstrated the impact of the FWB intervention via different research methods; 4) the universality of the FWB approach which led to widespread application. CONCLUSIONS: The FWB research impact exercise reinforced the view that assessing research impact is best approached as a "wicked problem" for which there are no easy fixes. It requires flexible, open-ended, collaborative learning-by-doing approaches to build the evidence base over time. Steps and approaches that research groups might take to build the research impact knowledge base within their disciplines are discussed.

Assessing research impact: Australian Research Council criteria and the case of Family Wellbeing research.

Researchers worldwide are increasingly reporting the societal impact of their research as part of national research productivity assessments. However, the challenges they encounter in developing their impact case studies against specified government assessment criteria and how pitfalls can be mitigated are not reported. This paper examines the key steps taken to develop an Aboriginal Family Wellbeing (FWB) empowerment research impact case study in the context of an Australian Research Council (ARC) pilot research impact assessment exercise and the challenges involved in applying the ARC criteria. The requirement that researchers demonstrate how their institutions support them to conduct impactful research has the potential to create supportive environments for researchers to be more responsive to the needs of users outside academia. However, the 15-year reference period for the associated research underpinning the reported impact and the focus on researcher's current institutional affiliation constitute potential constraints to demonstrating the true impact of research. For researchers working with Indigenous people, relationships that build over long periods of time, irrespective of university affiliation, are critical to conducting impactful research. A more open-ended time-frame, with no institutional restrictions for the 'associated research' provides the best opportunity to demonstrate the true benefits of research not only for Indigenous people but for Australian society more broadly.

The characteristics and reporting quality of research impact case studies: A systematic review.

Despite the growing expectation that researchers report the impact of their research using a case study approach, systematic reviews of research impact have focused on frameworks, indicators, methods of data collection and assessment rather than impact case studies. Our aim is to provide an overview of the characteristics of published research impact case studies, including translation activities, and their reporting quality. We searched for peer-reviewed impact studies published between 2000 and 2018 using a case study approach and selected 25 suitable papers. We applied descriptive statistics to study characteristics, conducted thematic analysis of research translation activities and assessed reporting quality using the 10-point ISRIA statement. 24 papers reported intermediate impacts, such as advocacy, or the development of statements, tools, or technology. 4 reported on longer-term societal impacts, such as health outcomes and economic return on investment. 7 reported on translation activities. Papers scored well against the ISRIA statement on 5 domains of reporting quality. Weakest scores centred around identification of stakeholder needs and stakeholder involvement, and ethics and conflict of interest. We identified the need for more consistency in reporting through a case study approach, more systematic reporting of translation pathways and greater transparency concerning estimated costs and benefits of the research and its translation and impact assessment.

Evaluation of a Residential Mental Health Recovery Service in North Queensland.

BACKGROUND: Evidence shows that subacute mental health recovery occurs best when a person remains active within the community and fulfils meaningful and satisfying roles of their choosing. Several residential care services that incorporate these values have been established in Australia and overseas. AIMS: This study describes (a) the development of an evaluation framework for a new subacute residential mental health recovery service in regional Australia and (b) reports on the formative evaluation outcomes. METHODS: Continuous quality improvement and participatory research approaches informed all stages of the development of the evaluation framework. A program logic was established and subsequently tested for practicability. The resultant logic utilizes the Scottish Recovery Indicator 2 (SRI 2) service development tool, Individual Recovery Plans (IRPs), and the impact assessment of the service on psychiatric inpatient admissions (reported separately). RESULTS: Service strengths included a recovery-focused practice that identifies and addresses the basic needs of residents (consumers). The consumers of the service were encouraged to develop their own goals and self-manage their recovery plans. The staff of the service were identified as working effectively in the context of the recovery process; the staff were seen as supported and valued. Areas for improvement included more opportunities for self-management for residents and more feedback from residents and carers.

Family-centred interventions by primary healthcare services for Indigenous early childhood wellbeing in Australia, Canada, New Zealand and the United States: a systematic scoping review.

BACKGROUND: Primary healthcare services in Australia, Canada, New Zealand and the United States have embraced the concept of family-centred care as a promising approach to supporting and caring for the health of young Indigenous children and their families. This scoping review assesses the quality of the evidence base and identifies the published literature on family- centred interventions for Indigenous early childhood wellbeing. METHODS: Fourteen electronic databases, grey literature sources and the reference lists of Indigenous maternal and child health reviews were searched to identify relevant publications from 2000 to 2015. Studies were included if the intervention was: 1) focussed on Indigenous children aged from conception to 5 years from the abovementioned countries; 2) led by a primary healthcare service; 3) described or evaluated; and 4) scored greater than 50% against a validated scale for family-centredness. The study characteristics were extracted and quality rated. Reported aims, strategies, enablers and outcomes of family-centredcare were identified using grounded theory methods. RESULTS: Eighteen studies (reported in 25 publications) were included. Three were randomised controlled studies; most were qualitative and exploratory in design. More than half of the publications were published from 2012 to 2015. The overarching aim of interventions was to promote healthy families. Six key strategies were to: support family behaviours and self- care, increase maternal knowledge, strengthen links with the clinic, build the Indigenous workforce, promote cultural/ community connectedness and advocate for social determinants of health. Four enablers were: competent and compassionate program deliverers, flexibility of access, continuity and integration of healthcare, and culturally supportive care. Health outcomes were reported for Indigenous children (nutritional status; emotional/behavioural; and prevention of injury and illness); parents/caregivers (depression and substance abuse; and parenting knowledge, confidence and skills); health services (satisfaction; access, utilization and cost) and community/cultural revitalisation. DISCUSSION AND CONCLUSION: The evidence for family-centred interventions is in the early stages of development, but suggests promise for generating diverse healthcare outcomes for Indigenous children and their parents/caregivers, as well as satisfaction with and utilisation of healthcare, and community/cultural revitalisation. Further research pertaining to the role of fathers in family-centred care, and the effects and costs of interventions is needed.

The Complexity of Health Service Integration: A Review of Reviews.

BACKGROUND: The aim of health service integration is to provide a sustainable and integrated health system that better meets the needs of the end user. Yet, definitions of health service integration, methods for integrating health services, and expected outcomes are varied. This review was commissioned by Queensland Health, the government department responsible for health service delivery in Queensland, Australia, to inform efforts to integrate their mental health services. This review reports on the characteristics, reported outcomes, and design quality of studies included in systematic reviews of health service integration research. METHOD: The review was developed by systematically searching nine electronic databases to find peer-reviewed Australian and international systematic reviews with a focus on health service integration. Reviews were included if they were in the English language and published between 2000 and 2015. A standardized assessment tool was used to analyze the study design quality of included reviews. Data relating to the integration types, methods, and reported outcomes of integration were synthesized. RESULTS: Seventeen publications met the inclusion criteria. Eleven (65%) reviews were published during the past 5 years, which may indicate a trend for increased awareness of the need for service integration. The majority of reviews were published by researchers in the UK (8/47%), USA (3/18%), and Australia (3/18%). Included reviews focused on a variety of integration types, including integrated care pathways, governance models, integration of interventions, collaborative/integrated care models, and integration of different types of health care. Most (53%) of the reviews reported on the cost-effectiveness of service integration, e.g., positive results, no effect, or inconclusive. Only one of the reviews reported on the importance of consumer involvement. The overall design of 70% of the reviews was high, 18% medium, and 12% low. CONCLUSION: There is no "one size fits all" approach to health service integration. Instead, this literature review highlighted the complexity of service integration, which in most primary studies involved a range of strategies. Rigorous assessments of cost-effectiveness and reporting on consumer involvement are required in future research.

Systematic review of youth mental health service integration research.

Quality mental health care is based on the integration of care across organisations and disciplines. The aims of this study were, first, to assess the extent, characteristics and reported outcomes of publications concerned with youth mental health service integration in Australia and internationally; and second, to investigate the study design quality of evaluative interventions and determine whether the studies report on the cost-effectiveness of the integration in order to inform the reform of youth mental health services by Queensland Health. A systematic search of the peer-reviewed literature and a narrative synthesis were undertaken of English language publications from 21 electronic databases. Inclusion criteria were: published 1998-2014 (inclusive); peer-reviewed research; focused on mental health services integration; reported data for youth aged 12-25 years. The methodological quality of evaluative interventions was assessed using the Quality Assessment Tool for Quantitative Studies developed by the Effective Public Health Practice Project (EPHPP). Twenty-five studies met the inclusion criteria: one (4%) was classified as a measurement research, 13 (52%) as descriptive, and 11 (44%) as interventions including five (45%) evaluative interventions. Four out of the five evaluative interventions reported positive effects of youth mental health service integration. Particular problems included ambiguity of definitions, absence of economic or cost analyses and insufficient consumer involvement. The methodological quality of the interventions was variable with, on average, a moderate level of selection bias and study design. Despite a slight increase in the number of studies in the last couple of years, there are important gaps in the evidence base for youth mental health service integration processes. The relatively small number of evaluative studies and lack of economic evaluations point to the need for additional research in this important area.