Introduction to the special issue on evaluator education.

After years in preparation, we are pleased to introduce this special issue of Evaluation and Program Planning that focuses on evaluator education, a topic that we believe is of considerable importance to the field's future. Before describing the issue's content, let us ground the articles in two ways: by examining the larger context within which evaluator education finds itself, and by briefly explaining who we are and describing how the issue came into being.

Cognitive complexity of evaluator competencies.

A critical aspect of evaluator education and professional learning is to educate evaluators who know the major evaluation models and learn how to manage relationships and solve complex problems when conducting, critiquing, developing and interpreting evaluations. The American and Australian Evaluation Associations have specified desired evaluator competencies, although developing a core curriculum for evaluation still seems elusive. It is suggested that these various competencies can be considered in terms of their levels of cognitive complexity. A model of cognitive complexity is utilised to explore the tasks and thinking of evaluators, leading to an important distinction between 'knowing that' and 'knowing how' in relation to evaluation tasks. As an illustration of this posited relationship, the Australian 'Evaluators Professional Learning Competencies' were coded according to their cognitive complexity. Two-thirds of these competencies were classed as 'knowing that' or surface thinking, and one third were classified as 'knowing how' or deeper thinking. A taxonomy is offered as a method to understand models of learning necessary for evaluator education and training, as well as for further development of professional evaluator competencies.

Can a teacher-led mindfulness intervention for new school entrants improve child outcomes? Protocol for a school cluster randomised controlled trial.

INTRODUCTION: The first years of school are critical in establishing a foundation for positive long-term academic, social and well-being outcomes. Mindfulness-based interventions may help students transition well into school, but few robust studies have been conducted in this age group. We aim to determine whether compared with controls, children who receive a mindfulness intervention within the first years of primary school have better: (1) immediate attention/short-term memory at 18 months post-randomisation (primary outcome); (2) inhibition, working memory and cognitive flexibility at 18 months post-randomisation; (3) socio-emotional well-being, emotion-regulation and mental health-related behaviours at 6 and 18 months post-randomisation; (4) sustained changes in teacher practice and classroom interactions at 18 months post-randomisation. Furthermore, we aim to determine whether the implementation predicts the efficacy of the intervention, and the cost effectiveness relative to outcomes. METHODS AND ANALYSIS: This cluster randomised controlled trial will be conducted in 22 primary schools in disadvantaged areas of Melbourne, Australia. 826 students in the first year of primary school will be recruited to detect between groups differences of Cohen's d=0.25 at the 18-month follow-up. Parent, teacher and child-assessment measures of child attention, emotion-regulation, executive functioning, socio-emotional well-being, mental health-related behaviour and learning, parent mental well-being, teacher well-being will be collected 6 and 18 months post-randomisation. Implementation factors will be measured throughout the study. Intention-to-treat analyses, accounting for clustering within schools and classes, will adopt a two-level random effects linear regression model to examine outcomes for the intervention versus control students. Unadjusted and analyses adjusted for baseline scores, baseline age, gender and family socioeconomic status will be conducted. ETHICS AND DISSEMINATION: Ethics approval has been received by the Human Research Ethics Committee at the University of Melbourne. Findings will be reported in peer-review publications, national and international conference presentations and research snapshots directly provided to participating schools and families. PRE-RESULTS TRIAL REGISTRATION NUMBER: Australian New Zealand Clinical Trials Registry (ACTRN12619000326190).

The current state of evaluator education: A situation analysis and call to action.

Education is critically important for the maintenance and growth of programme evaluation and its emergence as a profession. This position paper presents evaluator education as an initiative and provides a foundation for its evaluation. In it, the authors use Stufflebeam's Context, Input, Process, and Product (CIPP) model to frame a review of the literature and practice. We follow this analysis of the current state of evaluator education in formal settings, with discussion of its implications, and directions for future research and action. Our analysis suggests that there is much work to be done to understand the needs for evaluator education, to delineate standards for quality in both education and practice, to identify the inputs and processes most effective for addressing those needs, and to document its ultimate impacts. The paper provides a call to action for improving the quality, consistency, and integrity of this important work.

Testing of a synthetic phonics-based targeted reading intervention for students with reading difficulties in Year 1: protocol for an efficacy randomised controlled trial.

INTRODUCTION: Literacy is fundamental for educational achievement, and in the longer term contributes substantially to a range of life skills. Literacy difficulties during the early years of school are associated with long-term impacts on academic success, with differences in academic achievement sustained through children's schooling. Therefore, addressing literacy difficulties during the early years of school is essential in reducing the risk of children progressing onto negative academic, psychosocial and vocational trajectories. This trial will determine whether a phonics-based reading intervention can improve the reading comprehension of students identified as low-progress readers in the second year of primary school. METHODS/DESIGN: We recruited 236 students fromnine schools after screening for reading difficulties in the second year of primary school (Year 1). Schools in Sydney and Central Coast of New South Wales will be invited to participate via an opt-out consent process. All children identified as being in the bottom 25th percentile using the Wheldall Assessment of Reading Lists will be eligible for the trial. These children will be randomised into either 'usual teaching' or 'intervention' groups. Trained school support teachers will deliver the MiniLit intervention. Intervention: In groups of four, children will complete a daily 1-hour lesson with their MiniLit teacher over 20 school weeks. Follow-up: Immediately after intervention completion and 6 months later using child face-to-face assessments. Primary outcome: Reading comprehension at 6 months after intervention completion. The study will have an embedded process and cost-effectiveness evaluation. DISCUSSION: The Building Better Readers trial will be the first efficacy randomised controlled trial comparing usual teaching with a phonics-based reading intervention for children with reading difficulties in Year 1 of primary school in Australia. The randomised design will limit the effect of bias on outcomes seen in other studies. TRIAL REGISTRATION NUMBER: ACTRN12617000179336.

National variability in provision of health services for major long-term conditions in New Zealand (a report from the ABCC NZ study).

INTRODUCTION: Chronic illness is the leading cause of morbidity, mortality, and inequitable health outcomes in New Zealand. The ABCCNZ Stocktake aimed to identify extent of long-term conditions management evidence-based practices in stroke, cardiovascular disease, chronic obstructive pulmonary disease and congestive heart failure in New Zealand's District Health Boards (DHBs). METHODS: Eleven 'dimensions' of care for long-term conditions, identified by literature review and confirmed at workshops with long-term conditions professionals, formed the basis of the Stocktake of all 21 DHBs. It comprised two questionnaires: a generic component capturing perceptions of practice; and a disease-specific component assessing service provision. RESULTS: Fifteen DHBs completed all or parts of the questionnaires. Data accrual was completed in July 2008. Although most DHBs had developed long-term conditions management strategies to a moderate degree, there was considerable variability of practice between DHBs. DHBs thought their PHOs had developed strategies in some areas to a low to moderate level, though cardiovascular disease provision rated more highly. Regarding disease-specific services, larger DHBs had greater long-term conditions management provision not only of tertiary services, but of standard care, leadership, self-management, case-management, and audit. CONCLUSIONS: There is considerable variability in perceptions of long-term conditions management service provision across DHBs. In many instances variability in actual disease-specific service provision appears to relate to DHB size.

Health equity in the New Zealand health care system: a national survey.

INTRODUCTION: In all countries people experience different social circumstances that result in avoidable differences in health. In New Zealand, Maori, Pacific peoples, and those with lower socioeconomic status experience higher levels of chronic illness, which is the leading cause of mortality, morbidity and inequitable health outcomes. Whilst the health system can enable a fairer distribution of good health, limited national data is available to measure health equity. Therefore, we sought to find out whether health services in New Zealand were equitable by measuring the level of development of components of chronic care management systems across district health boards. Variation in provision by geography, condition or ethnicity can be interpreted as inequitable. METHODS: A national survey of district health boards (DHBs) was undertaken on macro approaches to chronic condition management with detail on cardiovascular disease, chronic obstructive pulmonary disease, congestive heart failure, stroke and diabetes. Additional data from expert informant interviews on program reach and the cultural needs of Maori and Pacific peoples was sought. Survey data were analyzed on dimensions of health equity relevant to strategic planning and program delivery. Results are presented as descriptive statistics and free text. Interviews were transcribed and NVivo 8 software supported a general inductive approach to identify common themes. RESULTS: Survey responses were received from the majority of DHBs (15/21), some PHOs (21/84) and 31 expert informants. Measuring, monitoring and targeting equity is not systematically undertaken. The Health Equity Assessment Tool is used in strategic planning but not in decisions about implementing or monitoring disease programs. Variable implementation of evidence-based practices in disease management and multiple funding streams made program implementation difficult. Equity for Maori is embedded in policy, this is not so for other ethnic groups or by geography. Populations that conventional practitioners find hard to reach, despite recognized needs, are often underserved. Nurses and community health workers carried a disproportionate burden of care. Cultural and diversity training is not a condition of employment. CONCLUSIONS: There is a struggle to put equity principles into practice, indicating will without enactment. Equity is not addressed systematically below strategic levels and equity does not shape funding decisions, program development, implementation and monitoring. Equity is not incentivized although examples of exceptional practice, driven by individuals, are evident across New Zealand.

Diabetes self-management education in South Auckland, New Zealand, 2007-2008.

INTRODUCTION: Self-management education programs seek to help patients realize that they are their own principal caregivers and that health care professionals are consultants who support them in this role. The aim of this study was to evaluate a diabetes self-management education program implemented as part of a district-wide approach in South Auckland, New Zealand, which has some of the highest prevalence rates for diabetes and is one of the most ethnically diverse and deprived regions of New Zealand. METHODS: Self-management attitudes and behaviors were monitored with the use of questionnaires before and after program implementation. Clinical outcomes such as hemoglobin A1c, body mass index, and blood pressure were also tracked before the program began and 3 months after the program ended. Participant focus groups and facilitator interviews were conducted to explore perceptions of the program. RESULTS: Participants showed improvement in attitudes toward their own ability to manage their diabetes; in diet, physical activity, and foot care; and in hemoglobin A1c levels 3 months after the end of participation. Participants also reduced their sense of isolation when dealing with their diabetes. However, catering to the needs of a multiethnic community is extremely resource-intensive because of the need to provide adequate language and cultural interpretation. CONCLUSION: Self-management education can work in multiethnic, high-needs communities in New Zealand. Programs must ensure they enable the appropriate mechanisms and have appropriate resources to support the community's needs.

Bringing health home: householder and provider perspectives on the healthy housing programme in Auckland, New Zealand.

This paper describes the Healthy Housing Programme, an ongoing intervention initiated for New Zealand public housing tenants in 2000 and presents findings from an evaluation conducted over three consecutive years. The Programme aims to improve well-being by addressing the housing circumstances of families at high risk of infectious diseases, experiencing high levels of deprivation, and living in areas with high concentrations of low-income, and largely public, housing. This is achieved through improving the housing stock and better integrating housing, health and social services. The evaluation was based on Brinkerhoff's Success Case Methodology and sought to address the question: 'how have providers and householders responded to an intervention that addresses the dynamism of the physical and social aspects of housing?' Members of 30 households were interviewed, along with all available Programme providers (n=19). Thematic analysis reveals that in the households evaluated the Programme promotes participation in housing decisions and, indirectly, neighbourhood life more generally. Benefits include a larger stock of social housing units appropriate to residents' needs, increased co-ordination between sectors and organisations, strengthened community networks through referrals to helping agencies, and heightened insight by government officials into the housing conditions of tenants. We argue that a programme originally seeking only to address specific health problems and risk factors has been strengthened as it has evolved to adopt a more holistic approach to promoting household well-being.