Process Evaluation of a Work Stress Prevention Approach in Primary Education: Exploring the Added Value of Real-Time Feedback During Implementation.

OBJECTIVE: Participatory organisational-level interventions carry a risk of implementation failure. The current study evaluates the implementation of a work stress prevention approach in primary education and reflects on the use of real-time feedback as implementation strategy to prevent this risk. METHODS: The process evaluation was conducted at four primary schools in the Netherlands. A framework for evaluating organisational-level interventions was applied using mixed methods. RESULTS: Results show the implementation level varied between schools and was hindered by the intervention context, school size, and planning of the approach. Management commitment and employee involvement appeared important factors for successful implementation. Real-time feedback appeared valuable to further improve implementation, but not to prevent implementation failure. CONCLUSIONS: Collecting data on implementation factors before the active phase of the approach, may provide the possibility to anticipate on implementation problems earlier.

Effects of a participatory work stress prevention approach for employees in primary education: results of a quasi-experimental study.

OBJECTIVE: Work stress is a serious problem for employees in primary education. This study evaluates the effects of a work stress prevention approach on emotional exhaustion and work stress determinants (job crafting behavior, quantitative and emotional demands, leadership, support, autonomy, team culture and feelings of competence), and the impact of implementation success (management commitment, employee involvement, communication during implementation) on these outcomes. METHODS: A quasi-experimental study was conducted with an intervention group (4 schools, N=102 employees) and a control group (26 schools, N=656 employees) using questionnaires at baseline (T0), one-year (T1) and two-year (T2) follow-up. Multilevel mixed model analyses were performed to test effects of condition and implementation success on changes in emotional exhaustion and work stress determinants between T0 and T2 in the intervention and control group. RESULTS: No effect were found for emotional exhaustion. Improvement of quality of leadership between T0 and T2 was significantly larger in the intervention compared to the control group. Additionally, implementation success was associated with a decrease in unnecessary demands and an increase in quality of leadership, team culture and job crafting behavior. CONCLUSIONS: This study shows no direct effect of the approach on emotional exhaustion, but it does show beneficial effects on quality of leadership. Additionally, results suggest that, when successfully implemented, the approach also has beneficial effects on other work stress determinants (ie, job crafting behavior, unnecessary demands and team culture). Results indicate that - if implemented successfully - the organizational-level intervention has the potential to improve the psychosocial work context.

Psychosocial work factors affecting mental health of young workers: a systematic review.

OBJECTIVE: For the general working population, robust evidence exists for associations between psychosocial work exposures and mental health. As this relationship is less clear for young workers, this systematic review aims at providing an overview of the evidence concerning psychosocial work factors affecting mental health of young workers. METHODS: The electronic databases used were PubMed, Web of Science, and PsycINFO and were last searched in October 2021. The eligible outcomes included depression-, stress-, burnout- and anxiety-related complaints, and fatigue, excluding clinical diagnoses and suicide-related outcomes. Only studies with workers aged 35 years or younger were included, which reported at least one association between a psychosocial work factor as exposure and a mental health complaint as outcome. Studies had to be in English, German or Dutch. Risk of bias was assessed using an instrument from the National Heart, Lung, and Blood Institute. Data synthesis was conducted using GRADE. RESULTS: In total 17 studies were included in this systematic review, including data from 35,600 young workers in total. Across these studies 86 exposure-outcome associations were reported. Nine exposure-outcome associations could be synthesised. The application of the GRADE framework led to one "low" assessment for the association between psychosocial job quality and mental health. The certainty of evidence for the other eight associations in the synthesis was very low. CONCLUSIONS: The current systematic review disclosed a high degree of uncertainty of the evidence due to conceptually fuzzy outcomes and exposures as well as large heterogeneity between studies.

Design of a Participatory Organizational-Level Work Stress Prevention Approach in Primary Education.

Background: Work stress is a serious problem in primary education. Decades of research underline the importance of participatory, organizational-level work stress prevention approaches. In this approach, measures are planned to tackle causes of work stress in a participatory manner and implemented by a working group consisting of members of the organization. This approach can only be effective if the measures contain effective ingredients to decrease work stress risks and are successfully implemented. The aim of this paper is to present an outline of a work stress prevention approach that is evaluated in primary education. To ensure the appropriateness of measures, a logic model of change is built as part of the risk assessment to facilitate the selection of appropriate measures. Progression on target behaviors as well as implementation factors are real-time monitored during implementation and fed back to the working groups, to provide the opportunity to adjust action plans when needed to optimize implementation. Methods: The approach consists of five steps: (1) preparation: installing an advisory board and working groups, (2) risk assessment: inventory of work stress risks (questionnaires and focus groups). In addition, a behavioral analysis is performed to build a logic model of change to facilitate selection of measures, (3) action planning: conducting an action plan with appropriate measures (focus groups), (4) implementation: implementing the action plan. During implementation progression on target behaviors and implementation factors are monthly monitored and fed back to the working groups, and (5) evaluation: effects of the approach are studied in a controlled trial with measurements at baseline (T0), 1 year (T1), and 2 years (T2) follow-up. A process evaluation is carried out using quantitative (questionnaires and real-time monitoring data) and qualitative (interviews and data logs) data to study the implementation process of all steps of the work stress approach. Discussion: We believe that building a logic model of change and real-time monitoring of implementation could be of added value to improve the success of the work stress prevention approach. With this study, we aim to provide more insights into work stress intervention research, especially in primary education. Clinical Trial Registration: The study is registered in Netherlands Trial Register (ClinicalTrials.gov #NL9797, October 18, 2021).

Use of Intervention Mapping for Occupational Risk Prevention and Health Promotion: A Systematic Review of Literature.

Aim: Intervention mapping (IM) is a method to systematically design interventions that is applied regularly within the public health domain. This study investigates whether IM is effectively used within the occupational safety and health domain as well. Specifically, this study explores the relation between the fidelity regarding the use of the IM protocol for intervention development, the implementation process and the effectiveness of the occupational risk prevention and health promotion interventions. Methods: A systematic review was conducted including articles on development, implementation, and effects of occupational risk prevention and health promotion interventions that were developed according to the IM-protocol. By means of a checklist, two authors reviewed the articles and rated them on several indicators regarding the fidelity of the IM-protocol, the implementation process, and the intervention effect. Results: A literature search resulted in a total of 12 interventions as described in 38 articles. The fidelity to the IM-protocol was relatively low for participation throughout the development process and implementation planning. No relationship was found between fidelity of the IM-protocol and the intervention effect. A theory-based approach (as one of the core elements of IM) appears to be positively related to a successful implementation process. Conclusion: Results of the review suggest that organizing a participative approach and implementation planning is difficult in practice. In addition, results imply that conducting matrices of change objectives as part of the intervention development, although challenging and time-consuming, may ultimately pay off, resulting in a tailored intervention that matches the target group.

Decreasing employees' work stress by a participatory, organizational level work stress prevention approach: a multiple-case study in primary education.

BACKGROUND: Work stress is an important problem among employees in education in the Netherlands. The present study aims to investigate the effects of a participatory organizational level work stress prevention approach to reduce (quantitative) job demands, increase resources (i.e. autonomy, supervisor and coworker support) and to reduce work stress and increase job satisfaction of employees in primary education. METHODS: This study makes use of a multiple case study research design. The stress prevention approach is implemented at 5 primary schools and questionnaires were filled out by 119 employees of the 5 schools at baseline and 1 year later, measuring job demands, resources, work stress, job satisfaction and implementation factors. RESULTS: Multilevel analyses showed a significant decrease in job demands and a significant increase in job satisfaction between baseline and follow up. In addition, employees that were more satisfied with the communication about the intervention showed more improvements in autonomy and job satisfaction. However, employees reporting an increased dialogue in work stress between employees and management showed a smaller decrease in job demands. CONCLUSION: The study shows a decrease in job demands and an increase in job satisfaction in the schools that implemented a stress prevention approach. Results of the study underline the importance of communication about the intervention as part of the implementation process, impacting the effectiveness of the intervention to improve autonomy and job satisfaction. TRIAL REGISTRATION: ISRCTN registry, study ID: ISRCTN14697835, registration date: 11-10-2019 (retrospectively registered).

Relations Between Psychosocial Job Characteristics and Work Ability in Employees with Chronic Headaches.

Purpose The aim of the study was to determine (a) to which extent job demands and job resources predict work ability in employees with chronic headaches, and (b) whether work ability in these employees is more hampered by high demands and more enhanced by resources than in employees without chronic disease. Methods All employees with chronic headaches (n = 593) and without chronic disease (n = 13,742) were selected from The Netherlands Working Conditions Survey conducted in 2013. This survey assessed amongst others job characteristics and various indicators of work ability, i.e. sick leave, employability, work engagement, and emotional exhaustion. Hierarchical regression analyses were conducted for employees with chronic headaches and compared to employees without chronic disease, controlling for age, gender and educational level. Results In employees with chronic headaches higher quantitative and emotional demands contributed to higher emotional exhaustion, and higher emotional demands to higher sick leave. Higher cognitive demands were however associated with higher work engagement. Higher autonomy was related to higher employability and lower emotional exhaustion. Higher supervisor and colleague support was associated with higher employability, higher engagement and lower emotional exhaustion. Higher supervisor support was associated with lower sick leave. Supervisor support emerged as a stronger predictor for emotional exhaustion in the employees with chronic headaches than in the employees without chronic disease. Conclusions Job demands and job resources are important for work ability in employees with chronic headaches. Furthermore, results suggest that these employees benefit more strongly from supervisor support than employees without chronic disease.

The effect of an organizational level participatory intervention in secondary vocational education on work-related health outcomes: results of a controlled trial.

BACKGROUND: Work-related stress is highly prevalent in the educational sector. The aim of the current study was to evaluate the effectiveness of an organizational level, participatory intervention on need for recovery and vitality in educational workers. It was hypothesized that the intervention would decrease need for recovery and increase vitality. METHODS: A quasi-experiment was conducted at two secondary Vocational Education and Training schools (N = 356) with 12- and 24-months follow-up measurements. The intervention consisted of 1) a needs assessment phase, wherein staff and teachers developed actions for happy and healthy working under supervision of a facilitator, and 2) an implementation phase, wherein these actions were implemented by the management teams. Mixed model analysis was applied in order to assess the differences between the intervention and control group on average over time. All analyses were corrected for baseline values and several covariates. RESULTS: No effects of the intervention were found on need for recovery, vitality and most of the secondary outcomes. Two small, statistically significant effects were in unfavorable direction: the intervention group scored on average over time significantly lower on absorption (i.e. a subscale of work engagement) and organizational efficacy than the control group. CONCLUSIONS: Since no beneficial effects of this intervention were found on the primary and most of the secondary outcomes, further implementation of the intervention in its current form is not eligible. We recommend that future organizational level interventions for occupational health 1) incorporate an elaborate implementation strategy, 2) are more specific in relating actions to stressors in the context, and 3) are integrated with secondary preventive, individual focused stress management interventions. TRIAL REGISTRATION: Netherlands Trial Register NTR3284 (date registered: February 14 2012).

Evaluating the implementation process of a participatory organizational level occupational health intervention in schools.

BACKGROUND: The importance of process evaluations in examining how and why interventions are (un) successful is increasingly recognized. Process evaluations mainly studied the implementation process and the quality of the implementation (fidelity). However, in adopting this approach for participatory organizational level occupational health interventions, important aspects such as context and participants perceptions are missing. Our objective was to systematically describe the implementation process of a participatory organizational level occupational health intervention aimed at reducing work stress and increasing vitality in two schools by applying a framework that covers aspects of the intervention and its implementation as well as the context and participants perceptions. METHODS: A program theory was developed, describing the requirements for successful implementation. Each requirement was operationalized by making use of the framework, covering: initiation, communication, participation, fidelity, reach, communication, satisfaction, management support, targeting, delivery, exposure, culture, conditions, readiness for change and perceptions. The requirements were assessed by quantitative and qualitative data, collected at 12 and 24 months after baseline in both schools (questionnaire and interviews) or continuously (logbooks). RESULTS: The intervention consisted of a needs assessment phase and a phase of implementing intervention activities. The needs assessment phase was implemented successfully in school A, but not in school B where participation and readiness for change were insufficient. In the second phase, several intervention activities were implemented at school A, whereas this was only partly the case in school B (delivery). In both schools, however, participants felt not involved in the choice of intervention activities (targeting, participation, support), resulting in a negative perception of and only partial exposure to the intervention activities. Conditions, culture and events hindered the implementation of intervention activities in both schools. CONCLUSIONS: The framework helped us to understand why the implementation process was not successful. It is therefore considered of added value for the evaluation of implementation processes in participatory organizational level interventions, foremost because of the context and mental models dimensions. However, less demanding methods for doing detailed process evaluations need to be developed. This can only be done if we know more about the most important process components and this study contributes to that knowledge base. TRIAL REGISTRATION: Netherlands Trial Register NTR3284 .

Process variables in organizational stress management intervention evaluation research: a systematic review.

OBJECTIVES: This systematic review aimed to explore which process variables are used in stress management intervention (SMI) evaluation research. METHODS: A systematic review was conducted using seven electronic databases. Studies were included if they reported on an SMI aimed at primary or secondary stress prevention, were directed at paid employees, and reported process data. Two independent researchers checked all records and selected the articles for inclusion. Nielsen and Randall's model for process evaluation was used to cluster the process variables. The three main clusters were context, intervention, and mental models. RESULTS: In the 44 articles included, 47 process variables were found, clustered into three main categories: context (two variables), intervention (31 variables), and mental models (14 variables). Half of the articles contained no reference to process evaluation literature. The collection of process evaluation data mostly took place after the intervention and at the level of the employee. CONCLUSIONS: The findings suggest that there is great heterogeneity in methods and process variables used in process evaluations of SMI. This, together with the lack of use of a standardized framework for evaluation, hinders the advancement of process evaluation theory development.

Evaluation of occupational health interventions using a randomized controlled trial: challenges and alternative research designs.

Occupational health researchers regularly conduct evaluative intervention research for which a randomized controlled trial (RCT) may not be the most appropriate design (eg, effects of policy measures, organizational interventions on work schedules). This article demonstrates the appropriateness of alternative designs for the evaluation of occupational health interventions, which permit causal inferences, formulated along two study design approaches: experimental (stepped-wedge) and observational (propensity scores, instrumental variables, multiple baseline design, interrupted time series, difference-in-difference, and regression discontinuity). For each design, the unique characteristics are presented including the advantages and disadvantages compared to the RCT, illustrated by empirical examples in occupational health. This overview shows that several appropriate alternatives for the RCT design are feasible and available, which may provide sufficiently strong evidence to guide decisions on implementation of interventions in workplaces. Researchers are encouraged to continue exploring these designs and thus contribute to evidence-based occupational health.

Design of the Bottom-up Innovation project--a participatory, primary preventive, organizational level intervention on work-related stress and well-being for workers in Dutch vocational education.

BACKGROUND: In the educational sector job demands have intensified, while job resources remained the same. A prolonged disbalance between demands and resources contributes to lowered vitality and heightened need for recovery, eventually resulting in burnout, sickness absence and retention problems. Until now stress management interventions in education focused mostly on strengthening the individual capacity to cope with stress, instead of altering the sources of stress at work at the organizational level. These interventions have been only partly effective in influencing burnout and well-being. Therefore, the "Bottom-up Innovation" project tests a two-phased participatory, primary preventive organizational level intervention (i.e. a participatory action approach) that targets and engages all workers in the primary process of schools. It is hypothesized that participating in the project results in increased occupational self-efficacy and organizational efficacy. The central research question: is an organization focused stress management intervention based on participatory action effective in reducing the need for recovery and enhancing vitality in school employees in comparison to business as usual? METHODS/DESIGN: The study is designed as a controlled trial with mixed methods and three measurement moments: baseline (quantitative measures), six months and 18 months (quantitative and qualitative measures). At first follow-up short term effects of taking part in the needs assessment (phase 1) will be determined. At second follow-up the long term effects of taking part in the needs assessment will be determined as well as the effects of implemented tailored workplace solutions (phase 2). A process evaluation based on quantitative and qualitative data will shed light on whether, how and why the intervention (does not) work(s). DISCUSSION: "Bottom-up Innovation" is a combined effort of the educational sector, intervention providers and researchers. Results will provide insight into (1) the relation between participating in the intervention and occupational and organizational self-efficacy, (2) how an improved balance between job demands and job resources might affect need for recovery and vitality, in the short and long term, from an organizational perspective, and (3) success and fail factors for implementation of an organizational intervention. TRIAL REGISTRATION NUMBER: Netherlands Trial Register NTR3284.