Can complex health interventions be evaluated using routine clinical and administrative data? - a realist evaluation approach.

RATIONALE, AIMS AND OBJECTIVES: Interventions aimed at improving chronic care typically consist of multiple interconnected parts, all of which are essential to the effect of the intervention. Limited attention has been paid to the use of routine clinical and administrative data in the evolution of these complex interventions. The purpose of this study is to examine the feasibility of routinely collected data when evaluating complex interventions and to demonstrate how a theory-based, realist approach to evaluation may increase the feasibility of routine data. METHODS: We present a case study of evaluating a complex intervention, namely, the chronic care model (CCM), in Finnish primary health care. Issues typically faced when evaluating the effects of a complex intervention on health outcomes and resource use are identified by using routine data in a natural setting, and we apply context-mechanism-outcome (CMO) approach from the realist evaluation paradigm to improve the feasibility of using routine data in evaluating complex interventions. RESULTS: From an experimentalist approach that dominates the medical literature, routine data collected from a single centre offered a poor starting point for evaluating complex interventions. However, the CMO approach offered tools for identifying indicators needed to evaluate complex interventions. CONCLUSIONS: Applying the CMO approach can aid in a typical evaluation setting encountered by primary care managers: one in which the intervention is complex, the primary data source is routinely collected clinical and administrative data from a single centre, and in which randomization of patients into two research arms is too resource consuming to arrange.

The influence of selective participation in a physical activity intervention on the generalizability of findings.

OBJECTIVE: To evaluate factors that characterize employees who did not participate in a physical activity intervention in an occupational setting and assess how selective participation affects inferences from the data. METHODS: Employees were asked to complete a health risk appraisal. The respondents were invited to participate in a physical activity intervention. We compared predictors of sickness absence (register data) among all respondents and those who participated in the intervention, using Bayesian regression analysis. RESULTS: Of 1116 employees, 817 (73%) responded, of whom 544 (67%) participated in the intervention. Participants had better health behaviors and fewer health problems than those who did not participate. The predictors of sickness absence in all respondents differed from those who participated in the intervention. CONCLUSIONS: Selective participation may reduce the potential benefit of interventions and limit generalizability of findings.

The effectiveness of physical activity monitoring and distance counseling in an occupational setting - results from a randomized controlled trial (CoAct).

BACKGROUND: Lack of physical activity (PA) is a known risk factor for many health conditions. The workplace is a setting often used to promote activity and health. We investigated the effectiveness of an intervention on PA and productivity-related outcomes in an occupational setting. METHODS: We conducted a randomized controlled trial of 12 months duration with two 1:1 allocated parallel groups of insurance company employees. Eligibility criteria included permanent employment and absence of any condition that risked the participant's health during PA. Subjects in the intervention group monitored their daily PA with an accelerometer, set goals, had access to an online service to help them track their activity levels, and received counseling via telephone or web messages for 12 months. The control group received the results of a fitness test and an information leaflet on PA at the beginning of the study. The intervention's aim was to increase PA, improve work productivity, and decrease sickness absence. Primary outcomes were PA (measured as MET minutes per week), work productivity (quantity and quality of work; QQ index), and sickness absence (SA) days at 12 months. Participants were assigned to groups using block randomization with a computer-generated scheme. The study was not blinded. RESULTS: There were 544 randomized participants, of which 521 were included in the analysis (64% female, mean age 43 years). At 12 months, there was no significant difference in physical activity levels between the intervention group (n = 264) and the control group (n = 257). The adjusted mean difference was -206 MET min/week [95% Bayesian credible interval -540 to 128; negative values favor control group]. There was also no significant difference in the QQ index (-0.5 [-4.4 to 3.3]) or SA days (0.0 [-1.2 to 0.9]). Of secondary outcomes, body weight (0.5 kg [0.0 to 1.0]) and percentage of body fat (0.6% [0.2% to 1.1%]) were slightly higher in the intervention group. An exploratory subgroup analysis revealed no subgroups in which the intervention affected physical activity. No adverse events were reported. CONCLUSIONS: The intervention was not found effective, and this study does not provide support for the effectiveness of the workplace PA intervention used here. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT00994565.