Re-evaluating randomized clinical trials of psychological interventions: Impact of response shift on the interpretation of trial results.

BACKGROUND: Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. However, comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called 'response shift'. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients' frames of reference. AIMS: The overall aim is to gain insight into the general relevance of response shift for psychological health intervention research. Specifically, the aim is to re-analyse data of published randomized controlled trials (RCTs) investigating the effectiveness of psychological interventions targeting different health aspects, to assess (1) the occurrence of response shift, (2) the impact of response shift on interpretation of treatment effectiveness, and (3) the predictive role of clinical and background variables for detected response shift. METHOD: We re-analysed data from RCTs on guided internet delivered cognitive behavioural treatment (CBT) for insomnia in the general population with and without elevated depressive symptoms, an RCT on meaning-centred group psychotherapy targeting personal meaning for cancer survivors, and an RCT on internet-based CBT treatment for persons with diabetes with elevated depressive symptoms. Structural equation modelling was used to test the three objectives. RESULTS: We found indications of response shift in the intervention groups of all analysed datasets. However, results were mixed, as response shift was also indicated in some of the control groups, albeit to a lesser extent or in opposite direction. Overall, the detected response shifts only marginally impacted trial results. Relations with selected clinical and background variables helped the interpretation of detected effects and their possible mechanisms. CONCLUSION: This study showed that response shift effects can occur as a result of psychological health interventions. Response shift did not influence the overall interpretation of trial results, but provide insight into differential treatment effectiveness for specific symptoms and/or domains that can be clinically meaningful.

Using structural equation modeling to investigate change and response shift in patient-reported outcomes: practical considerations and recommendations.

BACKGROUND: Patient-reported outcomes (PROs) are of increasing importance for health-care evaluations. However, the interpretation of change in PROs may be obfuscated due to changes in the meaning of the self-evaluation, i.e., response shift. Structural equation modeling (SEM) is the most widely used statistical approach for the investigation of response shift. Yet, non-technical descriptions of SEM for response shift investigation are lacking. Moreover, application of SEM is not straightforward and requires sequential decision-making practices that have not received much attention in the literature. AIMS: To stimulate appropriate applications and interpretations of SEM for the investigation of response shift, the current paper aims to (1) provide an accessible description of the SEM operationalizations of change that are relevant for response shift investigation; (2) discuss practical considerations in applying SEM; and (3) provide guidelines and recommendations for researchers who want to use SEM for the investigation and interpretation of change and response shift in PROs. CONCLUSION: Appropriate applications and interpretations of SEM for the detection of response shift will help to improve our understanding of response shift phenomena and thus change in PROs. Better understanding of patients' perceived health trajectories will ultimately help to adopt more effective treatments and thus enhance patients' wellbeing.

The Analysis of Multivariate Longitudinal Data: An Instructive Application of the Longitudinal Three-Mode Model.

Structural equation modeling is a common technique to assess change in longitudinal designs. However, these models can become of unmanageable size with many measurement occasions. One solution is the imposition of Kronecker product restrictions to model the multivariate longitudinal structure of the data. The resulting longitudinal three-mode models (L3MMs) are very parsimonious and have attractive interpretation. This paper provides an instructive description of L3MMs. The models are applied to health-related quality of life (HRQL) data obtained from 682 patients with painful bone metastasis, with eight measurements at 13 occasions; before and every week after treatment with radiotherapy. We explain (1) how the imposition of Kronecker product restrictions can be used to model the multivariate longitudinal structure of the data, (2) how to interpret the Kronecker product restrictions and the resulting L3MM parameters, and (3) how to test substantive hypotheses in L3MMs. In addition, we discuss the challenges for the evaluation of (differences in) fit of these complex and parsimonious models. The L3MM restrictions lead to parsimonious models and provide insight in the change patterns of relationships between variables in addition to the general patterns of change. The L3MM thus provides a convenient model for multivariate longitudinal data, as it not only facilitates the analysis of complex longitudinal data but also the substantive interpretation of the dynamics of change.

Model Fit after Pairwise Maximum Likelihood.

Maximum likelihood factor analysis of discrete data within the structural equation modeling framework rests on the assumption that the observed discrete responses are manifestations of underlying continuous scores that are normally distributed. As maximizing the likelihood of multivariate response patterns is computationally very intensive, the sum of the log-likelihoods of the bivariate response patterns is maximized instead. Little is yet known about how to assess model fit when the analysis is based on such a pairwise maximum likelihood (PML) of two-way contingency tables. We propose new fit criteria for the PML method and conduct a simulation study to evaluate their performance in model selection. With large sample sizes (500 or more), PML performs as well the robust weighted least squares analysis of polychoric correlations.

Measurement bias detection through Bayesian factor analysis.

Measurement bias has been defined as a violation of measurement invariance. Potential violators-variables that possibly violate measurement invariance-can be investigated through restricted factor analysis (RFA). The purpose of the present paper is to investigate a Bayesian approach to estimate RFA models with interaction effects, in order to detect uniform and nonuniform measurement bias. Because modeling nonuniform bias requires an interaction term, it is more complicated than modeling uniform bias. The Bayesian approach seems especially suited for such complex models. In a simulation study we vary the type of bias (uniform, nonuniform), the type of violator (observed continuous, observed dichotomous, latent continuous), and the correlation between the trait and the violator (0.0, 0.5). For each condition, 100 sets of data are generated and analyzed. We examine the accuracy of the parameter estimates and the performance of two bias detection procedures, based on the DIC fit statistic, in Bayesian RFA. Results show that the accuracy of the estimated parameters is satisfactory. Bias detection rates are high in all conditions with an observed violator, and still satisfactory in all other conditions.

The effects of sleep extension and sleep hygiene advice on sleep and depressive symptoms in adolescents: a randomized controlled trial.

OBJECTIVE: Sleep problems are common and persistent during adolescence and can have negative effects on adolescents' mood. To date, studies that investigate the effects of sleep extension on adolescents' sleep and depressive symptoms are still lacking. This study aims to investigate the effects of gradual sleep extension combined with sleep hygiene advice in adolescents with chronic sleep reduction on objectively measured sleep, self-reported sleep problems and depressive symptoms. METHODS: Fifty-five adolescents with chronic sleep reduction (mean age: 15.44 years; 85.5% females) were included in the study. Participants were randomly assigned to either a sleep extension group (gradual sleep extension by advancing bedtimes in the evening and receiving sleep hygiene advice) or to a control group (no instruction). Sleep was measured with actigraphy during three weeks, the first week was the baseline week, and the last two weeks were the experimental weeks during which sleep was extended. Other outcome variables were self-reported sleep problems (daytime sleepiness, symptoms of insomnia and circadian rhythm sleep disorder) and depressive symptoms, which were assessed before and after the experimental manipulation. RESULTS: During the third week of the experiment, adolescents in the sleep extension group had earlier bedtimes, earlier sleep onsets, spent more time in bed and slept longer than adolescents in the control group. Their chronic sleep reduction, insomnia symptoms and depressive symptoms diminished significantly. In addition, there was a trend of improved circadian rhythm sleep disorder symptoms and sleep quality. CONCLUSION: Gradual sleep extension combined with sleep hygiene advice seems to have beneficial effects on sleep, self-reported sleep problems and depressive symptoms of adolescents with chronic sleep reduction. Although we cannot distinguish between the effects of sleep extension and sleep hygiene advice, the results suggest that advancing bedtimes can extend sleep and improve depressive symptoms.

The effects of sleep extension on sleep and cognitive performance in adolescents with chronic sleep reduction: an experimental study.

OBJECTIVE: To investigate the effects of gradual sleep extension in adolescents with chronic sleep reduction. Outcome variables were objectively measured sleep and cognitive performance. METHODS: Participants were randomly assigned to either a sleep extension group (gradual sleep extension by advancing bedtimes in the evening) or to a control group (no instruction). Our sample consisted of 55 adolescents (mean age, 15.44 y; 85.5% girls) with symptoms of chronic sleep reduction (loss of energy, shortness of sleep, sleepiness, and irritation). Sleep was monitored with actigraphy over 3 weeks; the first week was the baseline week and the last two weeks were the experimental weeks. Participants in the experimental group were instructed to extend their sleep during the week by gradually advancing their bedtimes by 5 minutes each night. Additionally participants were asked to prevent bedtime shifts on weekend nights. Cognitive performance was assessed before and after the experimental manipulation. RESULTS: During the last week of the experiment, adolescents in the sleep extension group had earlier bedtimes, earlier sleep onsets, spent more time in bed, and slept longer than adolescents in the control group. These results indicate that the experimental manipulation was successful and that adolescents in the experimental group fell asleep earlier and slept longer than adolescents in the control group. Furthermore some aspects of cognitive performance, especially visuospatial processing, significantly changed in the sleep extension group. CONCLUSION: Gradual sleep extension has beneficial effects on adolescents' sleep and is related to changes in some aspects of cognitive performance.

Quality of life and disease-specific functioning of patients with anorectal malformations or Hirschsprung's disease: a review.

The objective of this review was to examine disease-specific functioning (ie, faecal functioning and disease-related psychosocial problems) and quality of life (QoL)in patients with anorectal malformations (ARMs) or Hirschsprung's disease across different developmental stages. A search on PubMed yielded 22 eligible studies which were analysed. Patients with ARM or Hirschsprung's disease report slightly more QoL problems than comparison groups. As compared with adolescents, children reported better QoL but worse disease-specific functioning. Relationships between disease-specific functioning and QoL remain unclear. Therefore, to disentangle the complex relationship between the faecal functioning of these patients and their QoL through childhood, adolescence and adulthood, longitudinal studies should examine disease-specific functioning with validated QoL questionnaires that include age-specific versions.

Information gain in patients using a multimedia website with tailored information on anaesthesia.

BACKGROUND: The preoperative visit is an appropriate time to educate the patient on anaesthesia. The aim of this study was to determine if a website, as an information source for anaesthesia before the visit to the preoperative assessment clinic (PAC), increases patients' knowledge on anaesthesia. METHODS: A multimedia website was developed containing specific information about anaesthesia relevant to the patient. A questionnaire was developed to measure knowledge gain. Patients were divided into three groups: (i) those who read the existing brochure; (ii) those who looked at the new website; and (iii) a cluster of non-brochure and non-website users: those who did not read the brochure or website but had completed the questionnaire. An anaesthesiologist also informed all three groups during the preoperative visit at the PAC. RESULTS: Patients visiting the website had a higher educational level than others. A significant increase in knowledge was observed after using the website information compared with the other two groups (P<0.001). The group with higher education levels had higher knowledge gains, and the website independently contributed to the knowledge gain. CONCLUSIONS: A patient-tailored multimedia website is an effective way to support the information provided by the anaesthesiologist in order to inform patients about their upcoming anaesthetic procedure. The use of such a website gives a significant increase in knowledge compared with only spoken information, or spoken information combined with a brochure.

Problem behavior in children of chronically ill parents: a meta-analysis.

The aim of this meta-analysis is to examine whether children of chronically ill parents differ from norm groups in problem behavior. We report moderator effects and overall effect sizes for internalizing, externalizing and total problem behavior assessed by children and parents. In fixed effect models, we found a significant overall effect size for internalizing problem behavior (number of studies k = 19, total sample size N = 1,858, Cohen's d = .23, p < .01) and externalizing problem behavior (k = 13, N = 1,525, d = .09, p < .01) but not for total problem behavior (k = 7; N = 896). Effects for internalizing and externalizing problem behavior were larger in non-cancer studies, in samples including younger children and younger ill parents, in samples defined by low average SES and in studies including parents with longer illness duration. In addition, effects for externalizing problem behavior were larger in studies characterized by a higher percentage of ill mothers and single parents. With exclusive self-report, effect sizes were significant for all problem behaviors. Based on these results, a family-centered approach in health care is recommended.

School-aged children after the end of successful treatment of non-central nervous system cancer: longitudinal assessment of health-related quality of life, anxiety and coping.

The aim of the study was to investigate: (1) health-related quality of life (HRQoL) and anxiety in school-aged cancer survivors during the first 4 years of continuous remission after the end of treatment; and (2) correlations of disease-related coping with HRQoL and anxiety. A total of 76 survivors aged 8-15 years completed questionnaires about HRQoL, anxiety and disease-related cognitive coping at one to five measurement occasions. Their HRQoL was compared with norm data, 2 months (n = 49) and 1 year (n = 41), 2 years (n = 41), 3 years (n = 42) and 4 years (n = 27) after treatment. Through longitudinal mixed models analyses it was investigated to what extent disease-related cognitive coping was associated with HRQoL and anxiety over time, independent of the impact of demographic and medical variables. Survivors reported worse Motor Functioning (HRQoL) 2 months after the end of treatment, but from 1 year after treatment they did no longer differ from the norm population. Lower levels of anxiety were associated with male gender, being more optimistic about the further course of the disease (predictive control) and less searching for information about the disease (interpretative control). Stronger reliance on the physician (vicarious control) was associated with better mental HRQoL. As a group, survivors regained good HRQoL from 1 year after treatment. Monitoring and screening survivors are necessary to be able to trace the survivors at risk of worse HRQoL.

Structural equation modeling of health-related quality-of-life data illustrates the measurement and conceptual perspectives on response shift.

OBJECTIVE: To illustrate different perspectives on response shift with cancer patients' health-related quality-of-life (HRQL) data. In measurement perspective, the focus is on bias in the measurement of HRQL. In conceptual perspective, the focus is on bias in the explanation of HRQL. STUDY DESIGN AND SETTING: Data came from a consecutive series of 202 newly diagnosed cancer patients, heterogeneous to cancer site, all undergoing surgery. A HRQL questionnaire was administered before and after surgery. Using structural equation modeling, biases and response shifts in measurement and explanation of HRQL were investigated with respect to patient's cancer site, health status, sex, age, optimism, and social comparison. RESULTS: Six measurement biases were found, five of which were considered response shift. The "general health" (GH) scale appeared most susceptible to response shift. For example, GH scores were not fully determined by HRQL but also by optimism before surgery and female sex and downward social comparison after surgery. Additionally, two explanation biases were found, neither of which were considered response shift-before and after surgery the mental component of HRQL was not only affected by cancer site and health status but also by optimism and downward social comparison. CONCLUSION: Our approach enables the distinction and testing of biases and response shifts in the measurement and explanation of HRQL.

A predictive model of health-related quality of life in young adult survivors of childhood cancer.

A predictive model of health-related quality of life in young adult survivors of childhood cancerThis study aimed to examine factors that affect survivors' health-related quality of life (HRQoL), using a theoretical model in which demographic and medical characteristics explain HRQoL mediated by course of life, coping and social support. In a cross-sectional design, 353 survivors aged 18-30 years completed questionnaires. Structural equation modelling was performed to investigate the relationships among the variables in the model and to test whether the model fitted the data. The model fitted the data closely: chi(2)(14) = 21.61, P = 0.087; root mean square error of approximation = 0.039, 90% confidence interval [0.00; 0.070]. The effect of medical and demographic characteristics on HRQoL was mediated by coping. Survivors having been treated with both chemotherapy and radiotherapy were most at risk for worse HRQoL because they suffer more from current health complaints and were less inclined to predictive and active coping. Screening survivors medically as well as psychosocially could help to identify patients with the greatest needs and direct interventions by which the follow-up care could be improved.

Under what conditions do patients want to be informed about their risk of a complication? A vignette study.

BACKGROUND: Discussing treatment risks has become increasingly important in medical communication. Still, despite regulations, physicians must decide how much and what kind of information to present. OBJECTIVE: To investigate patients' preference for information about a small risk of a complication of colonoscopy, and whether medical and personal factors contribute to such preference. To propose a disclosure policy related to our results. DESIGN: Vignettes study. SETTING: Department of Gastroenterology, Academic Medical Centre, the Netherlands. PATIENTS: 810 consecutive colonoscopy patients. INTERVENTION: A home-sent questionnaire containing three vignettes. Vignettes varied in the indication for colonoscopy, complication severity and level of risk. Patients were invited to indicate their wish to be informed and the importance of such information. In addition, sociodemograhic, illness-related and psychological characteristics were assessed. MAIN OUTCOME MEASUREMENTS: Wish to be informed and importance of information. RESULTS: Of 810 questionnaires, 68% were returned. Patients generally wished to be informed about low-risk complications, regardless of the indication for colonoscopy or the severity of the complication. The level of risk did matter, though (OR = 2.48, SE = 0.28, p = 0.001). The information was considered less important if done for population screening purposes or diagnosis of colon cancer, if the complication was less severe (bleeding) and if the risk was smaller (0.01% and 0.1%). Patients' information preference was also related to age, mood and coping style. LIMITATIONS: Difficulty of vignettes. CONCLUSIONS: Patients generally wish to be informed about all possible risks. However, this might become uninformative. A stepwise approach is suggested.

Setting priorities for improving the preoperative assessment clinic: the patients' and the professionals' perspective.

BACKGROUND: The quality of the preoperative assessment clinic (PAC) is determined by many factors. Patients' experiences are important indicators, but often overlooked. We prepare to set priorities to improve the PAC by obtaining detailed patients' feedback on the quality of the PAC, and establishing the value patients and professionals attach to different care aspects, using the Patient Experiences with the Preoperative Assessment Clinic questionnaire. METHODS: The PAC's standard of service was determined for five care aspects (dimensions), using patients' feedback. The importance of a dimension to patients was determined by calculating the effects of the dimensions on patients' overall appraisal. In addition, professionals were asked to rate the importance of the different care aspects. RESULTS: Patients had the most positive experiences with the nurse, and the least positive experiences with waiting. However, waiting was least important to patients. When combining the PAC's standard of service with the value given to the dimensions by patients and professionals separately, we found in both instances that waiting was in greatest need of improvement. This was followed by reception, the anaesthetist, remaining experiences, and finally the nurse. CONCLUSIONS: Quality improvement of the PAC can be achieved by obtaining patients' feedback on the quality, determine a PAC's standard of service, recognize service areas that require improvement, and identify actions appropriate to bring about improvement. The value patients and professionals attach to different aspects of care can then be used to prioritize improvements.

Patient Experiences with the Preoperative Assessment Clinic (PEPAC): validation of an instrument to measure patient experiences.

BACKGROUND: Presently, no comprehensive and validated questionnaire to measure patient experiences of the preoperative assessment clinic (PAC) is available. We developed and validated the Patient Experiences with the Preoperative Assessment Clinic (PEPAC) questionnaire, which can be used for quantitative measurements of patient experiences of the PAC. METHODS: We adapted the National Health Service outpatient questionnaire, incorporating questions specific for anaesthesiology. To make the PEPAC appropriate for quantitative measurements, dimensions and single items suitable for statistical analysis were constructed. Each dimension consists of multiple items measuring the same aspect of care. Reliability was established by computing Cronbach's alpha coefficients. Construct validity was assessed by correlating the dimensions with the patient's overall appraisal (Pearson's r). These dimensions should explain a substantial level of variance of the patients' overall appraisal; therefore, regression analysis was performed. RESULTS: After a pilot phase, the questionnaire was sent to 700 consecutive patients (response 74%). Five scales measuring five dimensions of patient experiences were constructed. Cronbach's alpha ranged from 0.56 to 0.84, supporting reliability of the PEPAC. Correlations between the dimensions and patients' overall appraisal ranged from 0.22 to 0.56. Collectively, the five scales explained 51% of patients' overall appraisal. CONCLUSIONS: The PEPAC is a comprehensive, reliable, and validated questionnaire to measure patient experiences with the PAC. It might be a useful tool to identify the service areas of the PAC that require improvement and to determine which actions can bring about improvement.

A comparison of counselee and counselor satisfaction in reproductive genetic counseling.

Important insights in the process of genetic counseling can be provided by establishing levels of satisfaction. The aim of our study was to compare counselees' and counselors' satisfaction with the initial consultation in reproductive genetic counseling and to gain insight into the factors associated with their contentment. One hundred and fifty-one women and 11 counselors participated in this study. Pre-test questionnaires included counselees' socio-demographic, physical and psychological characteristics, i.e. their degree of worry, expectations, preferred participation in decision making and experienced degree of control. Post-visit questionnaires asked for counselees' and counselors' satisfaction, counselees' participation in decision making and counselees' Perceived Personal Control (PPC). Little difference was found between counselees' and counselors' overall visit-specific satisfaction (mean 79 vs 74, respectively, on a visual analogue scale from 0 to 100). The correlation between counselees' and counselors' satisfaction was medium sized (r = 0.26, p < 0.01). Counselees' satisfaction was positively associated with being pregnant and with their post-visit PPC. Counselors' satisfaction was positively associated with counselees' post-visit PPC. No other counselee and counselor related variables appeared to be associated with satisfaction, nor was the duration of the consultation. Our findings suggest that, although both groups were satisfied with the consultation, counselees and counselors do not always have equal perceptions of the consultation process and may form their evaluation in different ways. In the assessment of quality of care, evaluation of both counselees' and counselors' satisfaction deserves more attention.

Using standardized video cases for assessment of medical communication skills: reliability of an objective structured video examination by computer.

OBJECTIVE: Using standardized video cases in a computerized objective structured video examination (OSVE) aims to measure cognitive scripts underlying overt communication behavior by questions on knowledge, understanding and performance. In this study the reliability of the OSVE assessment is analyzed using the generalizability theory. METHODS: Third year undergraduate medical students from the Academic Medical Center of the University of Amsterdam answered short-essay questions on three video cases, respectively about history taking, breaking bad news, and decision making. Of 200 participants, 116 completed all three video cases. Students were assessed in three shifts, each using a set of parallel case editions. About half of all available exams were scored independently by two raters using a detailed rating manual derived from the other half. Analyzed were the reliability of the assessment, the inter-rater reliability, and interrelatedness of the three types of video cases and their parallel editions, by computing a generalizability coefficient G. RESULTS: The test score showed a normal distribution. The students performed relatively well on the history taking type of video cases, and relatively poor on decision making and did relatively poor on the understanding ('knows why/when') type of questions. The reliability of the assessment was acceptable (G = 0.66). It can be improved by including up to seven cases in the OSVE. The inter-rater reliability was very good (G = 0.93). The parallel editions of the video cases appeared to be more alike (G = 0.60) than the three case types (G = 0.47). DISCUSSION: The additional value of an OSVE is the differential picture that is obtained about covert cognitive scripts underlying overt communication behavior in different types of consultations, indicated by the differing levels of knowledge, understanding and performance. The validation of the OSVE score requires more research. CONCLUSION AND PRACTICE IMPLICATIONS: A computerized OSVE has been successfully applied with third year undergraduate medical students. The test score meets psychometric criteria, enabling a proper discrimination between adequately and poorly performing students. The high inter-rater reliability indicates that a single rater is permitted.

Summative assessment of medical students' communication skills and professional attitudes through observation in clinical practice.

UNLABELLED: To establish medical students' professional competence for the medical profession, we designed a standardized observation procedure and the Amsterdam Attitude and Communication Scale (AACS) with nine five-point scale items, for summative assessment of their communication skills and professional attitudes. This study examines the reliability of the AACS assessment in clinical practice. In the Academic Medical Centre, Amsterdam, The Netherlands, the performance of 442 fifth year clinical students was judged six times in two settings: behaviour in clinical practice was judged independently twice by a doctor and a nurse; one videotaped patient interview was judged independently by a doctor and by a psychologist. The final mark was obtained by averaging ratings across all six assessments. Raters were 88 doctors, 29 nurses, and three psychologists. MAIN OUTCOME MEASURES: Standard errors (SEs) for absolute judgements indicate measurement precision. Precision of AACS scores is considered sufficient with SEs smaller than 0.25. Multi-disciplinary assessment of students' clinical performance using the AACS is feasible and sufficiently precise (with an overall mean of 3.97 and standard deviation of 0.55, the absolute SE is 0.21). Judgements of behaviour in the clinic were more precise (SEs range from 0.11 to 0.16) than judgements of videotaped interviews (SEs are 0.25 and 0.29). The procedure is sufficiently precise if five or six assessments are combined.