How strong should my anchor be for estimating group and individual level meaningful change? A simulation study assessing anchor correlation strength and the impact of sample size, distribution of change scores and methodology on establishing a true meaningful change threshold.

PURPOSE: Treatment benefit as assessed using clinical outcome assessments (COAs), is a key endpoint in many clinical trials at both the individual and group level. Anchor-based methods can aid interpretation of COA change scores beyond statistical significance, and help derive a meaningful change threshold (MCT). However, evidence-based guidance on the selection of appropriately related anchors is lacking. METHODS: A simulation was conducted which varied sample size, change score variability and anchor correlation strength to assess the impact of these variables on recovering the simulated MCT for interpreting individual and group-level results. To assess MCTs derived at the individual-level (i.e. responder definitions; RDs), Receiver Operating Characteristic (ROC) curves and Predictive Modelling (PM) analyses were conducted. To assess MCTs for interpreting change at the group-level, the mean change method was conducted. RESULTS: Sample sizes, change score variability and magnitude of anchor correlation affected accuracy of the estimated MCT. For individual-level RDs, ROC curves were less accurate than PM methods at recovering the true MCT. For both methods, smaller samples led to higher variability in the returned MCT, but higher variability still using ROC. Anchors with weaker correlations with COA change scores had increased variability in the estimated MCT. An anchor correlation of around 0.50-0.60 identified a true MCT cut-point under certain conditions using ROC. However, anchor correlations as low as 0.30 were appropriate when using PM under certain conditions. For interpreting group-level results, the MCT derived using the mean change method was consistently underestimated regardless of the anchor correlation. CONCLUSION: Sample size and change score variability influence the necessary anchor correlation strength when recovering individual-level RDs. Often, this needs to be higher than the commonly accepted threshold of 0.30. Stronger correlations than 0.30 are required when using the mean change method. Results can assist researchers selecting and assessing the quality of anchors.

How do the number of missing daily diary days impact the psychometric properties and meaningful change thresholds arising from a weekly average summary score?

PURPOSE: Quality of life research often collects daily information and averages this over a week, producing a summary score. When data are missing, arbitrary rules (such as requiring at least 4/7 observations) are used to determine whether a patient's summary score is created or set to missing. This simulation work aimed to assess the impact of missing data on the estimates produced by summary scores, the psychometric properties of the resulting summary score estimates and the impact on interpretation thresholds. METHODS: Complete longitudinal data were simulated for 1000 samples of 400 patients with different day-to-day variability. Data were deleted from these samples in line with missingness mechanisms to create scenarios with up to six days of missing data. Summary scores were created for complete and missing data scenarios. Summary score estimates, psychometric properties and meaningful change estimates were assessed for missing data scenarios compared to complete data. RESULTS: In most cases, the 4/7 day rule was supported, but this depended on daily variability. Fewer days of data were sometimes acceptable, but this was also dependent on the proportion of patients with missing data. Tables and figures allow researchers to assess the potential impact of missing data in their own studies. CONCLUSIONS: This work suggests that the missing data rule used to create summary scores impacts on the estimate, measurement properties and interpretation thresholds. Although a general rule of 4/7 days is supported, the way the summary score is derived does not have a uniform impact across psychometric analyses. Recommendations are to use the 4/7 rule, but plan for sensitivity analyses with other missing data rules.

Psychometric validation of the Ostomy Skin Tool 2.0.

Background: Peristomal skin complications (PSCs) pose a major challenge for people living with an ostomy. To avoid severe PSCs, it is important that people with an ostomy check their peristomal skin condition on a regular basis and seek professional help when needed. Aim: To validate a new ostomy skin tool (OST 2.0) that will make regular assessment of the peristomal skin easier. Methods: Seventy subjects participating in a clinical trial were eligible for the analysis and data used for the validation. Item-level correlation with anchors, inter-item correlations, convergent validity of domains, test-retest reliability, anchor- and distribution-based methods for assessment of meaningful change were all part of the psychometric validation of the tool. Results: A final tool was established including six patient reported outcome items and automatic assessment of the discolored peristomal area. Follow-up with cognitive debriefing interviews assured that the concepts were considered relevant for people with an ostomy. Conclusion: The OST 2.0 demonstrated evidence supporting its reliability and validity as an outcome measure to capture both visible and non-visible peristomal skin complications.