Establishing severity levels for patient-reported measures of functional communication, participation, and perceived cognitive function for adults with acquired cognitive and language disorders.

PURPOSE: To empirically assign severity levels (e.g., mild, moderate) to four relatively new patient-reported outcome measures (PROMs) for adults with acquired cognitive/language disorders. They include the Communicative Participation Item Bank, the Aphasia Communication Outcome Measure, and Neuro-QoL's item banks of Cognitive Function (v2.0) and Ability to Participate in Social Roles and Activities (v1.0). METHOD: We conducted 17 focus groups that comprised 22 adults with an acquired cognitive/language disorder from stroke, Parkinson's disease, or traumatic brain injury; 30 care partners of an adult with an acquired cognitive/language disorder; and 42 speech-language pathologists who had experience assessing/treating individuals with those and other cognitive/language disorders. In a small, moderated focus-group format, participants completed "PROM-bookmarking" procedures: They discussed hypothetical vignettes based on PROM item responses about people with cognitive/language disorders and had to reach consensus regarding whether their symptoms/function should be categorized as within normal limits or mild, moderate, or severe challenges. RESULTS: There was generally good agreement among the stakeholder groups about how to classify vignettes, particularly when they reflected very high or low functioning. People with aphasia described a larger range of functional communication challenges as "mild" compared to other stakeholder types. Based on a consensus across groups, we present severity levels for specific score ranges for each PROM. CONCLUSION: Standardized, stakeholder-informed severity levels that aid interpretation of PROM scores can help clinicians and researchers derive better clinical meaning from those scores, for example, by identifying important clinical windows of opportunity and assessing when symptoms have returned to a "normal" range.

Applying a Bookmarking Approach to Setting Clinically Relevant Interpretive Standards for the Spinal Cord Injury-Functional Index/Capacity Basic Mobility and Self-Care Item Bank Scores.

OBJECTIVES: To develop clinically relevant interpretive standards for the Spinal Cord Injury-Functional Index/Capacity (SCI-FI/C) Basic Mobility and Self-Care item bank scores. DESIGN: Modified "bookmarking" standard-setting methodology, including 2 stakeholder consensus meetings with individuals with spinal cord injury (SCI) and SCI clinicians, respectively, and a final, combined (consumers and clinicians) "convergence" meeting. SETTING: Two SCI Model System centers in the United States. PARTICIPANTS: Fourteen adults who work with individuals with traumatic SCI and 14 clinicians who work with individuals with SCI. MAIN OUTCOME MEASURES: Placement of bookmarks between vignettes based on SCI-FI Basic Mobility and Self-Care T scores. Bookmarks were placed between vignettes representing "No Problems," "Mild Problems," "Moderate Problems," and "Severe Problems" for each item bank. RESULTS: Each consensus group resulted in a single set of scoring cut points for the SCI-FI/C Basic Mobility and Self-Care item banks. The cut points were similar but not identical between the consumer and clinician groups, necessitating a final convergence meeting. For SCI-FI/C Basic Mobility, the convergence group agreed on cut scores of 61.25 (no problems/mild problems), 51.25 (mild problems/moderate problems), and 41.25 (moderate problems/severe problems). For SCI-FI/C Self-Care, the convergence group agreed on cut scores of 56.25 (no/mild), 51.25 (mild/moderate), and 38.75 (moderate/severe). CONCLUSIONS: The results of this study provide straightforward interpretive guidelines for SCI researchers and clinicians using the SCI-FI/C Basic Mobility and Self-Care instruments. These results are appropriate for the full bank, computer adaptive test, and short-form versions of the SCI-FI/C Basic Mobility and Self-Care item banks.

Differential item functioning in the Patient Reported Outcomes Measurement Information System Pediatric Short Forms in a sample of children and adolescents with cerebral palsy.

AIM: The present study examined the Patient Reported Outcomes Measurement Information System (PROMIS) Mobility, Fatigue, and Pain Interference Short Forms (SFs) in children and adolescents with cerebral palsy (CP) for the presence of differential item functioning (DIF) relative to the original calibration sample. METHOD: Using the Graded Response Model we compared item parameter estimates generated from a sample of 303 children and adolescents with CP (175 males, 128 females; mean age 15y 5mo) to parameter estimates from the PROMIS calibration sample, which served as the reference group. DIF was assessed in a two-step process using the item response theory-likelihood ratio-differential item functioning detection procedure. RESULTS: Significant DIF was identified for four of eight items in the PROMIS Mobility SF, for two of eight items in the Pain Interference Scale, and for one item out of 10 on the Fatigue Scale. Impact of DIF on total score estimation was notable for Mobility and Pain Interference, but not for Fatigue. INTERPRETATION: Results suggest differences in the responses of adolescents with CP to some items on the PROMIS Mobility and Pain Interference SFs. Cognitive interviews about the PROMIS items with adolescents with varying degrees of mobility limitations would provide better understanding of how they are interpreting and selecting responses to the PROMIS items and thus help guide selection of the most appropriate way to address this issue.

Development of a survey to assess adolescent perceptions of teen parenting.

Initiatives designed to prevent teen pregnancy are often based on adult perceptions of the negative aspects of a teen birth. Qualitative research has revealed that teens may perceive positive rewards associated with teen parenting. These perceptions have not yet been examined through survey research. The theory of reasoned action proposes that individuals assess the costs and rewards prior to engaging in a behavior and provides a framework for the development of a survey instrument designed to measure adolescent thoughts about the costs and rewards of the teen parenting experience. This manuscript describes the development and testing of a quantitative survey instrument designed to measure adolescents' perceptions. Pretesting, piloting, exploratory factor analysis, and a variety of reliability and validity measures were used to determine the value of the measure. The thoughts on teen parenting survey (TTPS) demonstrates an alpha level of .90. The TTPS yields a cumulative score of teen perceptions about the impact of a teen birth during the adolescent years that may be used to assess youth beliefs, correlated with demographic data, used to identify teens at risk for pregnancy/parenting, or provide a pretest/posttest to assess the effectiveness of interventions designed to foster realistic attitudes toward teen parenting.

Adapting a Patient-Reported Outcome Bookmarking Task to be Accessible to Adults With Cognitive and Language Disorders.

Background Patient-reported outcome (PRO) measures produce scores that do not always have obvious clinical meaning. The PRO-bookmarking procedure is a new and promising way to make PRO measures more meaningful and interpretable. However, the materials and procedures of the task may benefit from adaptations to be more accessible to individuals with cognitive and language disorders. Aims This study aims to provide an overview of the iterative refinement process used to modify the materials and procedures of the PRO-bookmarking task so that they are more accessible to adults with acquired cognitive and language impairments. Method and Procedures Our team of health psychologists, neuropsychologists, and speech-language pathologists (SLPs) conducted two focus groups with SLPs and care partners of people with aphasia using the same PRO-bookmarking materials and procedures as previous reports. These PRO-bookmarking materials and procedures were then refined iteratively based on discussion with those who participated in focus groups and among the research team, and three more times in the course of 16 additional focus groups of different stakeholders: people with Parkinson's disease, aphasia, or traumatic brain injury; care partners of people with those conditions; and SLPs who have experience with those, and other adult-acquired conditions. Outcomes and Results The PRO-bookmarking materials and procedures underwent four iterations to make them clearer, simpler, and more accessible. For example, the materials included more structured text and graphic supports where appropriate and the procedures were clustered into smaller discrete tasks and displayed graphically when possible and appropriate. Conclusions PRO-bookmarking materials and procedures were made simpler and more structured to increase their accessibility to adults with cognitive and language impairments. In fact, these adaptations made the tasks simpler and clearer for all types of stakeholders.

Comparing the Bayesian Unknown Change-Point Model and Simulation Modeling Analysis to Analyze Single Case Experimental Designs.

Recently, there has been an increased interest in developing statistical methodologies for analyzing single case experimental design (SCED) data to supplement visual analysis. Some of these are simulation-driven such as Bayesian methods because Bayesian methods can compensate for small sample sizes, which is a main challenge of SCEDs. Two simulation-driven approaches: Bayesian unknown change-point model (BUCP) and simulation modeling analysis (SMA) were compared in the present study for three real datasets that exhibit "clear" immediacy, "unclear" immediacy, and delayed effects. Although SMA estimates can be used to answer some aspects of functional relationship between the independent and the outcome variables, they cannot address immediacy or provide an effect size estimate that considers autocorrelation as required by the What Works Clearinghouse (WWC) Standards. BUCP overcomes these drawbacks of SMA. In final analysis, it is recommended that both visual and statistical analyses be conducted for a thorough analysis of SCEDs.

How Normal Are "Normal" Errors of Language and Cognition?

Purpose The aim of this study was to estimate the frequency and severity of common errors and inefficiencies of language and cognition in the general population. Method Item-response theory parameters were analyzed from the Neuro-QOL Cognitive Function Item Bank v2.0, which references a large normative sample that mirrors the demographics of the U.S. Census. Item parameters were analyzed to reveal the most likely responses to items about errors and inefficiencies of language and cognition. These most likely responses were estimated for each of 25 levels of theta to reveal normal and gradations of abnormal experiences of errors and inefficiencies. Results A typical experience with language and cognitive errors and inefficiencies (e.g., producing a T score between 40 and 60) is to have "a little" difficulty with each task that was assessed or to experience nearly every error/inefficiency at least once a week. Word-finding effort was particularly ubiquitous, as was the experience of walking into a room and forgetting one's intention. Conclusions It is typical to experience errors and inefficiencies of language and cognition. This analysis describes a typical experience for an average person in a highly representative sample. However, more work is needed in order to learn what is typical for more specific subgroups, for example, based on age.

Identifying an appropriate measurement modeling approach for the Mini-Mental State Examination.

The Mini-Mental State Examination (MMSE) is a 30-item, dichotomously scored test of general cognition. A number of benefits could be gained by modeling the MMSE in an item response theory (IRT) framework, as opposed to the currently used classical additive approach. However, the test, which is built from groups of items related to separate cognitive subdomains, may violate a key assumption of IRT: local item independence. This study aimed to identify the most appropriate measurement model for the MMSE: a unidimensional IRT model, a testlet response theory model, or a bifactor model. Local dependence analysis using nationally representative data showed a meaningful violation of the local item independence assumption, indicating multidimensionality. In addition, the testlet and bifactor models displayed superior fit indices over a unidimensional IRT model. Statistical comparisons showed that the bifactor model fit MMSE respondent data significantly better than the other models considered. These results suggest that application of a traditional unidimensional IRT model is inappropriate in this context. Instead, a bifactor model is suggested for future modeling of MMSE data as it more accurately represents the multidimensional nature of the scale. (PsycINFO Database Record

Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes.

This study investigated the impact of three prior distributions: matched, standard vague, and hierarchical in Bayesian estimation parameter recovery in two and one parameter models. Two Bayesian estimation methods were utilized: Markov chain Monte Carlo (MCMC) and the relatively new, Variational Bayesian (VB). Conditional (CML) and Marginal Maximum Likelihood (MML) estimates were used as baseline methods for comparison. Vague priors produced large errors or convergence issues and are not recommended. For both MCMC and VB, the hierarchical and matched priors showed the lowest root mean squared errors (RMSEs) for ability estimates; RMSEs of difficulty estimates were similar across estimation methods. For the standard errors (SEs), MCMC-hierarchical displayed the largest values across most conditions. SEs from the VB estimation were among the lowest in all but one case. Overall, VB-hierarchical, VB-matched, and MCMC-matched performed best. VB with hierarchical priors are recommended in terms of their accuracy, and cost and (subsequently) time effectiveness.

Alternative Hypothesis Testing Procedures for DIMTEST.

Many commonly used item response models make the unidimensionality assumption of a single latent trait underlying the response data. The validity of this assumption needs to be tested before these models can be applied. One option is to use Stout's non-parametric hypothesis test of essential unidimensionality, which is operationalized in the DIMTEST procedure. Although generally successful, Type I error rates of this procedure are usually deflated in small samples and inflated in large samples, while power can be low in small samples. A possible cause for the unfavorable Type I error rates and power may be that estimates of the sampling distribution, bias, and standard error of the test statistic are not sufficiently accurate in finite samples. In this study, five alternative hypothesis testing procedures were formulated that replace the (asymptotically correct) approximations in the current DIMTEST procedure with computational alternatives. The performance of these procedures was investigated in two simulation studies. One of these alternative procedures, which uses a conditional covariance statistic directly in a bootstrap hypothesis test, exhibited better controlled Type I errors and higher power than the current DIMTEST procedure in most conditions. Averaged over all sample sizes and correlations between two underlying dimensions, power increased by 5 percentage points for simple structure and by 7 percentage points for approximate simple structure.