R Statistics: survey and review of packages for the estimation of Rasch models.

Abstract: R Statistics is a comprehensive and widely-used suite of packages for statistical operations. From 27 R packages indexed with the word "Rasch", 11 packages capable of Rasch estimation and analysis are identified and critiqued. A commercial Rasch application is included for comparison. Three R data frames are used. A larger and a smaller 0/1 data frame are analyzed with the Dichotomous Rasch Model. A polytomous 0/1/2 data frame is analyzed with the Partial Credit Model. The R packages can all use the same data frame. They are easy to use and mostly fast, though their documentation is generally skimpy. Every package has obvious shortcomings, but the unique features of each package could make them all useful. For general Rasch estimation and fit analysis of dichotomous data, three packages stand out: eRm, TAM and autoRasch. Two packages stand out for polytomous data: TAM and autoRasch.

Predicting responses from Rasch measures.

There is a growing family of Rasch models for polytomous observations. Selecting a suitable model for an existing dataset, estimating its parameters and evaluating its fit is now routine. Problems arise when the model parameters are to be estimated from the current data, but used to predict future data. In particular, ambiguities in the nature of the current data, or overfit of the model to the current dataset, may mean that better fit to the current data may lead to worse fit to future data. The predictive power of several Rasch and Rasch-related models are discussed in the context of the Netflix Prize. Rasch-related models are proposed based on Singular Value Decomposition (SVD) and Boltzmann Machines.

Reliability of performance examinations: revisited.

This article discusses the reliability for performance examinations with respect to the reproducibility of candidate pass-fail decisions. The multi-facet Rasch model accounts for the difficulty of test forms (examiners + tasks + items) taken by each candidate, so that all candidates are measured against the same criterion. The examiners provide analytic ratings of candidate performance using a defined rating scale. The examination is standardized so that sufficient information is collected about the candidates, and facets of the examination are linked. When the examination data are properly collected, there are high levels of confidence in the reproducibility of candidate outcomes for a high percentage of candidates.

Rasch-derived latent trait measurement of outcomes: insightful use leads to precision case management and evidence-based practices in functional healthcare.

The use of Rasch-derived latent trait measurement of outcomes for persons with chronic disease and disablement evolved from other fields, particularly education. Person-metrics is the measurement of how much chronic disease and disablement affects an individual's daily activities physically, cognitively, and through vocational and social role participation. The ability of the Rasch model to assume that the probability of a given person/item interaction is governed by the difficulty of the item and the ability of the person is invaluable to disability measurement. The difference between raw scores and true measures is illustrated by an example of a patient whose physical difficulty is rated on rising from a wheelchair and walking 100m (known to be more difficult), and then walking an additional 200m. Though number ratings of 0-1-2 are assigned to these tasks, they are not equidistant, and only a true measure shows the actual levels of physical difficulty.

Local independence and residual covariance: a study of olympic figure skating ratings.

Rasch fit analysis has focused on tests of global fit and tests of the fit of individual parameter estimates. Critics have noted that slight, but pervasive, patterns of misfit to a Rasch model within the data may escape detection using these approaches. These patterns contradict the Rasch axiom of local independence, and so degrade measurement and may bias measures. Misfit to a Rasch model is captured in the observation residuals. Traces of pervasive, but faint, secondary dimensions within the observations may be identified using factor analytic techniques. To illustrate these techniques, the ratings awarded during the Pairs Figure Skating competition at the 2002 Winter Olympic Games are examined. The intention is to detect analytically the patterns of rater bias admitted publicly after the event. It is seen that the one-parameter-at-a-time fit statistics and differential item functioning approaches fail to detect the crucial misfit patterns. Factor analytic methods do. In fact, the competition was held in two stages. Factor analytic techniques already detect the rater bias after the first stage. This suggests that remedial rater retraining or other rater-related actions could be taken before the final ratings are collected.

Rasch analysis of rank-ordered data.

Theoretical and practical aspects of several methods for the construction of linear measures from rank-ordered data are presented. The final partial-rankings of 356 professional golfers participating in 47 stroke-play tournaments are used for illustration. The methods include decomposing the rankings into independent paired comparisons without ties, into dependent paired comparisons without ties and into independent paired comparisons with ties. A further method, which is easier to implement, entails modeling each tournament as a partial-credit item in which the rank of each golfer is treated as the observation of a category on a partial-credit rating scale. For the golf data, the partial-credit method yields measures with greater face validity than the paired comparison methods. The methods are implemented with the computer programs FACETS and WINSTEPS.

Optimizing rating scale category effectiveness.

Rating scales are employed as a means of extracting more information out of an item than would be obtained from a mere "yes/no", "right/wrong" or other dichotomy. But does this additional information increase measurement accuracy and precision? Eight guidelines are suggested to aid the analyst in optimizing the manner in which rating scales categories cooperate in order to improve the utility of the resultant measures. Though these guidelines are presented within the context of Rasch analysis, they reflect aspects of rating scale functioning which impact all methods of analysis. The guidelines feature rating-scale-based data such as category frequency, ordering, rating-to-measure inferential coherence, and the quality of the scale from measurement and statistical perspectives. The manner in which the guidelines prompt recategorization or reconceptualization of the rating scale is indicated. Utilization of the guidelines is illustrated through their application to two published data sets.

Rasch model estimation: further topics.

Building on Wright and Masters (1982), several Rasch estimation methods are briefly described, including Marginal Maximum Likelihood Estimation (MMLE) and minimum chi-square methods. General attributes of Rasch estimation algorithms are discussed, including the handling of missing data, precision and accuracy, estimate consistency, bias and symmetry. Reasons for, and the implications of, measure misestimation are explained, including the effect of loose convergence criteria, and failure of Newton-Raphson iteration to converge. Alternative parameterizations of rating scales broaden the scope of Rasch measurement methodology.

Construction of measures from many-facet data.

An extension to the Rasch model for fundamental measurement is described in which there is parameterization not only for examinee ability and item difficulty but also for judge severity. Variants of this model are discussed and judging plans reviewed. Its use and characteristics are explained by an application of the model to an empirical testing situation. A comparison with Generalizability Theory using a common data set is presented as a contrast in approaches to resolving judge indeterminacy.

Assessment of capacity for myoelectric control: evaluation of construct and rating scale.

OBJECTIVE: To examine the construct and rating scale of the Assessment of Capacity for Myoelectric Control, an assessment to evaluate ability in using a prosthetic hand. DESIGN: Cross-sectional study. SUBJECTS: Upper limb prosthesis users with different prosthetic levels/sides and prosthetic experience were included (n = 96). METHODS: Subjects' assessments with the Assessment of Capacity for Myoelectric Control were collected by 6 raters during their regular hospital visits. Rasch analysis was used, since it allowed an analysis of the data at the item and category levels. Dimension, item hierarchy and item fit statistics were used to examine the construct. Different Rasch parameters were used to examine rating scale structure and its use. RESULTS: The consistency of item difficulties with clinical knowledge and the unidimensionality confirmed that the construct is valid. Two items functioned unexpectedly (misfit), but the misfit was idiosyncratic to the sample, not systematic to the items. The 4-point rating scale usefully differentiated the subjects on the basis of their abilities. The use of category 2 was somewhat redundant. CONCLUSION: The Assessment of Capacity for Myoelectric Control is a valid assessment that evaluates ability in using a prosthetic hand. Revision of the category 2 definition would improve the functioning of the rating scale.

Scaling of the revised Oswestry low back pain questionnaire.

OBJECTIVE: To evaluate the measurement properties of the Revised Oswestry Disability Questionnaire (RODQ) by using rating scale analyses. DESIGN: Calibration of item responses by using a data set. SETTING: Four outpatient therapy clinics in the northeastern United States. PATIENTS: Convenience sample of 95 community-dwelling adults exhibiting mild to severe low back pain-related disability (LBP-D). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The RODQ. RESULTS: Several Rasch analyses were performed, with 1 item deleted and 2 response categories collapsed, creating a better test without increased error. A schema for item administration and evaluation was also developed. CONCLUSIONS: By using the abbreviated scale, suggested item order, and predicted responses, abbreviated versions of the instrument can be applied to measure LBP-D more efficiently.

The talking touchscreen: a new approach to outcomes assessment in low literacy.

PURPOSE: Cancer patients who are deficient in literacy skills are particularly vulnerable to experiencing different outcomes due to disparities in care or barriers to care. Outcomes measurement in low literacy patients may provide new insight into problems previously undetected due to the challenges of completing paper-and-pencil forms. DESCRIPTION OF STUDY: A multimedia program was developed to provide a quality of life assessment platform that would be acceptable to patients with varying literacy skills and computer experience. One item at a time is presented on the computer touchscreen, accompanied by a recorded reading of the question. Various colors, fonts and graphic images are used to enhance visibility, and a small picture icon appears near each text element allowing patients to replay the sound as many times as they wish. Evaluation questions are presented to assess patient burden and preferences. RESULTS: An ethnically diverse group of 126 cancer patients with a range of literacy skills and computer experience reported that the 'talking touchscreen' (TT) was easy to use, and commented on the usefulness of the multimedia approach. CLINICAL IMPLICATIONS: The TT is a practical, user-friendly data acquisition method that provides greater opportunities to measure self-reported outcomes in patients with a range of literacy skills.