Measuring teacher dispositions using the DAATS battery: a multifaceted Rasch analysis of rater effect.

The purpose of this study was to examine the extent to which raters' subjectivity impacts measures of teacher dispositions using the Dispositions Assessments Aligned with Teacher Standards (DAATS) battery. This is an important component of the collection of evidence of validity and reliability of inferences made using the scale. It also provides needed support for the use of subjective affective measures in teacher training and other professional preparation programs, since these measures are often feared to be unreliable because of rater effect. It demonstrates the advantages of using the Multi-Faceted Rasch Model as a better alternative to the typical methods used in preparation programs, such as Cohen's Kappa. DAATS instruments require subjective scoring using a six-point rating scale derived from the affective taxonomy as defined by Krathwohl, Bloom, and Masia (1956). Rater effect is a serious challenge and can worsen or drift over time. Errors in rater judgment can impact the accuracy of ratings, and these effects are common, but can be lessened through training of raters and monitoring of their efforts. This effort uses the multifaceted Rasch measurement models (MFRM) to detect and understand the nature of these effects.

Measuring teaching ability with the Rasch model by scaling a series of product and performance tasks.

Rasch measurement can provide a much needed solution to scaling teacher ability. Typically, decisions about teacher ability are based on dichotomously scored certification tests focused on knowledge of content or pedagogy. This paper presents early developmental work of a partial-credit teacher-ability scale of 42 tasks (performances and products) with 348 rated items or criteria. The tasks and criteria are aligned with national and state standards for expected teacher knowledge and skills. These tasks are being used in about two-thirds of Florida school districts and are spreading to colleges of education. Over time there will be many variations in both tasks and criteria, but here we focus on the initial system and the Rasch model as part of the plan for development of the system.

Measuring college sailing teams ability: an application of the many-facet Rasch model to ordinal data.

For those who look at typical approaches to sports ranking, sailing seems an almost impossible challenge, thereby making the evidence supporting Rasch measurement in this study even more intriguing. This article describes our application of MFRM and the results from our analysis of one year's data from the North American college sailing competitions. We discuss the following issues for inclusion in the development of a Rasch model useful to college sailing team ability: 1. The level of data quality (as described by Stevens, 1946) 2. The connectedness of the contests 3. Empty cells (missing data) 4. Interpreting outliers, trends, or unusual results 5. Judges' or polls' bias Our results indicate the utility and precision of MFRM as a tool generally appropriate for ordinal ranking applications and sailing ability specifically.