Measurement Matters: Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes.

There has been perennial interest in personal qualities other than cognitive ability that determine success, including self-control, grit, growth mindset, and many others. Attempts to measure such qualities for the purposes of educational policy and practice, however, are more recent. In this article, we identify serious challenges to doing so. We first address confusion over terminology, including the descriptor "non-cognitive." We conclude that debate over the optimal name for this broad category of personal qualities obscures substantial agreement about the specific attributes worth measuring. Next, we discuss advantages and limitations of different measures. In particular, we compare self-report questionnaires, teacher-report questionnaires, and performance tasks, using self-control as an illustrative case study to make the general point that each approach is imperfect in its own way. Finally, we discuss how each measure's imperfections can affect its suitability for program evaluation, accountability, individual diagnosis, and practice improvement. For example, we do not believe any available measure is suitable for between-school accountability judgments. In addition to urging caution among policymakers and practitioners, we highlight medium-term innovations that may make measures of these personal qualities more suitable for educational purposes.

The far-reaching effects of believing people can change: implicit theories of personality shape stress, health, and achievement during adolescence.

The belief that personality is fixed (an entity theory of personality) can give rise to negative reactions to social adversities. Three studies showed that when social adversity is common-at the transition to high school--an entity theory can affect overall stress, health, and achievement. Study 1 showed that an entity theory of personality, measured during the 1st month of 9th grade, predicted more negative immediate reactions to social adversity and, at the end of the year, greater stress, poorer health, and lower grades in school. Studies 2 and 3, both experiments, tested a brief intervention that taught a malleable (incremental) theory of personality--the belief that people can change. The incremental theory group showed less negative reactions to an immediate experience of social adversity and, 8 months later, reported lower overall stress and physical illness. They also achieved better academic performance over the year. Discussion centers on the power of targeted psychological interventions to effect far-reaching and long-term change by shifting interpretations of recurring adversities during developmental transitions.

Breaking the cycle of mistrust: wise interventions to provide critical feedback across the racial divide.

Three double-blind randomized field experiments examined the effects of a strategy to restore trust on minority adolescents' responses to critical feedback. In Studies 1 and 2, 7th-grade students received critical feedback from their teacher that, in the treatment condition, was designed to assuage mistrust by emphasizing the teacher's high standards and belief that the student was capable of meeting those standards--a strategy known as wise feedback. Wise feedback increased students' likelihood of submitting a revision of an essay (Study 1) and improved the quality of their final drafts (Study 2). Effects were generally stronger among African American students than among White students, and particularly strong among African Americans who felt more mistrusting of school. Indeed, among this latter group of students, the 2-year decline in trust evident in the control condition was, in the wise feedback condition, halted. Study 3, undertaken in a low-income public high school, used attributional retraining to teach students to attribute critical feedback in school to their teachers' high standards and belief in their potential. It raised African Americans' grades, reducing the achievement gap. Discussion centers on the roles of trust and recursive social processes in adolescent development.

An implicit theories of personality intervention reduces adolescent aggression in response to victimization and exclusion.

Adolescents are often resistant to interventions that reduce aggression in children. At the same time, they are developing stronger beliefs in the fixed nature of personal characteristics, particularly aggression. The present intervention addressed these beliefs. A randomized field experiment with a diverse sample of Grades 9 and 10 students (ages 14-16, n = 230) tested the impact of a 6-session intervention that taught an incremental theory (a belief in the potential for personal change). Compared to no-treatment and coping skills control groups, the incremental theory group behaved significantly less aggressively and more prosocially 1 month postintervention and exhibited fewer conduct problems 3 months postintervention. The incremental theory and the coping skills interventions also eliminated the association between peer victimization and depressive symptoms.

Does mentioning "some people" and "other people" in a survey question increase the accuracy of adolescents' self-reports?

A great deal of developmental research has relied on self-reports solicited using the "some/other" question format ("Some students think that… but other students think that…"). This article reports tests of the assumptions underlying its use: that it conveys to adolescents that socially undesirable attitudes, beliefs, or behaviors are not uncommon and legitimizes reporting them, yielding more valid self-reports than would be obtained by "direct" questions, which do not mention what other people think or do. A meta-analysis of 11 experiments embedded in four surveys of diverse samples of adolescents did not support the assumption that the some/other form increases validity. Although the some/other form led adolescents to think that undesirable attitudes, beliefs, or behaviors were more common and resulted in more reports of those attitudes and behaviors, answers to some/other questions were lower in criterion validity than were answers to direct questions. Because some/other questions take longer to ask and answer and require greater cognitive effort from participants (because they involve more words), and because they decrease measurement accuracy, the some/other question format seems best avoided.

The validity of self-reported nicotine product use in the 2001-2008 National Health and Nutrition Examination Survey.

BACKGROUND: Many researchers rely on high-quality face-to-face national surveys conducted by the federal government to estimate the prevalence of nicotine product use, but some scholars have suggested that adults' self-reports in such surveys are intentionally distorted by social desirability response bias, thus raising questions about the validity of those data. OBJECTIVES: To assess the validity of face-to-face survey self-reports by comparing them with physiological tests. RESEARCH DESIGN: Respondents in the National Health and Nutrition Examination Survey provided self-reports of nicotine product use and gave blood samples that were analyzed for levels of serum cotinine, an indicator of nicotine exposure. SUBJECTS: Nationally representative samples of thousands of American adults in National Health and Nutrition Examination Survey surveys conducted in 2001-2002, 2003-2004, 2005-2006, and 2007-2008. MEASURES: Serum cotinine levels and self-reports of nicotine product use. RESULTS: On average, only 1.17% to 1.25% of adult respondents said that they did not use a product containing nicotine, but had elevated cotinine levels. After eliminating the potential influence of passive smoking, these figures dropped to 0.89% to 0.94%. This small discrepancy between the 2 assessments could be due to measurement error in the cotinine test results or to recent use of cotinine-elevating medication. CONCLUSIONS: These data do not support the claim that a substantial number of adult respondents intentionally under-report nicotine consumption in face-to-face interviews. The remarkable accuracy of self-reports of nicotine consumption seen here justifies confidence in self-reports of this behavior in such surveys.