Protecting the integrity of survey research.

Although polling is not irredeemably broken, changes in technology and society create challenges that, if not addressed well, can threaten the quality of election polls and other important surveys on topics such as the economy. This essay describes some of these challenges and recommends remediations to protect the integrity of all kinds of survey research, including election polls. These 12 recommendations specify ways that survey researchers, and those who use polls and other public-oriented surveys, can increase the accuracy and trustworthiness of their data and analyses. Many of these recommendations align practice with the scientific norms of transparency, clarity, and self-correction. The transparency recommendations focus on improving disclosure of factors that affect the nature and quality of survey data. The clarity recommendations call for more precise use of terms such as "representative sample" and clear description of survey attributes that can affect accuracy. The recommendation about correcting the record urges the creation of a publicly available, professionally curated archive of identified technical problems and their remedies. The paper also calls for development of better benchmarks and for additional research on the effects of panel conditioning. Finally, the authors suggest ways to help people who want to use or learn from survey research understand the strengths and limitations of surveys and distinguish legitimate and problematic uses of these methods.

Defining welfare spells. Coping with problems of survey responses and administrative data.

The authors explore how to define a welfare spell and how well surveys measure welfare spells. By comparing survey and administrative data from the Work Pays Demonstration Project in California on the receipt of Aid to Families with Dependent Children (AFDC), they find that a substantial amount of administrative churning occurs in administrative data. Through a mixing model of several break lengths, the authors find that a single definition of a break in welfare is not applicable to all respondents. Additionally, it appears that there is substantial variation in the break lengths respondents utilize. Finally, the authors show that the complexity of defining an accurate break in spells creates difficulties for detecting biases in survey responses.