Proxy Survey Cost Indicators in Interviewer-Administered Surveys: Are they Actually Correlated with Costs?

Survey design decisions are-by their very nature-tradeoffs between costs and errors. However, measuring costs is often difficult. Furthermore, surveys are growing more complex. Many surveys require that cost information be available to make decisions during data collection. These complexities create new challenges for monitoring and understanding survey costs. Often, survey cost information lags behind reporting of paradata. Furthermore, in some situations, the measurement of costs at the case level is difficult. Given the time lag in reporting cost information and the difficulty of assigning costs directly to cases, survey designers and managers have frequently turned to proxy indicators for cost. These proxy measures are often based upon level-of-effort paradata. An example of such a proxy cost indicator is the number of attempts per interview. Unfortunately, little is known about how accurately these proxy indicators actually mirror the true costs of the survey. In this article, we examine a set of these proxy indicators across several surveys with different designs, including different modes of interview. We examine the strength of correlation between these indicators and two different measures of costs-the total project cost and total interviewer hours. This article provides some initial evidence about the quality of these proxies as surrogates for the true costs using data from several different surveys with interviewer-administered modes (telephone, face to face) across three organizations (University of Michigan's Survey Research Center, Westat, US Census Bureau). We find that some indicators (total attempts, total contacts, total completes, sample size) are correlated (average correlation ∼0.60) with total costs across several surveys. These same indicators are strongly correlated (average correlation ∼0.82) with total interviewer hours. For survey components, three indicators (total attempts, sample size, and total miles) are strongly correlated with both total costs (average correlation ∼0.77) and with total interviewer hours (average correlation ∼0.86).

Deriving Priors for Bayesian Prediction of Daily Response Propensity in Responsive Survey Design: Historical Data Analysis Versus Literature Review.

Responsive survey design (RSD) aims to increase the efficiency of survey data collection via live monitoring of paradata and the introduction of protocol changes when survey errors and increased costs seem imminent. Daily predictions of response propensity for all active sampled cases are among the most important quantities for live monitoring of data collection outcomes, making sound predictions of these propensities essential for the success of RSD. Because it relies on real-time updates of prior beliefs about key design quantities, such as predicted response propensities, RSD stands to benefit from Bayesian approaches. However, empirical evidence of the merits of these approaches is lacking in the literature, and the derivation of informative prior distributions is required for these approaches to be effective. In this paper, we evaluate the ability of two approaches to deriving prior distributions for the coefficients defining daily response propensity models to improve predictions of daily response propensity in a real data collection employing RSD. The first approach involves analyses of historical data from the same survey, and the second approach involves literature review. We find that Bayesian methods based on these two approaches result in higher-quality predictions of response propensity than more standard approaches ignoring prior information. This is especially true during the early-to-middle periods of data collection, when survey managers using RSD often consider interventions.

An experimental evaluation of a stopping rule aimed at maximizing cost-quality trade-offs in surveys.

Surveys face difficult choices in managing cost-error trade-offs. Stopping rules for surveys have been proposed as a method for managing these trade-offs. A stopping rule will limit effort on a select subset of cases to reduce costs with minimal harm to quality. Previously proposed stopping rules have focused on quality with an implicit assumption that all cases have the same cost. This assumption is unlikely to be true, particularly when some cases will require more effort and, therefore, more costs than others. We propose a new rule that looks at both predicted costs and quality. This rule is tested experimentally against another rule that focuses on stopping cases that are expected to be difficult to recruit. The experiment was conducted on the 2020 data collection of the Health and Retirement Study (HRS). We test both Bayesian and non-Bayesian (maximum-likelihood or ML) versions of the rule. The Bayesian version of the prediction models uses historical data to establish prior information. The Bayesian version led to higher-quality data for roughly the same cost, while the ML version led to small reductions in quality with larger reductions in cost compared to the control rule.

The anchoring method: Estimation of interviewer effects in the absence of interpenetrated sample assignment.

Methodological studies of the effects that human interviewers have on the quality of survey data have long been limited by a critical assumption: that interviewers in a given survey are assigned random subsets of the larger overall sample (also known as interpenetrated assignment). Absent this type of study design, estimates of interviewer effects on survey measures of interest may reflect differences between interviewers in the characteristics of their assigned sample members, rather than recruitment or measurement effects specifically introduced by the interviewers. Previous attempts to approximate interpenetrated assignment have typically used regression models to condition on factors that might be related to interviewer assignment. We introduce a new approach for overcoming this lack of interpenetrated assignment when estimating interviewer effects. This approach, which we refer to as the "anchoring" method, leverages correlations between observed variables that are unlikely to be affected by interviewers ("anchors") and variables that may be prone to interviewer effects to remove components of within-interviewer correlations that lack of interpenetrated assignment may introduce. We consider both frequentist and Bayesian approaches, where the latter can make use of information about interviewer effect variances in previous waves of a study, if available. We evaluate this new methodology empirically using a simulation study, and then illustrate its application using real survey data from the Behavioral Risk Factor Surveillance System (BRFSS), where interviewer IDs are provided on public-use data files. While our proposed method shares some of the limitations of the traditional approach - namely the need for variables associated with the outcome of interest that are also free of measurement error - it avoids the need for conditional inference and thus has improved inferential qualities when the focus is on marginal estimates, and it shows evidence of further reducing overestimation of larger interviewer effects relative to the traditional approach.

Expression and immunolocalization of aquaporins HC-1, -2, and -3 in Cope's gray treefrog, Hyla chrysoscelis.

We have previously identified two aquaporins (HC-1, HC-2) and a glyceroporin (HC-3), homologs, respectively, of mammalian AQP1, AQP2, and AQP3, from the freeze-tolerant treefrog Hyla chrysoscelis. The objective of the present study was to investigate by Western blotting and immunohistofluorescence the expression and localization of these proteins in warm-acclimated, hydrated treefrogs. We hypothesized that patterns of protein expression would reflect unique osmoregulatory roles for the three aquaporins. Western blots revealed a spectrum of protein bands from 28 kDa to 65+ kDa; treatment with N-glycosidase suggested that this reflected variable glycosidation of the aquaporins. HC-1 was expressed in all organs, including dermis of skin, sinusoids and septa of liver, Bowman's capsule of kidney, intestinal lacteal vessels, and perimysium and vasculature of muscle. HC-3 expression was also widespread, but with different localization, including epidermis and dermis of skin, renal collecting ducts, and colonic villous epithelium. HC-2 expression was limited to osmoregulatory organs (renal collecting ducts and epidermis). In many ways, the expression of these proteins paralleled their mammalian homologs. For example, HC-2 and HC-3 expression in collecting ducts appeared similar to the mammalian pattern (the former more apical, the latter more basal). However, some aspects of localization (e.g. HC-1 in Bowman's capsule) were unique, and the ubiquity of HC-3 expression may relate to its facilitation of glycerol transport in this animal that possesses glycerol-dependent freeze tolerance.

Comparing the Ability of Regression Modeling and Bayesian Additive Regression Trees to Predict Costs in a Responsive Survey Design Context.

Responsive survey designs rely upon incoming data from the field data collection to optimize cost and quality tradeoffs. In order to make these decisions in real-time, survey managers rely upon monitoring tools that generate proxy indicators for cost and quality. There is a developing literature on proxy indicators for the risk of nonresponse bias. However, there is very little research on proxy indicators for costs and almost none aimed at predicting costs under alternative design strategies. Predictions of survey costs and proxy error indicators can be used to optimize survey designs in real time. Using data from the National Survey of Family Growth, we evaluate alternative modeling strategies aimed at predicting survey costs (specifically, interviewer hours). The models include multilevel regression (with random interviewer effects) and Bayesian Additive Regression Trees (BART).

What Do You Think? Using Expert Opinion to Improve Predictions of Response Propensity Under a Bayesian Framework.

Responsive survey designs introduce protocol changes to survey operations based on accumulating paradata. Case-level predictions, including response propensity, can be used to tailor data collection features in pursuit of cost or quality goals. Unfortunately, predictions based only on partial data from the current round of data collection can be biased, leading to ineffective tailoring. Bayesian approaches can provide protection against this bias. Prior beliefs, which are generated from data external to the current survey implementation, contribute information that may be lacking from the partial current data. Those priors are then updated with the accumulating paradata. The elicitation of the prior beliefs, then, is an important characteristic of these approaches. While historical data for the same or a similar survey may be the most natural source for generating priors, eliciting prior beliefs from experienced survey managers may be a reasonable choice for new surveys, or when historical data are not available. Here, we fielded a questionnaire to survey managers, asking about expected attempt-level response rates for different subgroups of cases, and developed prior distributions for attempt-level response propensity model coefficients based on the mean and standard error of their responses. Then, using respondent data from a real survey, we compared the predictions of response propensity when the expert knowledge is incorporated into a prior to those based on a standard method that considers accumulating paradata only, as well as a method that incorporates historical survey data.

The effects of a progressive exercise program with surface electromyographic biofeedback on an adult with fecal incontinence.

BACKGROUND AND PURPOSE: Fecal incontinence often compromises a person's ability to participate in work and recreational activities. Incontinence may also diminish a person's willingness to take part in social events, leading to feelings of isolation. This case report describes physical therapy designed to reduce a patient's pelvic-floor muscle dysfunction and fecal incontinence. CASE DESCRIPTION: The patient was a 30-year-old woman whose fecal incontinence began after the complicated vaginal birth of her first child that required a vacuum extraction and episiotomy. Intervention included soft tissue techniques, electromyographic biofeedback, strength training, relaxation training, patient education, and a home program. The patient completed a questionnaire at initial evaluation and at discharge to assess her perceived limitations in functional activities. Electromyographic analysis was used to measure changes in the patient's pelvic-floor muscle control. OUTCOMES: The social, occupational, and sexual domains, which the patient initially judged to be the most compromised, showed the greatest improvement. Electromyographic data for the final treatment session indicated improved strength, endurance, and control of her pelvic-floor muscles. The patient reported no episodes of fecal incontinence over the last month of the 3 months of therapy. DISCUSSION: The physical therapy program may have led to improved bowel continence and greater control of the pelvic-floor muscles, resulting in greater confidence and comfort in social and work situations and less restriction in the patient's physical relationship with her spouse.