Effects of double-dose algebra on college persistence and degree attainment.

In 2003, Chicago Public Schools introduced double-dose algebra, requiring two periods of math-one period of algebra and one of algebra support-for incoming ninth graders with eighth-grade math scores below the national median. Using a regression discontinuity design, earlier studies showed promising results from the program: For median-skill students, double-dose algebra improved algebra test scores, pass rates, high school graduation rates, and college enrollment. This study follows the same students 12 y later. Our findings show that, for median-skill students in the 2003 cohort, double-dose significantly increased semesters of college attended and college degree attainment. These results were not replicated for the 2004 cohort. Importantly, the impact of the policy on median-skill students depended largely on how classes were organized. In 2003, the impacts on college persistence and degree attainment were large in schools that strongly adhered to the cut-score-based course assignment, but without grouping median-skill students with lower-skill peers. Few schools implemented the policy in such a way in 2004.

Longitudinally adaptive assessment and instruction increase numerical skills of preschool children.

Social inequality in mathematical skill is apparent at kindergarten entry and persists during elementary school. To level the playing field, we trained teachers to assess children's numerical and spatial skills every 10 wk. Each assessment provided teachers with information about a child's growth trajectory on each skill, information designed to help them evaluate their students' progress, reflect on past instruction, and strategize for the next phase of instruction. A key constraint is that teachers have limited time to assess individual students. To maximize the information provided by an assessment, we adapted the difficulty of each assessment based on each child's age and accumulated evidence about the child's skills. Children in classrooms of 24 trained teachers scored 0.29 SD higher on numerical skills at posttest than children in 25 randomly assigned control classrooms (P = 0.005). We observed no effect on spatial skills. The intervention also positively influenced children's verbal comprehension skills (0.28 SD higher at posttest, P < 0.001), but did not affect their print-literacy skills. We consider the potential contribution of this approach, in combination with similar regimes of assessment and instruction in elementary schools, to the reduction of social inequality in numerical skill and discuss possible explanations for the absence of an effect on spatial skills.

Effects of Time-Varying Parent Input on Children's Language Outcomes Differ for Vocabulary and Syntax.

Early linguistic input is a powerful predictor of children's language outcomes. We investigated two novel questions about this relationship: Does the impact of language input vary over time, and does the impact of time-varying language input on child outcomes differ for vocabulary and for syntax? Using methods from epidemiology to account for baseline and time-varying confounding, we predicted 64 children's outcomes on standardized tests of vocabulary and syntax in kindergarten from their parents' vocabulary and syntax input when the children were 14 and 30 months old. For vocabulary, children whose parents provided diverse input earlier as well as later in development were predicted to have the highest outcomes. For syntax, children whose parents' input substantially increased in syntactic complexity over time were predicted to have the highest outcomes. The optimal sequence of parents' linguistic input for supporting children's language acquisition thus varies for vocabulary and for syntax.

The pace of vocabulary growth helps predict later vocabulary skill.

Children vary widely in the rate at which they acquire words--some start slow and speed up, others start fast and continue at a steady pace. Do early developmental variations of this sort help predict vocabulary skill just prior to kindergarten entry? This longitudinal study starts by examining important predictors (socioeconomic status [SES], parent input, child gesture) of vocabulary growth between 14 and 46 months (n = 62) and then uses growth estimates to predict children's vocabulary at 54 months. Velocity and acceleration in vocabulary development at 30 months predicted later vocabulary, particularly for children from low-SES backgrounds. Understanding the pace of early vocabulary growth thus improves our ability to predict school readiness and may help identify children at risk for starting behind.

Confidence Bounds and Power for the Reliability of Observational Measures on the Quality of a Social Setting.

Social scientists are frequently interested in assessing the qualities of social settings such as classrooms, schools, neighborhoods, or day care centers. The most common procedure requires observers to rate social interactions within these settings on multiple items and then to combine the item responses to obtain a summary measure of setting quality. A key aspect of the quality of such a summary measure is its reliability. In this paper we derive a confidence interval for reliability, a test for the hypothesis that the reliability meets a minimum standard, and the power of this test against alternative hypotheses. Next, we consider the problem of using data from a preliminary field study of the measurement procedure to inform the design of a later study that will test substantive hypotheses about the correlates of setting quality. The preliminary study is typically called the "generalizability study" or "G study" while the later, substantive study is called the "decision study" or "D study." We show how to use data from the G study to estimate reliability, a confidence interval for the reliability, and the power of tests for the reliability of measurement produced under alternative designs for the D study. We conclude with a discussion of sample size requirements for G studies.

Durable effects of concentrated disadvantage on verbal ability among African-American children.

Disparities in verbal ability, a major predictor of later life outcomes, have generated widespread debate, but few studies have been able to isolate neighborhood-level causes in a developmentally and ecologically appropriate way. This study presents longitudinal evidence from a large-scale study of >2,000 children ages 6-12 living in Chicago, along with their caretakers, who were followed wherever they moved in the U.S. for up to 7 years. African-American children are exposed in such disproportionate numbers to concentrated disadvantage that white and Latino children cannot be reliably compared, calling into question traditional research strategies assuming common points of overlap in ecological risk. We therefore focus on trajectories of verbal ability among African-American children, extending recently developed counterfactual methods for time-varying causes and outcomes to adjust for a wide range of predictors of selection into and out of neighborhoods. The results indicate that living in a severely disadvantaged neighborhood reduces the later verbal ability of black children on average by approximately 4 points, a magnitude that rivals missing a year or more of schooling.

Measurement equivalence and differential item functioning in family psychology.

Several hypotheses in family psychology involve comparisons of sociocultural groups. Yet the potential for cross-cultural inequivalence in widely used psychological measurement instruments threatens the validity of inferences about group differences. Methods for dealing with these issues have been developed via the framework of item response theory. These methods deal with an important type of measurement inequivalence, called differential item functioning (DIF). The authors introduce DIF analytic methods, linking them to a well-established framework for conceptualizing cross-cultural measurement equivalence in psychology (C.H. Hui and H.C. Triandis, 1985). They illustrate the use of DIF methods using data from the Project on Human Development in Chicago Neighborhoods (PHDCN). Focusing on the Caregiver Warmth and Environmental Organization scales from the PHDCN's adaptation of the Home Observation for Measurement of the Environment Inventory, the authors obtain results that exemplify the range of outcomes that may result when these methods are applied to psychological measurement instruments.

New evidence about language and cognitive development based on a longitudinal study: hypotheses for intervention.

We review findings from a four-year longitudinal study of language learning conducted on two samples: a sample of typically developing children whose parents vary substantially in socioeconomic status, and a sample of children with pre- or perinatal brain injury. This design enables us to study language development across a wide range of language learning environments and a wide range of language learners. We videotaped samples of children's and parents' speech and gestures during spontaneous interactions at home every four months, and then we transcribed and coded the tapes. We focused on two behaviors known to vary across individuals and environments-child gesture and parent speech-behaviors that have the potential to index, and perhaps even play a role in creating, differences across children in linguistic and other cognitive skills. Our observations have led to four hypotheses that have promise for the development of diagnostic tools and interventions to enhance language and cognitive development and brain plasticity after neonatal injury. One kind of hypothesis involves tools that could identify children who may be at risk for later language deficits. The other involves interventions that have the potential to promote language development. We present our four hypotheses as a summary of the findings from our study because there is scientific evidence behind them and because this evidence has the potential to be put to practical use in improving education.

Efficient analysis of Q-level nested hierarchical general linear models given ignorable missing data.

This article extends single-level missing data methods to efficient estimation of a Q-level nested hierarchical general linear model given ignorable missing data with a general missing pattern at any of the Q levels. The key idea is to reexpress a desired hierarchical model as the joint distribution of all variables including the outcome that are subject to missingness, conditional on all of the covariates that are completely observed and to estimate the joint model under normal theory. The unconstrained joint model, however, identifies extraneous parameters that are not of interest in subsequent analysis of the hierarchical model and that rapidly multiply as the number of levels, the number of variables subject to missingness, and the number of random coefficients grow. Therefore, the joint model may be extremely high dimensional and difficult to estimate well unless constraints are imposed to avoid the proliferation of extraneous covariance components at each level. Furthermore, the over-identified hierarchical model may produce considerably biased inferences. The challenge is to represent the constraints within the framework of the Q-level model in a way that is uniform without regard to Q; in a way that facilitates efficient computation for any number of Q levels; and also in a way that produces unbiased and efficient analysis of the hierarchical model. Our approach yields Q-step recursive estimation and imputation procedures whose qth-step computation involves only level-q data given higher-level computation components. We illustrate the approach with a study of the growth in body mass index analyzing a national sample of elementary school children.

Using a hierarchical model to estimate risk-adjusted mortality for hospitals not included in the reference sample.

OBJECTIVE: To provide a method for any hospital to evaluate patient mortality using a hierarchical risk-adjustment equation derived from a reference sample. DATA SOURCE: American College of Surgeons National Trauma Data Bank (NTDB). STUDY DESIGN: Hierarchical logistic regression models predicting mortality were estimated from NTDB data. Risk-adjusted hospital effects obtained directly from models using standard software were compared with approximations derived from a summary equation and data from each individual hospital. PRINCIPAL FINDINGS: Theoretical approximations were similar to results using standard software. CONCLUSIONS: To allow independent verification, agencies using reference databases for hospital mortality "report cards" should publish their risk-adjustment equations. Similar hospitals not in the reference database may also use the published equations along with the approximations described to evaluate their own outcomes using their own data.

Just-identified versus overidentified two-level hierarchical linear models with missing data.

The development of model-based methods for incomplete data has been a seminal contribution to statistical practice. Under the assumption of ignorable missingness, one estimates the joint distribution of the complete data for thetainTheta from the incomplete or observed data y(obs). Many interesting models involve one-to-one transformations of theta. For example, with y(i) approximately N(mu, Sigma) for i= 1, ... , n and theta= (mu, Sigma), an ordinary least squares (OLS) regression model is a one-to-one transformation of theta. Inferences based on such a transformation are equivalent to inferences based on OLS using data multiply imputed from f(y(mis) | y(obs), theta) for missing y(mis). Thus, identification of theta from y(obs) is equivalent to identification of the regression model. In this article, we consider a model for two-level data with continuous outcomes where the observations within each cluster are dependent. The parameters of the hierarchical linear model (HLM) of interest, however, lie in a subspace of Theta in general. This identification of the joint distribution overidentifies the HLM. We show how to characterize the joint distribution so that its parameters are a one-to-one transformation of the parameters of the HLM. This leads to efficient estimation of the HLM from incomplete data using either the transformation method or the method of multiple imputation. The approach allows outcomes and covariates to be missing at either of the two levels, and the HLM of interest can involve the regression of any subset of variables on a disjoint subset of variables conceived as covariates.

The age at which young deaf children receive cochlear implants and their vocabulary and speech-production growth: is there an added value for early implantation?

OBJECTIVE: The age at which a child receives a cochlear implant seems to be one of the more important predictors of his or her speech and language outcomes. However, understanding the association between age at implantation and child outcomes is complex because a child's age, length of device use, and age at implantation are highly related. In this study, we investigate whether there is an added value to earlier implantation or whether advantages observed in child outcomes are primarily attributable to longer device use at any given age. DESIGN: Using hierarchical linear modeling, we examined latent-growth curves for 100 children who had received their implants when they were between 1 and 10 yr of age, had used oral communication, and had used their devices for between 1 and 12 yr. Children were divided into four groups based on age at implantation: between 1 and 2.5 yr, between 2.6 and 3.5 yr, between 3.6 and 7 yr, and between 7.1 and 10 yr. RESULTS: Investigation of growth curves and rates of growth over time revealed an additional value for earlier implantation over and above advantages attributable to longer length of use at any given age. Children who had received their implants before the age of 2.5 yr had exhibited early bursts of growth in consonant-production accuracy and vocabulary and also had significantly stronger outcomes compared with age peers who had received their implants at later ages. The magnitude of the early burst diminished systematically with increasing age at implantation and was not observed for children who were older than 7 yr at implantation for consonant-production accuracy or for children who were over 3.5 yr old at implantation for vocabulary. The impact of age at implantation on children's growth curves differed for speech production and vocabulary. CONCLUSIONS: There seems to be a substantial benefit for both speech and vocabulary outcomes when children receive their implant before the age of 2.5 yr. This benefit may combine a burst of growth after implantation with the impact of increased length of use at any given age. The added advantage (i.e., burst of growth) diminishes systematically with increasing age at implantation.

Retail tobacco outlet density and youth cigarette smoking: a propensity-modeling approach.

OBJECTIVES: We examined whether retail tobacco outlet density was related to youth cigarette smoking after control for a diverse range of neighborhood characteristics. METHODS: Data were gathered from 2116 respondents (aged 11 to 23 years) residing in 178 census tracts in Chicago, Ill. Propensity score stratification methods for continuous exposures were used to adjust for potentially confounding neighborhood characteristics, thus strengthening causal inferences. RESULTS: Retail tobacco outlets were disproportionately located in neighborhoods characterized by social and economic disadvantage. In a model that excluded neighborhood confounders, a marginally significant effect was found. Youths in areas at the highest 75th percentile in retail tobacco outlet density were 13% more likely (odds ratio [OR]=1.13; 95% confidence interval [CI]=0.99, 1.28) to have smoked in the past month compared with those living at the lowest 25th percentile. However, the relation became stronger and significant (OR=0.21; 95% CI=1.04, 1.41) after introduction of tract-level confounders and was statistically significant in the propensity score-adjusted model (OR = 1.20; 95% CI = 1.001, 1.44). Results did not differ significantly between minors and those legally permitted to smoke. CONCLUSIONS: Reductions in retail tobacco outlet density may reduce rates of youth smoking.

Predicting risk-adjusted mortality for CABG surgery: logistic versus hierarchical logistic models.

BACKGROUND: In recent years, several studies in the medical and health service research literature have advocated the use of hierarchical statistical models (multilevel models or random-effects models) to analyze data that are nested (eg, patients nested within hospitals). However, these models are computer-intensive and complicated to perform. There is virtually nothing in the literature that compares the results of standard logistic regression to those of hierarchical logistic models in predicting future provider performance. OBJECTIVE: We sought to compare the ability of standard logistic regression relative to hierarchical modeling in predicting risk-adjusted hospital mortality rates for coronary artery bypass graft (CABG) surgery in New York State. DESIGN, SETTING AND PATIENTS: New York State CABG Registry data from 1994 to 1999 were used to relate statistical predictions from a given year to hospital performance 2 years hence. MAIN OUTCOME MEASURES: Predicted and observed hospital mortality rates 2 years hence were compared using root mean square errors, the mean absolute difference, and the number of hospitals whose predicted mortality rate data was within a 95% confidence interval around the observed mortality rate. RESULTS: In these data, standard logistic regression performed similarly to hierarchical models, both with and without a second level covariate. Differences in the criteria used for comparison were minimal, and when the differences could be statistically tested no significant differences were identified. CONCLUSIONS: It is instructive to compare the predictive abilities of alternative statistical models in the process of assessing their relative performance on a specific database and application.

Statistical and substantive inferences in public health: issues in the application of multilevel models.

Multilevel statistical models have become increasingly popular among public health researchers over the past decade. Yet the enthusiasm with which these models are being adopted may obscure rather than solve some problems of statistical and substantive inference. We discuss the three most common applications of multilevel models in public health: (a) cluster-randomized trials, (b) observational studies of the multilevel etiology of health and disease, and (c) assessments of health care provider performance. In each area of investigation, we describe how multilevel models are being applied, comment on the validity of the statistical and substantive inferences being drawn, and suggest ways in which the strengths of multilevel models might be more fully exploited. We conclude with a call for more careful thinking about multilevel causal inference.

Neighborhood support and the birth weight of urban infants.

Differences in maternal characteristics only partially explain the lower birth weights of infants of African-American women. It is hypothesized that economic and social features of urban neighborhoods may further account for these differences. The authors conducted a household survey of 8,782 adults residing in 343 Chicago, Illinois, neighborhoods to assess mean levels of perceived social support and used US Census data to estimate neighborhood economic disadvantage. Data on birth weight and maternal risk factors were gathered from 95,711 birth certificates (1994-1996). Before statistical adjustment of the data, infants born to African-American mothers were found to be, on average, 297 g lighter than those born to White mothers. After adjustment for individual-level risk factors, this difference was reduced to 154 g. For African-American mothers only, mean birth weight decreased significantly as the neighborhood level of economic disadvantage increased. For White mothers only, a significant positive association was found between perceived levels of neighborhood social support and infant birth weight. Adding these neighborhood-level predictors to the model reduced the adjusted White versus African-American difference in birth weight to 124 g. Results support the hypothesis that neighborhood-level factors are significantly associated with infant birth weight.