Individual Mobility across Clusters: The Impact of Ignoring Cross-Classified Data Structures in Discrete-Time Survival Analysis.

A multilevel-discrete time survival model may be appropriate for purely hierarchical data, but when data are non-purely hierarchical due to individual mobility across clusters, a cross-classified discrete time survival model may be necessary. The purpose of this research was to investigate the performance of a cross-classified discrete-time survival model and assess the impact of ignoring a cross-classified data structure on the model parameters of a conventional discrete-time survival model and a multilevel discrete-time survival model. A Monte Carlo simulation was used to examine the performance of three discrete-time survival models when individuals are mobile across clusters. Simulation factors included the value of the between-clusters variance, number of clusters, within-cluster sample size, Weibull scale parameter, and mobility rate. The results suggest that substantial relative parameter bias, unacceptable coverage of the 95% confidence intervals, and severely biased standard errors are possible for all model parameters when a discrete-time survival model is used that ignores the cross-classified data structure. The findings presented in this study are useful for methodologists and practitioners in educational research, public health, and other social sciences where discrete-time survival analysis is a common methodological technique for analyzing event-history data.

The impacts of ignoring individual mobility across clusters in estimating a piecewise growth model.

A three-level piecewise growth model (3L-PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L-PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals (level 2) who are members of a single cluster (level 3). However, in longitudinal research, it is sometimes difficult for data structures to remain purely clustered during a study, such as when some students change classrooms or schools over time. One resulting data structure in this situation is known as a multiple membership structure, where some lower-level units are members of more than one higher-level unit. The new multiple membership PGM (MM-PGM) extends the 3L-PGM to handle multiple membership data structures frequently found in the social sciences. This study sought to examine the consequences of ignoring individual mobility across clusters when estimating a 3L-PGM in comparison to estimating a MM-PGM. MM-PGM estimates were less biased (especially in the cluster-level coefficient estimates), although we found substantial bias in cluster-level variance components across some conditions for both models.

A new way for handling mobility in longitudinal data.

In the social sciences, applied researchers often face a statistical dilemma when multilevel data is structured such that lower-level units are not purely clustered within higher-level units. To aid applied researchers in appropriately analyzing such data structures, this study proposes a multiple membership growth curve model (MM-GCM). The MM-GCM offers some advantages to other similar modeling approaches, including greater flexibility in modeling the intercept at the time-point most desired for interpretation. A real longitudinal dataset from the field of education with a multiple membership structure, where some students changed schools over time, was used to demonstrate the application of the MM-GCM. Baseline and conditional MM-GCMs are presented, and parameter estimates were compared with two other common approaches to handling such data structures - the final school-GCM that ignores mobile students by only modeling the final school attended and the delete-GCM that deletes mobile students. Additionally, a simulation study was conducted to further assess the impact of ignoring mobility on parameter estimates. The results indicate that ignoring mobility results in substantial bias in model estimates, especially for cluster-level coefficients and variance components.

Evaluation of a Noyce program: Development of teacher leaders in STEM education.

The development of teacher leaders in science, technology, engineering, and mathematics has become a focus as demand grows on the national scale to improve student learning in these disciplines. As teachers' role in leadership continues to be redefined, research and professional development in teacher leadership will continue to evolve. Given the lack of a clear conceptualization of teacher leadership in the empirical literature, there is a clear methodological challenge for evaluators who are charged with assessing the impact of teacher leadership professional development programs. This paper describes how both the Utilization-Focused Evaluation and Theory-Driven Evaluation frameworks were used concurrently to design evaluation methods that were effective for assessing the impact of a dynamic teacher leadership program. The evaluation is specifically situated within the context of a Robert Noyce Scholarship Program, which aimed to grow veteran science teachers into teacher leaders. The paper describes how the evaluation frameworks used guided the evaluation methods, provides illustrative evaluation results, and states lessons learned from the author's experiences working within this context. This paper aims to provide an example of evaluation methods that could be replicated by evaluators' working within a Noyce or teacher leadership context.