Equivalence testing to judge model fit: A Monte Carlo simulation.

Decades of published methodological research have shown the chi-square test of model fit performs inconsistently and unreliably as a determinant of structural equation model (SEM) fit. Likewise, SEM indices of model fit, such as comparative fit index (CFI) and root-mean-square error of approximation (RMSEA) also perform inconsistently and unreliably. Despite rather unreliable ways to statistically assess model fit, researchers commonly rely on these methods for lack of a suitable inferential alternative. Marcoulides and Yuan (2017) have proposed the first inferential test of SEM fit in many years: an equivalence test adaptation of the RMSEA and CFI indices (i.e., RMSEAt and CFIt). However, the ability of this equivalence testing approach to accurately judge acceptable and unacceptable model fit has not been empirically tested. This fully crossed Monte Carlo simulation evaluated the accuracy of equivalence testing combining many of the same independent variable (IV) conditions used in previous fit index simulation studies, including sample size (N = 100-1,000), model specification (correctly specified or misspecified), model type (confirmatory factor analysis [CFA], path analysis, or SEM), number of variables analyzed (low or high), data distribution (normal or skewed), and missing data (none, 10%, or 25%). Results show equivalence testing performs rather inconsistently and unreliably across IV conditions, with acceptable or unacceptable RMSEAt and CFIt model fit index values often being contingent on complex interactions among conditions. Proportional z-tests and logistic regression analyses indicated that equivalence tests of model fit are problematic under multiple conditions, especially those where models are mildly misspecified. Recommendations for researchers are offered, but with the provision that they be used with caution until more research and development is available. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Social predictors of doctoral student mental health and well-being.

Graduate students' mental health and well-being is a prominent concern across various disciplines. However, early predictors of mental health and well-being in graduate education, specifically doctoral education, have rarely been studied. The present study evaluated both the underlying latent classification of individuals' mental well-being and predictors of those classifications. Results estimated two latent classes of students' mental health and well-being: one class with generally high levels of mental well-being and one with lower levels of mental well-being. Regression analyses showed that mentoring in the second year of doctoral study, certainty of choice in the third year, and both academic development and sense of belonging in the fourth year were positive predictors of membership in the higher mental well-being class. In contrast to some prior studies, demographic variables were not related to the identified well-being classifications. Regression analyses further showed that mental well-being was negatively related to participants' number of publications and research self-efficacy, indicating a problematic relationship between scholarly productivity and confidence and well-being. These findings may be used to identify and provide targeted support for students who are at-risk for having or developing lower levels of mental well-being in their graduate programs.

Understanding the Transient Nature of STEM Doctoral Students' Research Self-Efficacy Across Time: Considering the Role of Gender, Race, and First-Generation College Status.

Developing research self-efficacy is an important part of doctoral student preparation. Despite the documented importance of research self-efficacy, little is known about the progression of doctoral students' research self-efficacy over time in general and for students from minoritized groups. This study examined both within- and between-person stability of research self-efficacy from semester to semester over 4 years, focusing on doctoral students in biological sciences (N = 336). Using random intercept autoregressive analyses, we evaluated differences in stability across gender, racially minoritized student status, and first-generation student status. Results showed similar mean levels of self-efficacy across demographic groups and across time. However, there were notable differences in between-person and within-person stability over time, specifically showing higher between-person and lower within-person stability for racially minoritized and first-generation students. These findings indicate that racially minoritized and first-generation students' research self-efficacy reports were less consistent from semester to semester. Such results may indicate that non-minoritized and continuing-generation students' experiences from semester to semester typically reinforce their beliefs about their own abilities related to conducting research, while such is not the case for racially minoritized nor first-generation students. Future research should examine what types of experiences impact self-efficacy development across doctoral study to offer more precise insights about factors that influence these differences in within-person stability.

Finding a Fit: Biological Science Doctoral Students' Selection of a Principal Investigator and Research Laboratory.

In the laboratory-based disciplines, selection of a principal investigator (PI) and research laboratory (lab) indelibly shapes doctoral students' experiences and educational outcomes. Framed by the theoretical concept of person-environment fit from within a socialization model, we use an inductive, qualitative approach to explore how a sample of 42 early-stage doctoral students enrolled in biological sciences programs made decisions about fitting with a PI and within a lab. Results illuminated a complex array of factors that students considered in selecting a PI, including PI relationship, mentoring style, and professional stability. Further, with regard to students' lab selection, peers and research projects played an important role. Students actively conceptualized trade-offs among various dimensions of fit. Our findings also revealed cases in which students did not secure a position in their first (or second) choice labs and had to consider their potential fit with suboptimal placements (in terms of their initial assessments). Thus, these students weighted different factors of fit against the reality of needing to secure financial support to continue in their doctoral programs. We conclude by presenting and framing implications for students, PIs, and doctoral programs, and recommend providing transparency and candor around the PI and lab selection processes.

"How Well Does Your Structural Equation Model Fit Your Data?": Is Marcoulides and Yuan's Equivalence Test the Answer?

Structural equation modeling is an ideal data analytical tool for testing complex relationships among many analytical variables. It can simultaneously test multiple mediating and moderating relationships, estimate latent variables on the basis of related measures, and address practical issues such as nonnormality and missing data. To test the extent to which a hypothesized model provides an appropriate characterization of the collective relationships among its variables, researchers must assess the "fit" between the model and the sample's data. However, interpreting estimates of model fit is a problematic process. The traditional inferential test of model fit, the chi-square test, is biased due to sample size. Fit indices provide descriptive (i.e., noninferential) values of model fit (e.g., comparative fit index, root-mean-square error of approximation), but are unable to provide a definitive "acceptable" or "unacceptable" fit determination. Marcoulides and Yuan have introduced an equivalence-testing technique for assessing model fit that combines traditional descriptive fit indices with an inferential testing strategy in the form of confidence intervals to facilitate more definitive fit conclusions. In this paper, we explain this technique and demonstrate its application, highlighting the substantial advantages it offers the life sciences education community for drawing robust conclusions from structural equation models. A structural equation model and data set (N = 1902) drawn from previously published research are used to illustrate how to perform and interpret an equivalence test of model fit using Marcoulides and Yuan's approach.

Identifying Faculty and Peer Interaction Patterns of First-Year Biology Doctoral Students: A Latent Class Analysis.

Faculty and peer interactions play a key role in shaping graduate student socialization. Yet, within the literature on graduate student socialization, researchers have primarily focused on understanding the nature and impact of faculty alone, and much less is known about how peer interactions also contribute to graduate student outcomes. Using a national sample of first-year biology doctoral students, this study reveals distinct categories that classify patterns of faculty and peer interaction. Further, we document inequities such that certain groups (e.g., underrepresented minority students) report constrained types of interactions with faculty and peers. Finally, we connect faculty and peer interaction patterns to student outcomes. Our findings reveal that, while the classification of faculty and peer interactions predicted affective and experiential outcomes (e.g., sense of belonging, satisfaction with academic development), it was not a consistent predictor of more central outcomes of the doctoral socialization process (e.g., research skills, commitment to degree). These and other findings are discussed, focusing on implications for future research, theory, and practice related to graduate training.

Postdocs' lab engagement predicts trajectories of PhD students' skill development.

The doctoral advisor-typically the principal investigator (PI)-is often characterized as a singular or primary mentor who guides students using a cognitive apprenticeship model. Alternatively, the "cascading mentorship" model describes the members of laboratories or research groups receiving mentorship from more senior laboratory members and providing it to more junior members (i.e., PIs mentor postdocs, postdocs mentor senior graduate students, senior students mentor junior students, etc.). Here we show that PIs' laboratory and mentoring activities do not significantly predict students' skill development trajectories, but the engagement of postdocs and senior graduate students in laboratory interactions do. We found that the cascading mentorship model accounts best for doctoral student skill development in a longitudinal study of 336 PhD students in the United States. Specifically, when postdocs and senior doctoral students actively participate in laboratory discussions, junior PhD students are over 4 times as likely to have positive skill development trajectories. Thus, postdocs disproportionately enhance the doctoral training enterprise, despite typically having no formal mentorship role. These findings also illustrate both the importance and the feasibility of identifying evidence-based practices in graduate education.

Null effects of boot camps and short-format training for PhD students in life sciences.

Many PhD programs incorporate boot camps and summer bridge programs to accelerate the development of doctoral students' research skills and acculturation into their respective disciplines. These brief, high-intensity experiences span no more than several weeks and are typically designed to expose graduate students to data analysis techniques, to develop scientific writing skills, and to better embed incoming students into the scholarly community. However, there is no previous study that directly measures the outcomes of PhD students who participate in such programs and compares them to the outcomes of students who did not participate. Likewise, no previous study has used a longitudinal design to assess these outcomes over time. Here we show that participation in such programs is not associated with detectable benefits related to skill development, socialization into the academic community, or scholarly productivity for students in our sample. Analyzing data from 294 PhD students in the life sciences from 53 US institutions, we found no statistically significant differences in outcomes between participants and nonparticipants across 115 variables. These results stand in contrast to prior studies presenting boot camps as effective interventions based on participant satisfaction and perceived value. Many universities and government agencies (e.g., National Institutes of Health and National Science Foundation) invest substantial resources in boot camp and summer bridge activities in the hopes of better supporting scientific workforce development. Our findings do not reveal any measurable benefits to students, indicating that an allocation of limited resources to alternative strategies with stronger empirical foundations warrants consideration.

Time-to-Credit Gender Inequities of First-Year PhD Students in the Biological Sciences.

Equitable gender representation is an important aspect of scientific workforce development to secure a sufficient number of individuals and a diversity of perspectives. Biology is the most gender equitable of all scientific fields by the marker of degree attainment, with 52.5% of PhDs awarded to women. However, equitable rates of degree completion do not translate into equitable attainment of faculty or postdoctoral positions, suggesting continued existence of gender inequalities. In a national cohort of 336 first-year PhD students in the biological sciences (i.e., microbiology, cellular biology, molecular biology, developmental biology, and genetics) from 53 research institutions, female participants logged significantly more research hours than males and were significantly more likely than males to attribute their work hours to the demands of their assigned projects over the course of the academic year. Despite this, males were 15% more likely to be listed as authors on published journal articles, indicating inequality in the ratio of time to credit. Given the cumulative advantage that accrues for students who publish early in their graduate careers and the central role that scholarly productivity plays in academic hiring decisions, these findings collectively point to a major potential source of persisting underrepresentation of women on university faculties in these fields.

Instructional design affects the efficacy of simulation-based training in central venous catheterization.

BACKGROUND: Simulation-based learning is a common educational tool in health care training and frequently involves instructional designs based on Experiential Learning Theory (ELT). However, little research explores the effectiveness and efficiency of different instructional design methodologies appropriate for simulations. The aim of this study was to compare 2 instructional design models, ELT and Guided Experiential Learning (GEL), to determine which is more effective for training the central venous catheterization procedure. METHODS: Using a quasi-experimental randomized block design, nurse anesthetists completed training under 1 of the 2 instructional design models. Performance was assessed using a checklist of central venous catheterization performance, pass rates, and critical action errors. RESULTS: Participants in the GEL condition performed significantly better than those in the ELT condition on the overall checklist score after controlling for individual practice time (F[1, 29] = 4.021, P = .027, Cohen's d = .71), had higher pass rates (P = .006, Cohen's d = 1.15), and had lower rates of failure due to critical action errors (P = .038, Cohen's d = .81). CONCLUSIONS: The GEL model of instructional design is significantly more effective than ELT for simulation-based learning of the central venous catheterization procedure, yielding large differences in effect size.

Graduate students' teaching experiences improve their methodological research skills.

Science, technology, engineering, and mathematics (STEM) graduate students are often encouraged to maximize their engagement with supervised research and minimize teaching obligations. However, the process of teaching students engaged in inquiry provides practice in the application of important research skills. Using a performance rubric, we compared the quality of methodological skills demonstrated in written research proposals for two groups of early career graduate students (those with both teaching and research responsibilities and those with only research responsibilities) at the beginning and end of an academic year. After statistically controlling for preexisting differences between groups, students who both taught and conducted research demonstrate significantly greater improvement in their abilities to generate testable hypotheses and design valid experiments. These results indicate that teaching experience can contribute substantially to the improvement of essential research skills.