Reliability of COVID-19 data: An evaluation and reflection.

IMPORTANCE: The rapid proliferation of COVID-19 has left governments scrambling, and several data aggregators are now assisting in the reporting of county cases and deaths. The different variables affecting reporting (e.g., time delays in reporting) necessitates a well-documented reliability study examining the data methods and discussion of possible causes of differences between aggregators. OBJECTIVE: To statistically evaluate the reliability of COVID-19 data across aggregators using case fatality rate (CFR) estimates and reliability statistics. DESIGN, SETTING, AND PARTICIPANTS: Cases and deaths were collected daily by volunteers via state and local health departments, as primary sources and newspaper reports, as secondary sources. In an effort to begin comparison for reliability statistical analysis, BroadStreet collected data from other COVID-19 aggregator sources, including USAFacts, Johns Hopkins University, New York Times, The COVID Tracking Project. MAIN OUTCOMES AND MEASURES: COVID-19 cases and death counts at the county and state levels. RESULTS: Lower levels of inter-rater agreement were observed across aggregators associated with the number of deaths, which manifested itself in state level Bayesian estimates of COVID-19 fatality rates. CONCLUSIONS AND RELEVANCE: A national, publicly available data set is needed for current and future disease outbreaks and improved reliability in reporting.

Selecting the "Best" Factor Structure and Moving Measurement Validation Forward: An Illustration.

Despite the broad literature base on factor analysis best practices, research seeking to evaluate a measure's psychometric properties frequently fails to consider or follow these recommendations. This leads to incorrect factor structures, numerous and often overly complex competing factor models and, perhaps most harmful, biased model results. Our goal is to demonstrate a practical and actionable process for factor analysis through (a) an overview of six statistical and psychometric issues and approaches to be aware of, investigate, and report when engaging in factor structure validation, along with a flowchart for recommended procedures to understand latent factor structures; (b) demonstrating these issues to provide a summary of the updated Posttraumatic Stress Disorder Checklist (PCL-5) factor models and a rationale for validation; and (c) conducting a comprehensive statistical and psychometric validation of the PCL-5 factor structure to demonstrate all the issues we described earlier. Considering previous research, the PCL-5 was evaluated using a sample of 1,403 U.S. Air Force remotely piloted aircraft operators with high levels of battlefield exposure. Previously proposed PCL-5 factor structures were not supported by the data, but instead a bifactor model is arguably more statistically appropriate.

Development and validation of PediaTrac™: A web-based tool to track developing infants.

OBJECTIVE: PediaTrac™, a 363-item web-based tool to track infant development, administered in modules of ∼40-items per sampling period, newborn (NB), 2--, 4--, 6--, 9-- and 12--months was validated. Caregivers answered demographic, medical, and environmental questions, and questions covering the sensorimotor, feeding/eating, sleep, speech/language, cognition, social-emotional, and attachment domains. METHODS: Expert Panel Reviews and Cognitive Interviews (CI) were conducted to validate the item bank. Classical Test Theory (CTT) and Item Response Theory (IRT) methods were employed to examine the dimensionality and psychometric properties of PediaTrac with pooled longitudinal and cross-sectional cohorts (N = 132). RESULTS: Intraclass correlation coefficients (ICC) for the Expert Panel Review revealed moderate agreement at 6 -months and good reliability at other sampling periods. ICC estimates for CI revealed moderate reliability regarding clarity of the items at NB and 4 months, good reliability at 2--, 9-- and 12--months and excellent reliability at 6 -months. CTT revealed good coefficient alpha estimates (α ≥ 0.77 for five of the six ages) for the Social-Emotional/Communication, Attachment (α ≥ 0.89 for all ages), and Sensorimotor (α ≥ 0.75 at 6-months) domains, revealing the need for better targeting of sensorimotor items. IRT modeling revealed good reliability (r = 0.85-0.95) for three distinct domains (Feeding/Eating, Social-Emotional/Communication and Attachment) and four subdomains (Feeding Breast/Formula, Feeding Solid Food, Social-Emotional Information Processing, Communication/Cognition). Convergent and discriminant construct validity were demonstrated between our IRT-modeled domains and constructs derived from existing developmental, behavioral and caregiver measures. Our Attachment domain was significantly correlated with existing measures at the NB and 2-month periods, while the Social-Emotional/Communication domain was highly correlated with similar constructs at the 6-, 9- and 12-month periods. CONCLUSION: PediaTrac has potential for producing novel and effective estimates of infant development via the Sensorimotor, Feeding/Eating, Social-Emotional/Communication and Attachment domains.

Learning curve analyses in neurodevelopmental disorders: are children with autism spectrum disorder truly visual learners?

Visual and auditory verbal learning using a selective reminding format was studied in a mixed clinical sample of children with autism spectrum disorder (ASD) (n = 42), attention-deficit hyperactivity disorder (n = 83), velocardiofacial syndrome (n = 17) and neurotypicals (n = 38) using the Test of Memory and Learning to (1) more thoroughly characterize and examine the integrity of learning and memory processes, (2) to better understand the mechanisms of learning impairment, and (3) to inform instructional practices in ASD. Contrary to expectations, children with ASD demonstrated a relative weakness in the rate of acquisition of visual in contrast to verbal learning compared to neurotypicals. They also showed a complex pattern of consolidation. Overall, between-group differences were more likely to emerge during the visual learning task, suggesting that it may be more sensitive for detecting neurodevelopmental differences. The heuristic value of assessing memory and learning across multiple trials and comparing performance during immediate and delayed recall is discussed.

Self-report methodology is insufficient for improving the assessment and classification of Axis II personality disorders.

Current approaches to the assessment and classification of personality disorders (PDs) rely almost exclusively on self-report methodology. In this paper, we document the many difficulties with self-reports, including limitations in their accuracy, the confounding effect of mood state, and problems with the selection and retention of factors in factor analytic approaches to self-report questionnaires. We also discuss the role of implicit processes in self-reports, with special attention directed to the phenomenon of priming and its effect on outcome. To rectify these issues, we suggest a transtheoretical, multimethod, multimodal approach to personality pathology assessment and diagnosis, which utilizes the richness of prototypes and empirical findings on PD categories and pathologies.

Comparing factor analytic models of the DSM-IV personality disorders.

There is little agreement about the latent factor structure of the Diagnostic and Statistical Manual of Mental Disorders (DSM) personality disorders (PDs). Factor analytic studies over the past 2 decades have yielded different results, in part reflecting differences in factor analytic technique, the measure used to assess the PDs, and the changing DSM criteria. In this study, we explore the latent factor structure of the DSM (4th ed.; IV) PDs in a sample of 1200 psychiatric outpatients evaluated with the Structured Interview for DSM-IV PDs (B. Pfohl, N. Blum, & M. Zimmerman, 1997). We first evaluated 2 a priori models of the PDs with confirmatory factor analysis (CFA), reflecting their inherent organization in the DSM-IV: a 3-factor model and a 10-factor model. Fit statistics did not suggest that these models yielded an adequate fit. We then evaluated the latent structure with exploratory factor analysis (EFA). Multiple solutions produced more statistically and theoretically reasonable results, as well as providing clinically useful findings. On the basis of fit statistics and theory, 3 models were evaluated further--the 4-, 5-, and 10-factor models. The 10-factor model, which did not resemble the 10-factor model of the CFA, was determined to be the strongest of all 3 models. Future research should use contemporary methods of evaluating factor analytic results in order to more thoroughly compare various factor solutions.

A Comparative Investigation of Rotation Criteria Within Exploratory Factor Analysis.

Exploratory factor analysis (EFA) is a commonly used statistical technique for examining the relationships between variables (e.g., items) and the factors (e.g., latent traits) they depict. There are several decisions that must be made when using EFA, with one of the more important being choice of the rotation criterion. This selection can be arduous given the numerous rotation criteria available and the lack of research/literature that compares their function and utility. Historically, researchers have chosen rotation criteria based on whether or not factors are correlated and have failed to consider other important aspects of their data. This study reviews several rotation criteria, demonstrates how they may perform with different factor pattern structures, and highlights for researchers subtle but important differences between each rotation criterion. The choice of rotation criterion is critical to ensure researchers make informed decisions as to when different rotation criteria may or may not be appropriate. The results suggest that depending on the rotation criterion selected and the complexity of the factor pattern matrix, the interpretation of the interfactor correlations and factor pattern loadings can vary substantially. Implications and future directions are discussed.

Assessing interpersonal fusion: reliability and validity of a new DSI fusion with others subscale.

The Differentiation of Self Inventory (DSI) is a multidimensional measure of differentiation consisting of four subscales focusing on adults (ages 25+), and their significant relationships, including current relationships with family of origin. Although the DSI full scale and three of its subscales are theoretically and psychometrically sound, the Fusion with Others (FO) subscale is lacking. Therefore, responses of 225 adults were used to revise the FO subscale. Results yielded a 12-item, revised FO subscale with improved internal consistency reliability and construct validity. Greater fusion with others was associated with greater spousal fusion and dimensions of adult attachment insecurity. Implications for Bowen theory and suggestions for future research with the DSI-R are discussed.