Your browser doesn't support javascript.
loading
Mostrar: 20 | 50 | 100
Resultados 1 - 20 de 64
Filtrar
Más filtros

Intervalo de año de publicación
1.
Behav Res Methods ; 2024 Feb 20.
Artículo en Inglés | MEDLINE | ID: mdl-38379114

RESUMEN

This study proposes a procedure for substantive dimensionality estimation in the presence of wording effects, the inconsistent response to regular and reversed self-report items. The procedure developed consists of subtracting an approximate estimate of the wording effects variance from the sample correlation matrix and then estimating the substantive dimensionality on the residual correlation matrix. This is achieved by estimating a random intercept factor with unit loadings for all the regular and unrecoded reversed items. The accuracy of the procedure was evaluated through an extensive simulation study that manipulated nine relevant variables and employed the exploratory graph analysis (EGA) and parallel analysis (PA) retention methods. The results indicated that combining the proposed procedure with EGA or PA achieved high accuracy in estimating the substantive latent dimensionality, but that EGA was superior. Additionally, the present findings shed light on the complex ways that wording effects impact the dimensionality estimates when the response bias in the data is ignored. A tutorial on substantive dimensionality estimation with the R package EGAnet is offered, as well as practical guidelines for applied researchers.

2.
Multivariate Behav Res ; 58(6): 1072-1089, 2023.
Artículo en Inglés | MEDLINE | ID: mdl-37038725

RESUMEN

Exploratory bi-factor analysis (EBFA) is a very popular approach to estimate models where specific factors are concomitant to a single, general dimension. However, the models typically encountered in fields like personality, intelligence, and psychopathology involve more than one general factor. To address this circumstance, we developed an algorithm (GSLiD) based on partially specified targets to perform exploratory bi-factor analysis with multiple general factors (EBFA-MGF). In EBFA-MGF, researchers do not need to conduct independent bi-factor analyses anymore because several bi-factor models are estimated simultaneously in an exploratory manner, guarding against biased estimates and model misspecification errors due to unexpected cross-loadings and factor correlations. The results from an exhaustive Monte Carlo simulation manipulating nine variables of interest suggested that GSLiD outperforms the Schmid-Leiman approximation and is robust to challenging conditions involving cross-loadings and pure items of the general factors. Thereby, we supply an R package (bifactor) to make EBFA-MGF readily available for substantive research. Finally, we use GSLiD to assess the hierarchical structure of a reduced version of the Personality Inventory for DSM-5 Short Form (PID-5-SF).


Asunto(s)
Algoritmos , Canales de Calcio , Simulación por Computador , Análisis Factorial , Método de Montecarlo , Psicometría
3.
Behav Res Methods ; 55(7): 3446-3460, 2023 10.
Artículo en Inglés | MEDLINE | ID: mdl-36127563

RESUMEN

Cognitive diagnosis models (CDMs) are used in educational, clinical, or personnel selection settings to classify respondents with respect to discrete attributes, identifying strengths and needs, and thus allowing to provide tailored training/treatment. As in any assessment, an accurate reliability estimation is crucial for valid score interpretations. In this sense, most CDM reliability indices are based on the posterior probabilities of the estimated attribute profiles. These posteriors are traditionally computed using point estimates for the model parameters as approximations to their populational values. If the uncertainty around these parameters is unaccounted for, the posteriors may be overly peaked, deriving into overestimated reliabilities. This article presents a multiple imputation (MI) procedure to integrate out the model parameters in the estimation of the posterior distributions, thus correcting the reliability estimation. A simulation study was conducted to compare the MI procedure with the traditional reliability estimation. Five factors were manipulated: the attribute structure, the CDM model (DINA and G-DINA), test length, sample size, and item quality. Additionally, an illustration using the Examination for the Certificate of Proficiency in English data was analyzed. The effect of sample size was studied by sampling subsets of subjects from the complete data. In both studies, the traditional reliability estimation systematically provided overestimated reliabilities, whereas the MI procedure offered more accurate results. Accordingly, practitioners in small educational or clinical settings should be aware that the reliability estimation using model parameter point estimates may be positively biased. R codes for the MI procedure are made available.


Asunto(s)
Concienciación , Humanos , Reproducibilidad de los Resultados , Simulación por Computador
4.
Behav Res Methods ; 54(3): 1476-1492, 2022 06.
Artículo en Inglés | MEDLINE | ID: mdl-34505277

RESUMEN

The use of multidimensional forced-choice questionnaires has been proposed as a means of improving validity in the assessment of non-cognitive attributes in high-stakes scenarios. However, the reduced precision of trait estimates in this questionnaire format is an important drawback. Accordingly, this article presents an optimization procedure for assembling pairwise forced-choice questionnaires while maximizing posterior marginal reliabilities. This procedure is performed through the adaptation of a known genetic algorithm (GA) for combinatorial problems. In a simulation study, the efficiency of the proposed procedure was compared with a quasi-brute-force (BF) search. For this purpose, five-dimensional item pools were simulated to emulate the real problem of generating a forced-choice personality questionnaire under the five-factor model. Three factors were manipulated: (1) the length of the questionnaire, (2) the relative item pool size with respect to the questionnaire's length, and (3) the true correlations between traits. The recovery of the person parameters for each assembled questionnaire was evaluated through the squared correlation between estimated and true parameters, the root mean square error between the estimated and true parameters, the average difference between the estimated and true inter-trait correlations, and the average standard error for each trait level. The proposed GA offered more accurate trait estimates than the BF search within a reasonable computation time in every simulation condition. Such improvements were especially important when measuring correlated traits and when the relative item pool sizes were higher. A user-friendly online implementation of the algorithm was made available to the users.


Asunto(s)
Algoritmos , Personalidad , Simulación por Computador , Humanos , Encuestas y Cuestionarios
5.
Multivariate Behav Res ; 56(1): 101-119, 2021.
Artículo en Inglés | MEDLINE | ID: mdl-32449372

RESUMEN

As general factor modeling continues to grow in popularity, researchers have become interested in assessing how reliable general factor scores are. Even though omega hierarchical estimation has been suggested as a useful tool in this context, little is known about how to approximate it using modern bi-factor exploratory factor analysis methods. This study is the first to compare how omega hierarchical estimates were recovered by six alternative algorithms: Bi-quartimin, bi-geomin, Schmid-Leiman (SL), empirical iterative empirical target rotation based on an initial SL solution (SLiD), direct SL (DSL), and direct bi-factor (DBF). The algorithms were tested in three Monte-Carlo simulations including bi-factor and second-order structures and presenting complexities such as cross-loadings or pure indicators of the general factor and structures without a general factor. Results showed that SLiD provided the best approximation to omega hierarchical under most conditions. Overall, neither SL, bi-quartimin, nor bi-geomin produced an overall satisfactory recovery of omega hierarchical. Lastly, the performance of DSL and DBF depended upon the average discrepancy between the loadings of the general and the group factors. The re-analysis of eight classical datasets further illustrated how algorithm selection could influence judgments regarding omega hierarchical.


Asunto(s)
Algoritmos , Juicio , Análisis Factorial , Método de Montecarlo , Rotación
6.
Sensors (Basel) ; 21(13)2021 Jun 23.
Artículo en Inglés | MEDLINE | ID: mdl-34201455

RESUMEN

High-resolution 3D scanning devices produce high-density point clouds, which require a large capacity of storage and time-consuming processing algorithms. In order to reduce both needs, it is common to apply surface simplification algorithms as a preprocessing stage. The goal of point cloud simplification algorithms is to reduce the volume of data while preserving the most relevant features of the original point cloud. In this paper, we present a new point cloud feature-preserving simplification algorithm. We use a global approach to detect saliencies on a given point cloud. Our method estimates a feature vector for each point in the cloud. The components of the feature vector are the normal vector coordinates, the point coordinates, and the surface curvature at each point. Feature vectors are used as basis signals to carry out a dictionary learning process, producing a trained dictionary. We perform the corresponding sparse coding process to produce a sparse matrix. To detect the saliencies, the proposed method uses two measures, the first of which takes into account the quantity of nonzero elements in each column vector of the sparse matrix and the second the reconstruction error of each signal. These measures are then combined to produce the final saliency value for each point in the cloud. Next, we proceed with the simplification of the point cloud, guided by the detected saliency and using the saliency values of each point as a dynamic clusterization radius. We validate the proposed method by comparing it with a set of state-of-the-art methods, demonstrating the effectiveness of the simplification method.


Asunto(s)
Algoritmos
7.
Qual Life Res ; 27(4): 1015-1025, 2018 Apr.
Artículo en Inglés | MEDLINE | ID: mdl-29143905

RESUMEN

PURPOSE: Developing valid and reliable instruments that can be used across countries is necessary. The present study aimed to test the comparability of quality of life scores across three European countries (Finland, Poland, and Spain). METHOD: Data from 9987 participants interviewed between 2011 and 2012 were employed, using nationally representative samples from the Collaborative Research on Ageing in Europe project. The WHOQOL-AGE questionnaire is a 13-item test and was employed to assess the quality of life in the three considered countries. First of all, two models (a bifactor model and a two-correlated factor model) were proposed and tested in each country by means of confirmatory factor models. Second, measurement invariance across the three countries was tested using multi-group confirmatory factor analysis for that model which showed the best fit. Finally, differences in latent mean scores across countries were analyzed. RESULTS: The results indicated that the bifactor model showed more satisfactory goodness-of-fit indices than the two-correlated factor model and that the WHOQOL-AGE questionnaire is a partially scalar invariant instrument (only two items do not meet scalar invariance). Quality of life scores were higher in Finland (considered as the reference category: mean = 0, SD = 1) than in Spain (mean = - 0.547, SD = 1.22) and Poland (mean = - 0.927, SD = 1.26). CONCLUSIONS: Respondents from Finland, Poland, and Spain attribute the same meaning to the latent construct studied, and differences across countries can be due to actual differences in quality of life. According to the results, the comparability across the different considered samples is supported and the WHOQOL-AGE showed an adequate validity in terms of cross-country validation. Caution should be exercised with the two items which did not meet scalar invariance, as potential indicator of differential item functioning.


Asunto(s)
Análisis Factorial , Psicometría/métodos , Calidad de Vida/psicología , Femenino , Humanos , Masculino , Persona de Mediana Edad , Proyectos de Investigación , Encuestas y Cuestionarios
8.
Hum Brain Mapp ; 38(2): 803-816, 2017 02.
Artículo en Inglés | MEDLINE | ID: mdl-27726264

RESUMEN

Neuroimaging research involves analyses of huge amounts of biological data that might or might not be related with cognition. This relationship is usually approached using univariate methods, and, therefore, correction methods are mandatory for reducing false positives. Nevertheless, the probability of false negatives is also increased. Multivariate frameworks have been proposed for helping to alleviate this balance. Here we apply multivariate distance matrix regression for the simultaneous analysis of biological and cognitive data, namely, structural connections among 82 brain regions and several latent factors estimating cognitive performance. We tested whether cognitive differences predict distances among individuals regarding their connectivity pattern. Beginning with 3,321 connections among regions, the 36 edges better predicted by the individuals' cognitive scores were selected. Cognitive scores were related to connectivity distances in both the full (3,321) and reduced (36) connectivity patterns. The selected edges connect regions distributed across the entire brain and the network defined by these edges supports high-order cognitive processes such as (a) (fluid) executive control, (b) (crystallized) recognition, learning, and language processing, and (c) visuospatial processing. This multivariate study suggests that one widespread, but limited number, of regions in the human brain, supports high-level cognitive ability differences. Hum Brain Mapp 38:803-816, 2017. © 2016 Wiley Periodicals, Inc.


Asunto(s)
Encéfalo/diagnóstico por imagen , Encéfalo/fisiología , Cognición/fisiología , Análisis Multivariante , Análisis de Regresión , Adolescente , Mapeo Encefálico , Femenino , Humanos , Imagenología Tridimensional , Imagen por Resonancia Magnética , Masculino , Vías Nerviosas/diagnóstico por imagen , Vías Nerviosas/fisiología , Pruebas Neuropsicológicas , Reproducibilidad de los Resultados , Adulto Joven
9.
Multivariate Behav Res ; 52(4): 416-429, 2017.
Artículo en Inglés | MEDLINE | ID: mdl-28375697

RESUMEN

The current study proposes a new bi-factor rotation method, Schmid-Leiman with iterative target rotation (SLi), based on the iteration of partially specified target matrices and an initial target constructed from a Schmid-Leiman (SL) orthogonalization. SLi was expected to ameliorate some of the limitations of the previously presented SL bi-factor rotations, SL and SL with target rotation (SLt), when the factor structure either includes cross-loadings, near-zero loadings, or both. A Monte Carlo simulation was carried out to test the performance of SLi, SL, SLt, and the two analytic bi-factor rotations, bi-quartimin and bi-geomin. The results revealed that SLi accurately recovered the bi-factor structures across the majority of the conditions, and generally outperformed the other rotation methods. SLi provided the biggest improvements over SL and SLt when the bi-factor structures contained cross-loadings and pure indicators of the general factor. Additionally, SLi was superior to bi-quartimin and bi-geomin, which performed inconsistently across the types of factor structures evaluated. No method produced a good recovery of the bi-factor structures when small samples (N = 200) were combined with low factor loadings (0.30-0.50) in the specific factors. Thus, it is recommended that larger samples of at least 500 observations be obtained.


Asunto(s)
Análisis Factorial , Modelos Estadísticos , Interpretación Estadística de Datos , Humanos , Método de Montecarlo , Análisis Multivariante , Calidad de Vida
10.
Hum Brain Mapp ; 35(8): 3805-18, 2014 Aug.
Artículo en Inglés | MEDLINE | ID: mdl-24677433

RESUMEN

Intelligence is composed of a set of cognitive abilities hierarchically organized. General and specific abilities capture distinguishable, but related, facets of the intelligence construct. Here, we analyze gray matter with three morphometric indices (volume, cortical surface area, and cortical thickness) at three levels of the intelligence hierarchy (tests, first-order factors, and a higher-order general factor, g). A group of one hundred and four healthy young adults completed a cognitive battery and underwent high-resolution structural MRI. Latent scores were computed for the intelligence factors and tests were also analyzed. The key finding reveals substantial variability in gray matter correlates at the test level, which is substantially reduced for the first-order and the higher-order factors. This supports a reversed hierarchy in the brain with respect to cognitive abilities at different psychometric levels: the greater the generality, the smaller the number of relevant gray matter clusters accounting for individual differences in intelligent performance.


Asunto(s)
Encéfalo/anatomía & histología , Cognición , Sustancia Gris/anatomía & histología , Inteligencia , Adolescente , Adulto , Análisis Factorial , Femenino , Humanos , Individualidad , Pruebas de Inteligencia , Imagen por Resonancia Magnética , Masculino , Modelos Psicológicos , Pruebas Neuropsicológicas , Tamaño de los Órganos , Psicometría , Procesamiento de Señales Asistido por Computador , Adulto Joven
11.
Neuroimage ; 72: 143-52, 2013 May 15.
Artículo en Inglés | MEDLINE | ID: mdl-23357078

RESUMEN

Evidence from neuroimaging studies suggests that intelligence differences may be supported by a parieto-frontal network. Research shows that this network is also relevant for cognitive functions such as working memory and attention. However, previous studies have not explicitly analyzed the commonality of brain areas between a broad array of intelligence factors and cognitive functions tested in the same sample. Here fluid, crystallized, and spatial intelligence, along with working memory, executive updating, attention, and processing speed were each measured by three diverse tests or tasks. These twenty-one measures were completed by a group of one hundred and four healthy young adults. Three cortical measures (cortical gray matter volume, cortical surface area, and cortical thickness) were regressed against psychological latent scores obtained from a confirmatory factor analysis for removing test and task specific variance. For cortical gray matter volume and cortical surface area, the main overlapping clusters were observed in the middle frontal gyrus and involved fluid intelligence and working memory. Crystallized intelligence showed an overlapping cluster with fluid intelligence and working memory in the middle frontal gyrus. The inferior frontal gyrus showed overlap for crystallized intelligence, spatial intelligence, attention, and processing speed. The fusiform gyrus in temporal cortex showed overlap for spatial intelligence and attention. Parietal and occipital areas did not show any overlap across intelligence and cognitive factors. Taken together, these findings underscore that structural features of gray matter in the frontal lobes support those aspects of intelligence related to basic cognitive processes.


Asunto(s)
Mapeo Encefálico , Cognición/fisiología , Lóbulo Frontal/anatomía & histología , Lóbulo Frontal/fisiología , Atención/fisiología , Encéfalo/anatomía & histología , Encéfalo/fisiología , Femenino , Humanos , Inteligencia , Imagen por Resonancia Magnética , Masculino , Pruebas Neuropsicológicas , Adulto Joven
12.
Educ Psychol Meas ; 83(2): 294-321, 2023 Apr.
Artículo en Inglés | MEDLINE | ID: mdl-36866066

RESUMEN

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled on-the-fly considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A-rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A-rule with questionnaires assembled on-the-fly, the T-rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT.

13.
Psychol Methods ; 2023 May 25.
Artículo en Inglés | MEDLINE | ID: mdl-37227893

RESUMEN

The number of available factor analytic techniques has been increasing in the last decades. However, the lack of clear guidelines and exhaustive comparison studies between the techniques might hinder that these valuable methodological advances make their way to applied research. The present paper evaluates the performance of confirmatory factor analysis (CFA), CFA with sequential model modification using modification indices and the Saris procedure, exploratory factor analysis (EFA) with different rotation procedures (Geomin, target, and objectively refined target matrix), Bayesian structural equation modeling (BSEM), and a new set of procedures that, after fitting an unrestrictive model (i.e., EFA, BSEM), identify and retain only the relevant loadings to provide a parsimonious CFA solution (ECFA, BCFA). By means of an exhaustive Monte Carlo simulation study and a real data illustration, it is shown that CFA and BSEM are overly stiff and, consequently, do not appropriately recover the structure of slightly misspecified models. EFA usually provides the most accurate parameter estimates, although the rotation procedure choice is of major importance, especially depending on whether the latent factors are correlated or not. Finally, ECFA might be a sound option whenever an a priori structure cannot be hypothesized and the latent factors are correlated. Moreover, it is shown that the pattern of the results of a factor analytic technique can be somehow predicted based on its positioning in the confirmatory-exploratory continuum. Applied recommendations are given for the selection of the most appropriate technique under different representative scenarios by means of a detailed flowchart. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

14.
Psicothema ; 35(1): 50-57, 2023 Feb.
Artículo en Inglés | MEDLINE | ID: mdl-36695850

RESUMEN

BACKGROUND: The emergence of digital technology in the field of psychological and educational measurement and assessment broadens the traditional concept of pencil and paper tests. New assessment models built on the proliferation of smartphones, social networks and software developments are opening up new horizons in the field. METHOD: This study is divided into four sections, each discussing the benefits and limitations of a specific type of technology-based assessment: ambulatory assessment, social networks, gamification and forced-choice testing. RESULTS: The latest developments are clearly relevant in the field of psychological and educational measurement and assessment. Among other benefits, they bring greater ecological validity to the assessment process and eliminate the bias associated with retrospective assessment. CONCLUSIONS: Some of these new approaches point to a multidisciplinary scenario with a tradition which has yet to be created. Psychometrics must secure a place in this new world by contributing sound expertise in the measurement of psychological variables. The challenges and debates facing the field of psychology as it incorporates these new approaches are also discussed.


Asunto(s)
Tecnología Digital , Programas Informáticos , Humanos , Estudios Retrospectivos , Psicometría , Evaluación Educacional
15.
Psychol Methods ; 2023 Jul 06.
Artículo en Inglés | MEDLINE | ID: mdl-37410419

RESUMEN

The accuracy of factor retention methods for structures with one or more general factors, like the ones typically encountered in fields like intelligence, personality, and psychopathology, has often been overlooked in dimensionality research. To address this issue, we compared the performance of several factor retention methods in this context, including a network psychometrics approach developed in this study. For estimating the number of group factors, these methods were the Kaiser criterion, empirical Kaiser criterion, parallel analysis with principal components (PAPCA) or principal axis, and exploratory graph analysis with Louvain clustering (EGALV). We then estimated the number of general factors using the factor scores of the first-order solution suggested by the best two methods, yielding a "second-order" version of PAPCA (PAPCA-FS) and EGALV (EGALV-FS). Additionally, we examined the direct multilevel solution provided by EGALV. All the methods were evaluated in an extensive simulation manipulating nine variables of interest, including population error. The results indicated that EGALV and PAPCA displayed the best overall performance in retrieving the true number of group factors, the former being more sensitive to high cross-loadings, and the latter to weak group factors and small samples. Regarding the estimation of the number of general factors, both PAPCA-FS and EGALV-FS showed a close to perfect accuracy across all the conditions, while EGALV was inaccurate. The methods based on EGA were robust to the conditions most likely to be encountered in practice. Therefore, we highlight the particular usefulness of EGALV (group factors) and EGALV-FS (general factors) for assessing bifactor structures with multiple general factors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

16.
Front Med (Lausanne) ; 10: 1215246, 2023.
Artículo en Inglés | MEDLINE | ID: mdl-37809329

RESUMEN

Introduction: SARS-CoV-2 viral load has been related to COVID-19 severity. The main aim of this study was to evaluate the relationship between SARS-CoV-2 viremia and SNPs in genes previously studied by our group as predictors of COVID-19 severity. Materials and methods: Retrospective observational study including 340 patients hospitalized for COVID-19 in the University Hospital La Princesa between March 2020 and December 2021, with at least one viremia determination. Positive viremia was considered when viral load was above the quantifiable threshold (20 copies/ml). A total of 38 SNPs were genotyped. To study their association with viremia a multivariate logistic regression was performed. Results: The mean age of the studied population was 64.5 years (SD 16.6), 60.9% patients were male and 79.4% white non-Hispanic. Only 126 patients (37.1%) had at least one positive viremia. After adjustment by confounders, the presence of the minor alleles of rs2071746 (HMOX1; T/T genotype OR 9.9 p < 0.0001), rs78958998 (probably associated with SERPING1 expression; A/T genotype OR 2.3, p = 0.04 and T/T genotype OR 12.9, p < 0.0001), and rs713400 (eQTL for TMPRSS2; C/T + T/T genotype OR 1.86, p = 0.10) were associated with higher risk of viremia, whereas the minor alleles of rs11052877 (CD69; A/G genotype OR 0.5, p = 0.04 and G/G genotype OR 0.3, p = 0.01), rs2660 (OAS1; A/G genotype OR 0.6, p = 0.08), rs896 (VIPR1; T/T genotype OR 0.4, p = 0.02) and rs33980500 (TRAF3IP2; C/T + T/T genotype OR 0.3, p = 0.01) were associated with lower risk of viremia. Conclusion: Genetic variants in HMOX1 (rs2071746), SERPING1 (rs78958998), TMPRSS2 (rs713400), CD69 (rs11052877), TRAF3IP2 (rs33980500), OAS1 (rs2660) and VIPR1 (rs896) could explain heterogeneity in SARS-CoV-2 viremia in our population.

17.
Span J Psychol ; 15(1): 424-41, 2012 Mar.
Artículo en Inglés | MEDLINE | ID: mdl-22379731

RESUMEN

This paper describes several simulation studies that examine the effects of capitalization on chance in the selection of items and the ability estimation in CAT, employing the 3-parameter logistic model. In order to generate different estimation errors for the item parameters, the calibration sample size was manipulated (N = 500, 1000 and 2000 subjects) as was the ratio of item bank size to test length (banks of 197 and 788 items, test lengths of 20 and 40 items), both in a CAT and in a random test. Results show that capitalization on chance is particularly serious in CAT, as revealed by the large positive bias found in the small sample calibration conditions. For broad ranges of theta, the overestimation of the precision (asymptotic Se) reaches levels of 40%, something that does not occur with the RMSE (theta). The problem is greater as the item bank size to test length ratio increases. Potential solutions were tested in a second study, where two exposure control methods were incorporated into the item selection algorithm. Some alternative solutions are discussed.


Asunto(s)
Algoritmos , Inteligencia Artificial , Evaluación Educacional/estadística & datos numéricos , Modelos Estadísticos , Psicometría/estadística & datos numéricos , Humanos , Reproducibilidad de los Resultados , Diseño de Software
18.
Span J Psychol ; 14(1): 500-8, 2011 May.
Artículo en Inglés | MEDLINE | ID: mdl-21568205

RESUMEN

In computerized adaptive testing, the most commonly used valuating function is the Fisher information function. When the goal is to keep item bank security at a maximum, the valuating function that seems most convenient is the matching criterion, valuating the distance between the estimated trait level and the point where the maximum of the information function is located. Recently, it has been proposed not to keep the same valuating function constant for all the items in the test. In this study we expand the idea of combining the matching criterion with the Fisher information function. We also manipulate the number of strata into which the bank is divided. We find that the manipulation of the number of items administered with each function makes it possible to move from the pole of high accuracy and low security to the opposite pole. It is possible to greatly improve item bank security with much fewer losses in accuracy by selecting several items with the matching criterion. In general, it seems more appropriate not to stratify the bank.


Asunto(s)
Diagnóstico por Computador/estadística & datos numéricos , Evaluación Educacional/estadística & datos numéricos , Sistemas de Información/estadística & datos numéricos , Pruebas Psicológicas/estadística & datos numéricos , Psicometría/estadística & datos numéricos , Programas Informáticos , Encuestas y Cuestionarios , Algoritmos , Inteligencia Artificial , Simulación por Computador , Humanos , Lingüística , Cómputos Matemáticos , Matemática , Reproducibilidad de los Resultados
19.
Appl Psychol Meas ; 45(2): 112-129, 2021 Mar.
Artículo en Inglés | MEDLINE | ID: mdl-33627917

RESUMEN

Decisions on how to calibrate an item bank might have major implications in the subsequent performance of the adaptive algorithms. One of these decisions is model selection, which can become problematic in the context of cognitive diagnosis computerized adaptive testing, given the wide range of models available. This article aims to determine whether model selection indices can be used to improve the performance of adaptive tests. Three factors were considered in a simulation study, that is, calibration sample size, Q-matrix complexity, and item bank length. Results based on the true item parameters, and general and single reduced model estimates were compared to those of the combination of appropriate models. The results indicate that fitting a single reduced model or a general model will not generally provide optimal results. Results based on the combination of models selected by the fit index were always closer to those obtained with the true item parameters. The implications for practical settings include an improvement in terms of classification accuracy and, consequently, testing time, and a more balanced use of the item bank. An R package was developed, named cdcatR, to facilitate adaptive applications in this context.

20.
Front Psychol ; 12: 614470, 2021.
Artículo en Inglés | MEDLINE | ID: mdl-33658962

RESUMEN

Cognitive diagnosis models (CDMs) allow classifying respondents into a set of discrete attribute profiles. The internal structure of the test is determined in a Q-matrix, whose correct specification is necessary to achieve an accurate attribute profile classification. Several empirical Q-matrix estimation and validation methods have been proposed with the aim of providing well-specified Q-matrices. However, these methods require the number of attributes to be set in advance. No systematic studies about CDMs dimensionality assessment have been conducted, which contrasts with the vast existing literature for the factor analysis framework. To address this gap, the present study evaluates the performance of several dimensionality assessment methods from the factor analysis literature in determining the number of attributes in the context of CDMs. The explored methods were parallel analysis, minimum average partial, very simple structure, DETECT, empirical Kaiser criterion, exploratory graph analysis, and a machine learning factor forest model. Additionally, a model comparison approach was considered, which consists in comparing the model-fit of empirically estimated Q-matrices. The performance of these methods was assessed by means of a comprehensive simulation study that included different generating number of attributes, item qualities, sample sizes, ratios of the number of items to attribute, correlations among the attributes, attributes thresholds, and generating CDM. Results showed that parallel analysis (with Pearson correlations and mean eigenvalue criterion), factor forest model, and model comparison (with AIC) are suitable alternatives to determine the number of attributes in CDM applications, with an overall percentage of correct estimates above 76% of the conditions. The accuracy increased to 97% when these three methods agreed on the number of attributes. In short, the present study supports the use of three methods in assessing the dimensionality of CDMs. This will allow to test the assumption of correct dimensionality present in the Q-matrix estimation and validation methods, as well as to gather evidence of validity to support the use of the scores obtained with these models. The findings of this study are illustrated using real data from an intelligence test to provide guidelines for assessing the dimensionality of CDM data in applied settings.

SELECCIÓN DE REFERENCIAS
DETALLE DE LA BÚSQUEDA