RESUMO
The present study extended the p-optimality method to the multistage computerized adaptive test (MST) context in developing optimal item pools to support different MST panel designs under different test configurations. Using the Rasch model, simulated optimal item pools were generated with and without practical constraints of exposure control. A total number of 72 simulated optimal item pools were generated and evaluated by an overall sample and conditional sample using various statistical measures. Results showed that the optimal item pools built with the p-optimality method provide sufficient measurement accuracy under all simulated MST panel designs. Exposure control affected the item pool size, but not the item distributions and item pool characteristics. This study demonstrated that the p-optimality method can adapt to MST item pool design, facilitate the MST assembly process, and improve its scoring accuracy.
RESUMO
How to compute and report subscores for a test that was originally designed for reporting scores on a unidimensional scale has been a topic of interest in recent years. In the research reported here, we describe an application of multidimensional item response theory to identify a subscore structure in a test designed for reporting results using a unidimensional scale. This research also dealt with the problem of planned missing data due to low levels of item overlap among multiple test forms. Furthermore, we provided evidence for the generalizability of the multidimensional structure using multiple forms of the same test. We also compared the subscores from multiple groups to show the usefulness of the subscores. The research provides evidence that subscores can be identified and produced to provide useful information about different constructs for multiple examinee groups even though the test data were well fit by a unidimensional model.
RESUMO
This study investigates the role of parenting in adolescent drug use for black, white and Latino adolescents. Parenting has been consistently identified as a crucial factor in drug use by adolescents. This study uses data from the National Longitudinal Study of Adolescent Health. Results show that parenting has a significant effect on drug use for these adolescents. The relationship between parenting and drug use is more strongly negative for the Latino adolescents, than for black and white adolescents. This indicates that greater parental warmth and family acceptance exert a stronger impact in reducing drug use for Latino adolescents than is the case for the black and white adolescents.
Assuntos
População Negra/psicologia , População Negra/estatística & dados numéricos , Hispânico ou Latino/psicologia , Hispânico ou Latino/estatística & dados numéricos , Poder Familiar , Papel (figurativo) , Transtornos Relacionados ao Uso de Substâncias/epidemiologia , Transtornos Relacionados ao Uso de Substâncias/psicologia , População Branca/psicologia , População Branca/estatística & dados numéricos , Adolescente , Comportamento do Adolescente/psicologia , Estudos Transversais , Demografia , Feminino , Humanos , Masculino , Relações Pais-FilhoRESUMO
This study provided initial validity evidence for multidimensional measures of coaching competency derived from the Coaching Competency Scale (CCS). Data were collected from intercollegiate men's (n = 8) and women's (n = 13) soccer and women's ice hockey teams (n = 11). The total number of athletes was 585. Within teams, a multidimensional internal model was retained in which motivation, game strategy, technique, and character building comprised the dimensions of coaching competency. Some redundancy among the dimensions was observed. Internal reliabilities ranged from very good to excellent. Practical recommendations for the CCS are given in the Discussion section.
Assuntos
Atitude , Competência Profissional , Esportes , Adolescente , Adulto , Feminino , Hóquei , Humanos , Masculino , Meio-Oeste dos Estados Unidos , Futebol , Inquéritos e QuestionáriosRESUMO
This study extended validity evidence for multidimensional measures of coaching competency derived from the Coaching Competency Scale (CCS; Myers, Feltz, Maier, Wolfe, & Reckase, 2006) by examining use of the original rating scale structure and testing how measures related to satisfaction with the head coach within teams and between teams. Motivation, game strategy, technique, and character building comprised the dimensions of coaching competency. Data were collected from athletes (N = 585) nested within intercollegiate men's (g = 8) and women's (g = 13) soccer and women's ice hockey (g = 11) teams (G = 32). Validity concerns were observed for the original rating scale structure and the predicted positive relationship between motivation competency and satisfaction with the coach between teams. Validity evidence was offered for a condensed post hoc rating scale and the predicted relationship between motivation competency and satisfaction with the coach within teams.
Assuntos
Hóquei , Conhecimento , Futebol , Adolescente , Adulto , Comportamento do Consumidor , Feminino , Humanos , Masculino , Modelos Teóricos , Reprodutibilidade dos Testes , Inquéritos e QuestionáriosRESUMO
The authors describe the development process for a portfolio that programs may use for evaluating psychiatry residents. A committee carefully developed the portfolio requirements for 13 necessary skills in psychiatry. Psychiatrists were trained to rate portfolios. Review of the development and training process suggested that portfolios provided information both on individual residents and on the residency program itself. The process requires orientation of both residents and faculty, attention to time demands, and availability of examples and resources.
RESUMO
Residency educators are identifying approaches to measure resident competence. Portfolios are well suited since they require work already completed as part of patient care where competency must be demonstrated. This paper describes assessment of the reliability and validity of portfolios in a psychiatry residency program. This was a cross-sectional study across 4years of residency education. Using guidelines, 18 residents assembled portfolios containing five entries chosen from 13 skills. Trained raters scored the portfolios. Residents and faculty were interviewed about their perceptions. Generalizability results indicated five entries and two raters were sufficient for relative decisions. Six entries or a third rater would be sufficient for absolute decisions. Portfolio scores tended to improve with years of training and correlated with psychiatric knowledge but not clinical performance. Residents and faculty identified benefits to assembling a portfolio. Portfolios incorporate tasks embedded in the residency to provide evidence of resident competency. The results support that the score is reliable and valid.