An Extension of the Internal/External Frame of Reference Model: A Response to Bong (1998).

Bong (1998) extended the internal/external frame of reference (I/E) model by attempting to operationalize the internal and external comparison processes that are central to the model and expanding the range of academic self-concept domains. Bong concluded that the "I/E model failed to receive clear support" (p. 102) in relation to predictions that she derived from her extension of the original model. Our critical evaluation and reanalysis, however, reveals problems in the operationalization of the internal and external comparisons, the rationale for post hoc confirmatory factor analysis models and, thus, the original conclusions. Our reanalysis, however, provide strong support for the original I/E model, some support for the separation of the internal and external comparison processes, and good support for a new extension to incorporate a wider range of academic domains. In particular, for these largely Hispanic-American students, Spanish achievement and self-concept were very distinct from achievements and self-concepts in all other school subjects. Accomplishments in Spanish provided one basis for evaluating accomplishments in other school subjects. In contrast, achievements and self-concepts in these other school subjects could be explained in terms of higher-order Verbal achievement and self-concept factors and higher-order Math achievement and self-concept factors.

Longitudinal multilevel models of the big-fish-little-pond effect on academic self-concept: counterbalancing contrast and reflected-glory effects in Hong Kong schools.

Longitudinal multilevel path models (7,997 students, 44 high schools, 4 years) evaluated effects of school-average achievement and perceived school status on academic self-concept in Hong Kong, which has a collectivist culture with a highly achievement-segregated high school system. Consistent with a priori predictions based on the big-fish-little-pond effect (BFLPE), higher school-average achievements led to lower academic self-concepts (contrast effect), whereas higher perceived school status had a counterbalancing positive effect on self-concept (reflected-glory, assimilation effect). The negative BFLPE is the net effect of counterbalancing influences, stronger negative contrast effects, and weaker positive assimilation effects so that controlling perceived school status led to purer--and even more negative--contrast effects. Attending a school where school-average achievement is high simultaneously resulted in a more demanding basis of comparison for one's own accomplishments (the stronger negative contrast effect) and a source of pride (the weaker positive assimilation effect).

Multiple Evaluations of Grant Proposals by Independent Assessors: Confirmatory Factor Analysis Evaluations of Reliability, Validity, and Structure.

External assessor ratings of the quality of the research team and of the proposed research were evaluated for proposals submitted to the highly competitive Australian Research Council large grants program. The structure of the responses by multiple assessors was evaluated using a series of nested models that are variations of the traditional congeneric, tau-equivalent, and parallel measurement models. The estimated reliability based on four independent assessors was modest but comparable to other peer review research: .494 for proposal ratings, .634 for researcher ratings, and .704 for a total assessment. Mean ratings (averaged over multiple assessors) of the proposal and the research team were highly correlated (r = .85), suggesting a lack of differentiation and a substantial method/halo effect in ratings by the same assessor. Confirmatory factor analysis models supported this suggestion, representing this method effect as correlated uniquenesses. Tests of whether proposal and researcher ratings reflected one or two latent variables, based on the internal structure of the assessment ratings and relations with external criteria (academic rank, prior funding, and publications), supported a one-factor solution.

Structure, stability, and development of young children's self-concepts: a multicohort-multioccasion study.

A new, individual administration procedure for assessing multiple dimensions of self-concept for young children 5-8 years of age (Marsh, Craven, & Debus) was the basis of this study. We expanded this application in a multicohort-multioccasion (MCMO) study that provides simultaneous multicohort comparisons (cross-sectional comparisons of different age cohorts) and longitudinal comparisons of the same children on multiple occasions. There was reasonable support for predictions that reliability, stability, factor structure, and the distinctiveness of the SDQ factors would improve with age (a between-group age cohort comparison) and from 1 year to the next (a longitudinal comparison), and that small gender differences were reasonably stable over age. Consistent with the proposal that children's self-perceptions become more realistic with age, Time 1 (T1) teacher ratings were more highly correlated with student self-ratings at T2 than T1 and contributed to the prediction of T2 self-concept beyond effects mediated by T1 self-concepts. The results support and expand the surprisingly good support for the multidimensionality of self-concept responses for very young children using this procedure.

Top-down, bottom-up, and horizontal models: the direction of causality in multidimensional, hierarchical self-concept models.

A new structural equation modeling approach to questions of the direction of causal flow between global and specific multidimensional measures of self-concept (SC) in two 2-wave, longitudinal studies demonstrated that (a) higher order factors were unable to explain relations among first-order factors at Time 1 (T1), at Time 2 (T2), or between T1 and T2; (b) T1 global SC had little effect on specific SC factors at T2 (a top-down model), but specific factors at T1 had even less effect on T2 global SC (a bottom-up model); and (c) many specific factors were more stable than global factors, but higher order factors were most stable. Results provide little support for top-down, bottom-up, or reciprocal models, instead arguing for a horizontal model in which each T2 SC factor is primarily a function of the matching T1 SC. This casts further doubt on the usefulness of hierarchical representations of SC.

Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis.

We evaluated whether "more is ever too much" for the number of indicators (p) per factor (p/f) in confirmatory factor analysis by varying sample size (N = 50-1000) and p/f (2-12 items per factor) in 35,000 Monte Carlo solutions. For all N's, solution behavior steadily improved (more proper solutions, more accurate parameter estimates, greater reliability) with increasing p/f. There was a compensatory relation between N and p/f: large p/f compensated for small N and large N compensated for small p/f, but large-N and large-p/f was best. A bias in the behavior of the χ(2) was also demonstrated where apparent goodness of fit declined with increasing p/f ratios even though approximating models were "true". Fit was similar for proper and improper solutions, as were parameter estimates form improper solutions not involving offending estimates. We also used the 12-p/f data to construct 2, 3, 4, or 6 parcels of items (e.g., two parcels of 6 items per factor, three parcels of 4 items per factor, etc.), but the 12-indicator (nonparceled) solutions were somewhat better behaved. At least for conditions in our simulation study, traditional "rules" implying fewer indicators should be used for smaller N may be inappropriate and researchers should consider using more indicators per factor that is evident in current practice.

Physical Self Description Questionnaire: stability and discriminant validity.

The Physical Self-Description Questionnaire (PSDQ) is a multidimensional, physical self-concept instrument designed to measure 11 scales: Strength, Body Fat, Activity, Endurance/Fitness, Sports Competence, Coordination, Health, Appearance, Flexibility, Global Physical Self-concept, and Global Esteem. High school students completed the PSDQ on four occasions over a 14-month period. Across the 11 PSDQ scales, the internal consistency at each occasion was good (median alpha = .92) and the stability over time varied from median r = .83 for a 3-month period to median r = .69 for the 14-month period. The data were used to demonstrate the application of confirmatory factor-analysis models of multitrait-multimethod (MTMM) data (with occasions as the multiple methods), which supported the discriminant validity of the PSDQ scales. Augmented MTMM models that included two field tests of cardiovascular endurance provided additional support for the construct validity of PSDQ responses and interpretation of the MTMM models. These results contribute to the growing body of support for the construct validity of physical-self-concept responses and illustrate the application of CFA MTMM models.

Positive and negative global self-esteem: a substantively meaningful distinction or artifactors?

Global self-esteem based on M. Rosenberg's (1965) scale is typically treated as a unidimensional scale. However, factor analyses suggest separate factors associated with positively and negatively worded items, and there is an ongoing debate about the substantive meaningfulness of this distinction. Confirmatory factor analysis (CFA) was used to evaluate alternative 1- and 2-factor models and to test hypotheses about how the factors vary with reading ability and age. Responses based on the National Longitudinal Study of 1988 (S.J. Ingles et al., 1992) reflected a relatively unidimensional factor and method effects associated with negatively worded items. Such effects are common in rating scale responses, and this CFA approach may be useful in evaluating whether factors associated with positively and negatively worded items are substantively meaningful or artifactors.

Predicting self-esteem from perceptions of actual and ideal ratings of body fatness: is there only one ideal "supermodel"?

Silhouette ratings of actual and ideal body fat (N = 258 high school students) were related to 4 self-concept scales (Body Fat, Physical Appearance, General Physical, Esteem) and objective body composition measures in order to test predictions from actual-ideal discrepancy models. Actual-ideal discrepancy scores were more strongly related to self-concept than actual scores alone, thus supporting the traditional discrepancy model. However, multiple regression analyses demonstrated that more sophisticated actual-ideal discrepancy models that considered discrepancies in either direction (feeling too fat or too thin) did significantly better. Because the traditional discrepancy model is a special case of this more general discrepancy model, the more general model should have broad applicability. Important aspects of this task were (a) that discrepancies were evaluated along a continuum in which the ideal was not one of the endpoints of the continuum, (b) that there was reasonable variation in ideal ratings, and (c) that positive and negative discrepancies were reported.

The multidimensional structure of physical fitness: invariance over gender and age.

This investigation extends the factor analytic approach pioneered by Fleishman (1964) by incorporating subsequent developments in the application of confirmatory factor analysis and the physical fitness literature. Specifically, it tests the ability of an a priori factor structure of physical fitness to fit (i.e., account for) data based on 25 indicators of fitness (field exercises, technical measures, and laboratory measures) for 2,817 boys and girls aged 9, 12, and 15. An eight-factor (Cardiovascular Endurance, Explosive/Dynamic Strength, Static Strength, Flexibility/Joint Mobility, Blood Pressure, Lung Function, Body Girth, and Skinfold) model derived from previous research fit the data well for each of the six age/gender groups, considered separately. Based on tests of factorial invariance, factor loadings and factor correlations were reasonably invariant across the six groups. This important finding indicates that all 25 indicators are equally valid for boys and girls aged 9, 12, and 15 and that the multidimensional structure of physical fitness generalizes over gender and age.

Confirmatory Factor Analysis of Multitrait-Multimethod Self-concept Data: Between-group and Within-group Invariance Constraints.

The purposes of the present investigation are to demonstrate extensions of the confirmatory factor analysis approach (CFA) to multitrait-multimethod (MTMM) data and to evaluate self-other agreement on multiple dimensions of self-concept. Two studies evaluated the ability of significant others to infer multiple dimensions of self-concept of university students in Australia (n = 151) and in Canada (n = 941). In both studies self-other agreement on the 13 Self Description Questionnaire 111 (SDQIII) factors was higher than reported in previous research. Traditional and CFA approaches to MTMM data demonstrated convergent and discriminant validity. An extension of the CFA-MTMM approach examined within-group invariance of responses by self and others and between group invariance across Australian and Canadian responses. Between-group invariance tests support the total invariance of CFA-MTMM solutions across Australian and Canadian responses. The imposition of invariance constraints produced substantially more parsimonious models and apparently offers one practical remedy to problems of ill-defined solutions that have plagued the CFA approach to MTMM data.

Overcoming Problems in Confirmatory Factor Analyses of MTMM Data: The Correlated Uniqueness Model and Factorial Invariance.

The general model typically used in the confirmatory factor analysis (CFA) approach to multitrait-multimethod (MTMM) data is plagued with methodological problems and frequently results in improper or unstable solutions. Here we reanalyze data from a previously published study, demonstrating that this model may lead to inappropriate interpretations even when it does converge to a proper solution, and describe safeguards against this occurrence. The results support the correlated uniqueness model, diagnostic tests of the validity of CFA-MTMM solutions, the inclusion of external validity criteria in the MTMM design as described by Marsh (1988; 1989; Marsh & Bailey, 19911, and the application of factorial invariance to test the stability of CFA-MTMM solutions. More generally, we demonstrate the flexibility of the CFA-MTMM approach for testing a variety of construct validity issues.

Differentiated additive androgyny model: relations between masculinity, femininity, and multiple dimensions of self-concept.

Masculinity (M) and femininity (F) were related to multiple dimensions of self-concept in 2 large studies. Androgyny theory predicts that M and F contribute positively to self-concept, but the effect of F has been nil in research using global self measures. Here, consistent with the new differentiated additive model, the relative contribution of M and F varied substantially, depending on the area of self-concept; F contributed more positively to self-concept facets that were more stereotypically feminine. Support for the model was consistent across self-responses and responses by significant others (Study 1) and across 5 age groups in early-to-middle adolescence (Study 2).

The Factorial Invariance of Responses by Males and Females to a Multidimensional Self-Concept Instrument: Substantive and Methodological Issues.

The factorial invariance of responses by preadolescent males and females to a multidimensional self-concept instrument was examined for responses to the Self Description Questionnaire (SDQ). Sets of responses by 500 males and by 500 females were each randomly divided in half to form four groups (M1, M2, Fl, and F2). The factorial invariance of an a priori structure demonstrated the replicability of the structure across cross-validation groups (M1 and M2, F1 and F2) and the generality of the structure across sex (M1 and Fl, M2 and F2). Additional a posteriori structures that better fit the data were derived on the basis of the initial analyses, but the estimated values of the new parameters were not invariant across cross-validation groups or across sex. This suggests that some of the improved fit was illusory and due to capitalizing on chance. However, factor loadings and factor correlations were invariant across sex for a priori and a posteriori structures. Hence the results support the replicability of SDQ factor structure across cross-validation samples and its generality across responses by preadolescent males and females.

The Multidimensionality of the Rotter I-E Scale and its Higher-order Structure: An Application of Confirmatory Factor Analysis.

Rotter (1966, 1975) concluded that responses to his Internal-External (I-E) scale were unidimensional, or at least that one general factor explained most of the variance in the total score. The purpose of the present investigation is to examine the factorial structure of the original Rotter scale. A review of 20 published factor analyses indicated that the scale is not unidimensional, that not even two or three factors may be able to adequately explain responses to the scale, and that six distinguishable factors have been identified. Confirmatory factor analyses indicated that a five-factor model provided an adequate fit to data from the present investigation, but that alternative models with fewer factors or a simpler structure did not fit the data as well. However, a single higher-order factor was able to explain much of the variance in the lower-order factors and also provided a reasonable fit to the data. Hence, while there is strong evidence against the unidimensionality of the Rotter scale, the findings suggest that the first-order factors do define a single higher-order construct that may represent the generalized IE construct that Rotter originally hypothesized. Nevertheless, the continued reliance of locus of control research on the Rotter scale and on global measures of the construct may be counterproductive.

Masculinity, Femininity and Androgyny: Their Relations With Multiple Dimensions of Self-Concept.

Masculinity (M) and femininity (F) were related to multiple dimensions of self-concept in responses from 962 high school students. Androgyny theory predicts that both M and F will contribute positively and uniquely to self-concept, but previous research, typically relying on undifferentiated self-concept measures, has found the unique contribution of F to be nil. In contrast the present investigation found that M and F each contributed positively and uniquely to the prediction of well differentiated facets of self-concept. Consistent with a new model to explain MF/self-concept relations, the differentiated additive model, the relative contribution of M and F varied substantially depending on the area of self-concept; F contributed more positively to the self-concept facets for which girls had higher self-concepts than boys, and in some areas the contribution of F was more positive than the contribution of M. Contrary to predictions for interactive androgyny models and the sex-typed model, M-by-F interactions were not significant, and the effects of M, F and M-by-F did not depend on gender. The social desirability of MF items, whether they were negatively or positively valued, was more highly correlated with the self-concept responses than whether the items were M or F.

The Structure of Masculinity/Femininity: An Application of Confirmatory Factor Analysis to Higher-Order Factor Structures and Factorial Invariance.

The purpose of this study is to examine the factor structure of responses to the masculinity/femininity (MF) scale of the Comrey Personality Scales for males (N = 366) and females (N = 384). The study also illustrates the use of confirmatory factor analysis for testing hierarchical factor structures and factorial invariance. The MF scale consists of M and F items designed to measure five distinguishable components of MF. Neither a one-factor solution with a single bipolar MF factor, nor a two-factor solution with separate M and F factors, was able to fit the data. However, a five-factor solution derived from the design of the MF scale provided an adequate fit, and the relationships among the five first-order factors could be explained by a single higher-order, bipolar MF factor. Contrary to the findings and proposals from androgyny research, when separate masculinity and femininity constructs were defined, the correlation between the constructs was nearly - 1.0. The findings argue that MF is a multifaceted, hierarchical construct instead of either a single bipolar MF construct, or separate M and F factors.

Prior Subject Interest, Students' Evaluations, And Instructional Effectiveness.

Students' Prior Subject Interest in a course showed similar correlations with student ratings of instructional effectiveness in two university settings (N = 1102 classes). Correlations between Prior Subject Interest and different dimensions of instructional effectiveness varied from approximately zero to .44. Though these correlations were not high, Prior Subject Interest predicted student ratings better than any of 15 other student/course/instructor characteristics considered (e.g., Expected Grade, Class Size, Workload/Difficulty, Teacher Rank). Instructor self-evaluations of their own teaching effectiveness (N = 329 classes) were also positively correlated with both their own and their students' perceptions of Prior Subject Interest; the dimensions that were most highly correlated -- particularly Learning/Value -- were the same as observed with student ratings. Since both student and instructor self evaluations were similarly related to Prior Subject Interest, it appears that this variable actually affects instructional effectiveness in a way that is accurately reflected in the student ratings.