The dependability of the general factor of intelligence: Why g is not a first principal component

Abstract

In a replication of a psychometric study by Floyd, Shands, Rafael, Bergeron & McGrew (2009), generalizability theory was used to isolate and compare three different sources of error in general-factor loadings: the test battery size, test battery composition, factor-extraction technique, and their interactions. Subtests from the Minnesota Study of Twins Reared Apart (MISTRA) were randomly selected to form independent and overlapping batteries of 2, 4 and 8 tests in size. Eight “probe” tests were then inserted into each of the batteries, and principal components analysis, principal factors analysis and maximum likelihood estimation were used to obtain their general-factor loadings. Results of the generalizability theory analysis initially indicated that the general-factor loadings were more dependable than in Floyd et al. (2009), but subsequent examination revealed this outcome to be largely a function of the greater diversity of probe tests selected in the present study. As in Floyd et al. (2009) the characteristics of the probe tests constituted the largest source of variance in general-factor loadings, followed by the effects of psychometric sampling, the factor-extraction method and test battery size. Our interpretation of these results differs from that by Floyd et al. (2009), however, in consideration of the standard errors of the factor loadings, and the correlation of general-factor scores to those for an estimated “true g”. These indices demonstrate that general-factors from small non-hierarchical test batteries are not accurate enough estimates of g for the purposes of theoretical research, in particular when they are derived from principal components analysis.