Of the three subtests of the GRE (verbal, quantitative and analytical) and the GRE advanced test in psychology, only the analytical subtest predicted any aspect of graduate success beyond the first-year grade point average (GPA), and this prediction held for men only. The verbal subtest and psychology test predicted first-year GPA, but this prediction vanished by the second year's GPA.

"With these exceptions, the GRE scores were not useful as predictors of various aspects of graduate performance, including ratings by primary advisers of analytical, creative, practical, research and teaching abilities by primary advisers and ratings of dissertation quality by independent faculty readers," said Wendy M. Williams, associate professor of human development at Cornell University.

Williams and her colleague, Robert J. Sternberg of Yale University -- both experts on measures and theories of intelligence -- reported their findings in the June issue of American Psychologist (Vol. 52, No. 6, pp. 630-641).

The researchers strongly suspect that the GREs may prove to lack validity in predicting performance in other fields as well.

"We know from other researchers' work that the GREs also have failed to predict success in the field of physics, and we suspect that the GREs will fail to prove predictive for the humanities as well," Williams said.

"Instead of relying so heavily on the GREs -- and many applicants aren't even considered if their GRE scores are not in the top group -- we need to develop and use tests that measure meaningful performances in specific areas. The GREs, including the one specifically for psychology, do not assess many of the types of abilities required for succeeding as a professional psychologist," Williams said.

She also pointed out that applicants from less privileged backgrounds, who are not as likely to do as well on the GRE as applicants from good preparatory schools, lose out even though they may have the appropriate skills for the profession they desire. "Graduate programs rely so heavily on GREs to make their initial cuts, many well-qualified applicants who are strong in the appropriate areas aren't even being considered. This is a huge disservice to the applicants, the graduate programs and society at large."

The researchers set out to test the validity of the GRE, working within the broader framework of the triarchic theory of human intelligence. The triarchic theory distinguishes academic or analytical abilities from creative and practical abilities.

"Academic-analytical abilities are used when one analyzes, compares and contrasts, evaluates, judges or critiques," said Sternberg, who has published widely on the theories of intelligence. "Creative abilities are used when one invents, discovers, supposes, hypothesizes or theorizes. Practical abilities are used when one applies, uses or implements."

To assess the validity of GREs in predicting success or failure of graduate students, the researchers asked 40 faculty members of psychology at Yale to provide ratings on five scales of the 166 graduate students they had had since 1980. In addition, the researchers looked at GPAs of students in their first and second years of graduate training and overall evaluations of dissertations by outside, independent raters.

When the researchers looked at GRE scores and GPAs, they did find a marginal relationship between the scores and grades in the first year of graduate study. When they looked in more detail at the GRE subtests and the genders separately, they found only one of them (the analytical test score) successfully predicted more consequential evaluations of student performance (dissertation reader ratings) -- but this was only true for men. For women, there was no prediction.

"This study suggests the need to reflect on the use of tests before they become firmly -- and, as it sometimes seems, irrevocably -- entrenched. Too often, we believe, the use of a test becomes self-perpetuating, without serious attempts to verify its effectiveness," the psychologists wrote. "We believe that our results underscore the need for serious validation studies of the GRE, not to mention other admissions indexes, against measures of consequential performances, whether of students or of professionals."

Next, Williams hopes to look at GRE scores of men and women in the social sciences, natural sciences and humanities. GREs are developed by the College Board of the Educational Testing Service.

The study was supported in part by the U.S. Department of Education.

-30-

-------------------------------- Cornell University News Service 324 Judd Falls Road Ithaca, NY 14853 607-255-4206 phone 607-255-5373 fax mailto:cunews@cornell.edu http://www.news.cornell.edu

Note: If no author is given, the source is cited instead.