SAT: No Benefit, Big Harm

From Stan Katz at the Chronicle Review:

The Center for Studies in Higher Education at UC Berkeley has quietly been doing some of the most significant higher education policy research I know of. Spurred by the vote in California in 1996 to end affirmative action in the state’s colleges and universities, and blessed with a vast database on student performance in the huge UC system, researchers at CSHE have been asking the Big Question about college admissions — what are the most reliable objective predictors of success in college for high-school students? When the first results were published several years ago, I found them both troubling and significant, for the researchers found that the best predictor of college success (measured by course grades) was grades in high-school college preparatory courses. They also argued that using grades as the criterion for
admissions had the least adverse effect on the success of poor and minority college applicants. And, significantly, high-school grades were not just predictive of success in the freshman year, but of cumulative GPA over four years, and four-year graduation rates.

An important follow-up report by Saul Geiser of CSHE was published a few days ago. It is entitled Back to the Basics: In Defense of Achievement (and Achievement Tests) in College Admissions (click here), and it is a strong confirmation of the earlier findings. It is, in particular, a criticism of the usefulness of the SAT I examination as a general predictor of college success. But it is even more critical of the assumption
that the SAT I will identify promising poor and minority students more adequately than high-school GPA — the theory was that if one could test for ability to learn rather than actual learning outcomes, one could discount the negative impact of the poor secondary schools to which the poor and minorities were assigned. But the California data, we are told, shows that the achievement test adds nothing to the predictive value of course grades and achievement tests. They show, in fact, that the SAT “has a more adverse impact on poor and minority applicants than high-school grades and achievement tests.” The report finds that AP courses (though not AP scores) are “good predictors of student success in college,” but AP courses are limited by their availability across high-school systems nationally.

I am not a testing expert, but I have been following the CSHE data analysis with great interest and increasing conviction. I share the center’s concern both with accuracy of prediction of college success, and the relationship of prediction to equity in admissions. These data seem to me to add tremendous force to the growing lack of confidence in the reliability of aptitude tests. And most important, in an era in which affirmative action in college admissions seems to be rapidly disappearing, tests that disadvantage the poor seem particularly objectionable, given the relationship between higher education and life outcomes. I have to confess that the CSHE conclusions reinforce my teacher’s bias for subject-matter mastery as the criterion of achievement. And, if nothing else, it should be reassuring to students to know that some fair measure of what they have actually learned will be the primary criterion upon which admissions decisions are based — rather than the black box of achievement tests. If nothing else, we should be grateful to CSHE for providing a solid statistical basis for public discussion of what is too often more an emotional and political than a policy debate.