Bracey on WASL

The results of years of grassroots activity in Washington by a persistent and brave group of parents and educators has led to big changes in the assessment scene in the state of Washington. Jerry Bracey has this op-ed in the Seattle Times that, if heeded, could bring sanity to the scene:

Washington schools chief Randy Dorn's plan to replace the Washington Assessment of Student Learning is a good one, but school-testing expert Gerald W. Bracey says the specific plan to replace it with shorter, multiple-choice tests is a bad one. What Washington should pursue is a course like Nebraska's, where testing ideas originated with teachers and evolved into something we might call instruction-driven measurement. Right now, what we have is measurement-driven instruction and it is a disaster, both in Washington and in the nation at large.

By Gerald W. Bracey
Special to The Times

WASHINGTON Superintendent of Public Instruction Randy Dorn wants to dump the Washington Assessment of Student Learning. Good idea. Dorn wants to replace the WASL with shorter, multiple-choice tests. Bad idea. And he wants the tests to be more "diagnostic." Dorn's impossible dream.

First off, in a KUOW interview, Dorn said the new tests would still be valid. He cannot know that. Validity is always an empirical question. Usually, if you make a test shorter, it becomes less valid. Of course, I've oversimplified in that statement. The question really is, valid for what? It's relatively easy to judge the content validity of test items — do they measure what they claim to measure? From the released WASL items I've seen, I'm not certain those items do.

In addition, a shorter test will cover less material. And even if it covers as much, rising test scores don't mean much. About 20 years ago, some researchers tracked test scores in a district that had changed tests. In the first year after the change, scores plummeted. Then they gradually rose over a four-year period back to where they had been. Then the sneaky researchers came in and gave the kids the original test, the one that four years earlier had been the official test. Test scores plummeted. The message: Kids learn what's on the test, but that learning doesn't generalize and generalization of learning, after all, is the point of education.

But the bigger validity question is: Does the test make any difference? Are college professors more pleased with students who have passed the test? Are employers? The answer is a resounding, "We don't know." States are afraid to ask this question because, if the answer comes up, "No," they will be seen to have spent millions, even billions, of dollars for nothing. But informal studies by journalists have yet to turn up a positive instance. So forget all the fear-mongering rhetoric that we need these tests in order to compete with China and in the global economy.

The political risk in trying to answer that validity question is too great. In Virginia, the state Board of Education assembled a first-rate Technical Advisory Committee (TAC) of well-known testing experts from all over the nation. The TAC repeatedly advised the board to conduct an evaluation to determine if the testing program made a difference in the quality of graduates. The board dissolved the TAC.

As a consequence, we have, in all states, meaningless passing scores. They are both arbitrary and, often, politically driven. The Virginia Board of Education, and some other boards, has had to lower passing scores because they failed politically unacceptable numbers of students. There is no technically sound, scientifically defensible method for setting a passing score.

Passing scores can be used for all kinds of political mischief. In Texas, the scores were set initially so that many blacks and Hispanics failed, but not by much. When these groups did better (and raising test scores is not rocket science; raising achievement is), many more passed and it looked like, well, a miracle.

We now know it was a mirage, all smoke and mirrors. A passing score tells you only how many kids jumped over the barrier you put in their path. It does not tell you how high they jumped.

The plan as presented by Dorn perpetuates two false dichotomies. The first, represented most clearly by the federal No Child Left Behind law, is that you are proficient or you are left behind. Any educator, psychologist or cognitive scientist knows this is absurd. If the passing score on a test is 80 and your child gets 79, is he or she "left behind?" The question answers itself.

The worst of the false dichotomies is that Dorn's proposal perpetuates the notion that testing is something you do after you stop teaching. The reasons this notion took hold are too complex to detail here, but it need not be the case.

What Washington should pursue is a course like Nebraska's, where testing ideas originated with teachers (remember them?) and evolved into something we might call instruction-driven measurement. Right now, what we have is measurement-driven instruction and it is a disaster, both in Washington and in the nation at large.

Gerald W. Bracey is an independent researcher and writer living in Port Townsend. He headed up the Virginia testing programs for nine years and those in Cherry Creek Schools in Colorado for five. He is the author of "Put to the Test: An Educator's and Consumer's Guide to Standardized Testing" and of the forthcoming "Getting Out of Education Hell: Moving Beyond 50 Years of Failed Punish-the-Schools Reforms."