Bitter Lessons from Chasing Better Tests

In a New York Times Op-Ed (22 March 2009), E. D. Hirsch Jr. argued, "We do not need to abandon either the principle of accountability or the fill-in-the-bubble format. Rather we need to move from teaching to the test to tests that are worth teaching to."

This refrain parallels the contradictory messages coming from the Obama administration that claims supporting a change to the culture of testing in NCLB, but then argues for better testing. 

Secretary of Education Duncan, in a speech about NCLB reauthorization (24 September 2009), acknowledged concerns about testing, but immediately took the same position as Hirsch: "Until states develop better assessments--which we will support and fund through Race to the Top--we must rely on standardized tests to monitor progress--but this is an important area for reform and an important conversation to have."

A better test is all we need?

Nothing could be further from the truth. We have been searching for the perfect test for a century now in education, and that has led us to the "reliability" and "validity" traps. In other words, technically Hirsch is right, but authentically, a test will never be anything more than a pale reflection of what any student knows.

Let's consider first something we claim is much less important than education (although that claim may be tenuous)--football. 

It would be quicker and cheaper to design a multiple-choice test that is well designed (high reliability and validity) and replace all football games, including playoffs, in order to determine high school, college, and professional championships.

Imagine Friday nights across the US with teams lined up in desks bubbling frantically to reach the state championship!

But we would never stand for this.

Somehow, however, this is exactly what we call for again and again as a solution for improving the education of our children in a free society--higher standards, more testing, and greater accountability.

The flaws of testing and accountability are failing our students and our society. Those failures include:

• A culture of testing perpetuates the misconception that teaching and learning somehow exist within an educational vacuum--as if the lives of children are suspended when they walk through the doors of school. Accountability principles that hold people accountable for conditions beyond their control will always fail, but that is what we do in education.

• Test data are never a pure representation of learning. A test score is impacted by effort of the student, quality of the test, conditions of the testing day and time, and a number of other factors that have nothing to do with learning. Multiple-choice tests, as well, are always impacted by guessing.

• Teaching to a test and seeing learning as a static body of knowledge are the lowest possible visions of teaching and learning. A basic argument of John Dewey that we have failed to see in this country is that education can never fully anticipate what any student needs to know, but schools can prepare children to be expert learners, something that a multiple-choice test as a goal or a measurement can never capture.

• The best test possible can only be an approximation of learning. Any test must reduce what is being measured and depend on statistical approximations to create the perception that we are measuring something much larger than we are. As James Popham has argued, a test provides us data from which to make inferences, but at best those inferences are approximate--unless you make the test so direct and simplistic as to create a situation where we are collecting data that means almost nothing.

• Once we make any test sacred, that test replaces the larger and more authentic goals of school. Instead of reading, students take test-prep to be tested on reading; instead of writing, students take test-prep to be tested on writing. . .and the list goes on. Testing and teaching to a test are always asking less of our students.

Correlations, validity, and reliability are powerful in the world of statistics, and they sound impressive when we call for making our schools more rigorous. But in the end, teaching and learning are human endeavors that are messy, chaotic, and nearly impossible to reduce to simple measurements that truly mean anything of value.

Calls for better tests are merely digging a deep hole even deeper. The solution to better schools is not better tests but inviting students to read, write, and think for hours each day throughout the school year, never lifting their heads or pencils to get ready for a test.

But we must place a more challenging and engaging school day--one not concerned with tests--within a social commitment to the lives of children outside of school as well. Hungry children care little about tests or reading--nor should they.

If we want better and even eager readers in this country, we don't need better reading tests, we need students who are actually reading; if we want highly literate and critically thinking young people to enter and rejuvenate our free land, spending much of their schooling trying to get a higher test score is not the solution, regardless of the test they are chasing.

And if we persist in calling for better tests while ignoring the burden of poverty on children's lives, those children will continue to learn the bitter lesson about what we truly value as a people.

[Originally published in a slightly different form at OpEdNews.com on 27 August 2010]