Cato Institute: Dump NCLB

It is a rare occasion that I find reason to post a study from the Cato Institute. Cato has their own extremist agenda, of course, for ending ANY federal participation in state or local school funding, but that fool notion does not detract from the points made here regarding the research that has been and continues to be ignored by Spellings, Miller, and Kennedy all.

From the Courier-Journal:

The looming expiration of the federal No Child Left Behind Act has prompted a flood of commission reports, studies and punditry.

Virtually all of those analyses have assumed that the law should and will be reauthorized, disagreeing only over how it should be revised. They have accepted the law's premises without argument: that government-imposed standards and bureaucratic "accountability" are effective mechanisms for improving American education and that Congress should be involved in their implementation.

Thorough review

In this paper, we put those preconceptions under a microscope and subject NCLB to a thorough review. We explore its effectiveness to date and ask whether its core principles are sound. We find that No Child Left Behind has been ineffective in achieving its intended goals, has had negative unintended consequences, is incompatible with policies that do work, is at the mercy of a political process that can only worsen its prospects, and is based on premises that are fundamentally flawed.

We further conclude that NCLB oversteps the federal government's constitutional limits -- treading on a responsibility that, by law and tradition, is reserved to the states and the people. We therefore recommend that NCLB not be reauthorized and that the federal government return to its constitutional bounds by ending its involvement in elementary and secondary education.

Virtually every study that has weighed in on the future of the No Child Left Behind Act has taken the law's underlying principles as given.

The voluminous "Beyond NCLB: Fulfilling the Promise to Our Nation's Children" report from the Aspen Institute's Commission on No Child Left Behind is typical, declaring that "Commission members . . . were united from the outset in [their] firm commitment to . . . harness the power of standards, accountability and increased student options."

Commission members' commitment to government standards and testing was not a product of their study but a foregone conclusion. Similarly, the "ESEA Reauthorization Policy Statement," published by the Council of Chief State School Officers, promises that "if we follow through," standards-based reform "has the potential to dramatically improve student achievement and meet our education goals."

No defense of, or evidence supporting, this claim is included in the statement. Despite the widespread assumption that government standards and accontability will prove effective, it is unwise to make policy decisions affecting tens of millions of children -- and costing tens of billions of dollars -- on the basis of preconceived, unscrutinized notions.

Assessing no child left behind

NCLB's supporters began declaring the law a success within a few years of its January 2002 passage. In July 2005, for instance, when the National Assessment of Educational Progress released its most recent Trends in Academic Progress report, then-chairman of the House Committee on Education and the Workforce John Boehner, R-Ohio, asserted that "[t]hrough No Child Left Behind, we made it a national priority to improve student achievement and close achievement gaps that have persisted between disadvantaged students and their peers. The culture of accountability is taking root in our nation's schools, and student achievement is on the rise."

In January 2006, Secretary of Education Margaret Spellings was similarly effusive:

"I am pleased to report that No Child Left Behind is working. The long-term Nation's Report Card results released this past summer showed elementary school student math and reading achievement at an all-time high and the achievement gap closing."

This sort of triumphalism has continued ever since, with Secretary Spellings in May 2007 even giving NCLB credit for improving scores on NAEP U.S. history and civics exams, despite the fact that NCLB does not address those subjects:

"For the past five years, No Child Left Behind (NCLB) has focused attention and support on helping students become stronger readers. The release today by The Nation's Report Card on U.S. History and Civics proves NCLB is working and preparing our children to succeed."

Policymakers like Boehner and Spellings, who helped to craft and pass NCLB, are not the only ones who have touted the law's supposed success. In its final report, released in February 2007, the Aspen Institute's NCLB commission rang a similarly positive note, claiming that "[t]here is growing evidence that NCLB is producing . . . improved student achievement. According to NAEP, scores in mathematics increased nationwide for fourth and eighth graders from 2003 to 2005. . . . In reading, the national average of fourth graders' scores improved from 2003 to 2005."

Consider, however, that NCLB was passed in January 2002, and fourth-grade reading scores did not in fact change at all between 2002 and 2005. The one-point uptick between 2003 and 2005 only offset a one-point downtick between 2002 and 2003. Furthermore, the Aspen commission neglects to mention that eighth-grade reading scores fell by two points after 2002. At least according to NAEP scores since NCLB's passage, it seems that the law has achieved nothing of consequence.

But post-passage scores don't tell us the whole story. To judge whether the law is working, we also have to look at preexisting trends in achievement. It is quite possible, for example, that math scores were already rising, and reading scores stagnating or falling, before the law was passed and that NCLB affected neither. To have any hope of isolating NCLB's actual effect on student achievement and test score gaps, we have to compare score trends before and after the law's passage.

According to the NAEP Long-Term Trends report, fourth- and eighth-grade math scores did improve between 1999 and 2004, as did fourth grade reading scores (eighth grade reading was flat). Attributing those results to NCLB is highly problematic, however, given that the law was only enacted in January 2002 and not fully implemented until the 2005-06 school year.

But suppose NCLB really did start transforming American education after just a year or two in existence. A rough idea of its effects could then be gleaned by looking at the standard NAEP mathematics and reading results (a data set that is separate from the Long-Term Trends report mentioned earlier). The news wouldn't be good: The trends in those results are virtually unchanged.

While both 4th- and 8th-grade math scores rose between 2003 and 2005 (the only period during which score changes can be reasonably attributed to NCLB), the rate of improvement actually slowed from that achieved between 2000 and 2003, a period before the law's effects would have been felt. In reading, the results were worse, with the period covered by NCLB seeing a score decline for 8th graders and stagnation for 4th graders, following an appreciable improvement between 2000 and 2002 (before the law's passage).

The analysis above is admittedly cursory, providing only tentative evidence of NCLB's effects. In June 2006 Harvard University's Civil Rights Project released a more rigorous review of NAEP score trends before and after passage of NCLB. After comparing the trends from 1990 all the way through 2005, the study's author, Jaekyung Lee, concluded that:

NCLB does not appear to have had a significant impact on improving reading or math achievement. Average achievement remains flat in reading and grows at the same pace in math as it did before NCLB was passed. In grade four math, there was a temporary improvement right after NCLB, but it was followed by a return to the pre-reform growth rate.

NCLB does not seem to have helped the nation and states significantly narrow the achievement gap. The racial and socioeconomic achievement gap in NAEP reading and math persists after NCLB. Despite some improvement in reducing the gap in math right after NCLB, the progress was not sustained.

NCLB's attempt to scale up the alleged success of states that already had test-driven accountability programs does not appear to have worked. It neither enhanced the earlier academic improvements seen in some of those states nor transferred them to other states.

Harvard study ignored

NCLB supporters have responded to the Harvard study by ignoring it. At the time of this writing, the only reference to Lee's study on the Department of Education's Web site was its routine entry in the department's database of education research papers (the ERIC database). And although the Aspen commission lists the Harvard study in its bibliography, the commission's report does not address -- indeed, does not even mention -- Jaekyung Lee's findings.

Interestingly, the Aspen commission released a background paper of its own, investigating post-NCLB test score gaps in seven states.

The paper did not compare score trends before and after the law's passage and was not nationally representative, so it is less useful than the Harvard study, but it is notable in that it offers little support for the commission's own positive views on the effects of NCLB. The paper finds that post-NCLB changes in ethnic and other achievement gaps have been "mixed." Some gaps have shrunk, some have grown larger, others haven't changed much at all.

Another recent report that bears on NCLB's academic effects was conducted by the Northwest Evaluation Association, a nonprofit test provider that works with 2,400 school districts. Using its database of test scores from more than 300,000 students, NWEA researchers compared how much students learned over the course of the 2003–04 school year with how much they learned in 2001–02. What the researchers found was that students learned less in a year after NCLB's passage than they did before it, a result that held true for every ethnic group analyzed and for both mathematics and reading.

The NWEA's results, it should be noted, were not necessarily nationally representative -- data from only 23 states were used -- and they do not conclusively prove that NCLB was responsible for the observed decline in student learning. However, they are a further piece of evidence that NCLB has not improved American education. Those results have also largely been ignored by the people who wish to reauthorize the law.

There is one last, important componen to NCLB that might offer evidence that the law is working: NCLB requires all states to create math and reading standards and to test student mastery of them. Perhaps the results of those assessments are promising. Indeed, many states have been reporting gains on state test scores. Most recently, a June 2007 report from the Center on Education Policy found that many states have seen overall state test score improvements and shrinking achievement gaps under NCLB, a finding that Secretary Spellings declared "confirms that No Child Left Behind has struck a chord of success with our nation's schools and students. . . We know that the law is working."

Despite the seemingly rosy findings when it comes to state test results, there is more bad news than good. For one thing, the CEP study identified huge holes and inconsistencies in state data, the result of most states' having altered their standards, tests, definitions of "proficiency," and other achievement measures since NCLB was passed. Indeed, there were so many holes in the data that CEP had usable pre- and post-NCLB data for only 13 states, and only enough information to conduct full analyses for seven.

And data holes are not the only problem. Several studies have found that students' score on state tests often greatly outstrip their performance on NAEP exams, suggesting that states make success on their own tests relatively easy to achieve, compared with the more rigorous NAEP. A June 2006 University of California, Berkeley, analysis comparing scores on state tests with those on NAEP for 12 states, for instance, concluded that "state results consistently exaggerate the percentage of fourth graders deemed proficient or above in reading and math -- for any given year and for reported rates of annual progress, compared with NAEP results."

More recently, the Institute of Education Sciences equated scores on state tests in schools that administered NAEP with those schools' NAEP results. (NAEP is based on representative sampling of schools and students rather than testing every student in every school.) This revealed that most states' "proficient" levels are equivalent to NAEP's "basic"designation. That is, except in fourth grade reading, where most state proficiency levels are actually below NAEP's basic level.

Taking all these findings together, NCLB appears to have done little good, despite rhetoric from NCLB supporters to the contrary.

Indeed, if anything, there is appreciable evidence that NCLB may have slowed or even partly reversed gains achieved before its passage.