NY Test Gains Reason for Suspicion?

Could this be another version of the latest FCAT fiasco? It would seem reason enough for the heroic Bloomberg and Klein to put down their trumpets long enough to have a look. From NY Times:

We know the script. It opens with a Greek chorus lamenting how poorly students are reading. A pedagogic hero — a new chancellor or state commissioner — appears on the scene with a fresh quiver of weapons and schedules improved tests.

The results come back, and — alakazam! — achievement surges, and our hero is hailed as rescuer of the school system. That is, until tests in later years reveal that students are back to about where they were.

Such a pattern has stamped the history of standardized testing. And so the heartening results last month on the annual reading tests in New York State and New York City, and the results on the math tests announced yesterday, should be taken in perspective.

Officials trumpeted these results. In reading, the city’s proportion of passing eighth graders — for years the subject of hand-wringing — rose a breathtaking 7.9 percentage points, with 46.4 percent of fluent English speakers tested qualifying as proficient compared with 38.5 percent the year before. Reading results for eighth graders statewide were as comforting. In math, almost 73 percent of students from third through eighth grade met standards compared with almost 66 percent last year.

But some skeptics who have been on this roller coaster before wonder whether these increases are animated more by the content of the tests or by how the results are measured than by anything administrators or teachers did or did not do. In these critics’ view, a test may show an individual student’s progress, but is not as precise at measuring the progress of an entire grade or school system.

That’s not to say that New York students did not genuinely improve. State officials say scores rose because tougher curriculum standards were spelled out, teachers were given better training and students were given extra tutoring. But a little humility may be called for.

Robert Tobias ran New York City’s office of assessment for 13 years under seven chancellors, so he knows in his marrow the vagaries of test scores. He was there when chancellors flaunted the results and when they had to sheepishly explain why scores fell. He has learned neither to get too intoxicated by the leaps nor too downhearted by the plunges.

Mr. Tobias, who directs the Center for Research on Teaching and Learning at New York University’s Steinhardt School of Education, gets suspicious when test results rise too high from one year to the next, or when one grade rises spectacularly and others register only a modest change. . . .