Georgia State Officials Knew of Mass Failures Ahead of Tests

From Atlanta Journal-Constitution:

By HEATHER VOGELL, LAURA DIAMOND, ALAN JUDD
The Atlanta Journal-Constitution
Published on: 05/22/08
The state Department of Education knew as early as July 2007 that tens of thousands of sixth- and seventh-graders were on track to bomb on this year's mandatory social studies test, documents obtained by the Atlanta Journal-Constitution show.

But officials allowed the testing to go forward, apparently without warning schools, teachers, parents or students of the likelihood of widespread failures.

State school officials released the documents as criticism mounted Thursday of how they handled this year's statewide Criterion-Referenced Competency Test.

"This is atrocious and unforgivable," said Jason Adams, a seventh-grade teacher at Lost Mountain Middle School in Cobb County. "This is the kind of thing where a heads-up to teachers would have been nice."

Dana Tofig, the education agency's spokesman, said early projections were based on pilot questions given to students who hadn't been taught the state's new social studies curriculum. Officials assumed students would score higher this year.

The documents show students taking the pilot test answered large numbers of questions incorrectly.

By February, six weeks before testing began, officials had put a precise number on the predicted failures: 69 percent of students in both grades would likely not meet the bar.

The prediction proved generous.

Students, teachers and parents learned this week that 70 to 80 percent of middle-schoolers in the two grades had failed to pass the social studies test this spring. On the eighth-grade math test, which students must pass to go on to high school, only about 60 percent had passed — 20 percentage points fewer than the year before.

Teachers 'devastated'

The results are preliminary. Official and complete results are due next month.

Yet on Wednesday, state Superintendent Kathy Cox announced the state was throwing out the social studies results, blaming a vague curriculum and imprecise direction for teachers. She said the math results would stand and defended the test as necessarily more rigorous.

The state's testing contractor, CTB/McGraw-Hill, tried out 80 potential new questions with a sampling of Georgia students in the spring of 2007, according to state education documents. Committees made up largely of Georgia teachers chose 60 questions for the 2008 test, despite the poor results from the pilot.

Tofig said Cox was not available for an interview Thursday.

He said the pilot, or "field," test results were speculative, and useful only for setting the minimum score, known as a cut score, needed to pass the test. Pilot scores are not always predictive, Tofig said, noting that a cut-score committee projected 52 percent of eighth-graders would fail the math exam, while only 40 percent actually did.

"You really don't know what's going to happen until you get the data," Tofig said.

The department made no changes based on the anticipated social studies scores, he said. Nor did it share the projections beyond a small circle of state officials.

"A limited number of people had seen that in February," Tofig said. "That whole process is secure." To protect the tests' integrity, he added, "it has to be secure."

The state made the projections public in April when the state board approved cut scores for the tests.

The results "came as a great surprise to curriculum leaders" in the school districts, said Deborah White, executive director of the Georgia Association of Curriculum and Instructional Supervisors. "Teachers were devastated."

Cherokee County Superintendent Frank Petruzielo also said school systems had no idea what was coming. But he said the result should not have surprised state officials.

"Maybe they underestimated," he said. "But they knew the failure rate was going to be extraordinary."

Petruzielo agreed that vague teaching guidelines contributed to the high failure fate in social studies. But he said the state also raised the standards on eighth-grade math enough to trip up even accomplished students. He said state officials may have been "overreaching" to improve student test performance.

"The bar was simply set too high too soon," Petruzielo said. "We weren't able to show how much progress kids have made year to year when just getting over the bar was such a Herculean task."

The state Board of Education raised the cut score in sixth- and seventh-grade social studies from 23 and 22 correct answers, respectively, to 32 and 31 right answers out of 60 questions. In eighth-grade math, the cut score decreased, from 35 to 32.

State board member William Bradley Bryant said he expected a gap between performance last year and this year.

"The only thing we could have done with the cut scores was say, 'Are we more comfortable with more people passing the test even if that meant lowering the bar?' " said Bryant, whose district includes Gwinnett, DeKalb and Decatur. "It would look good on paper, but it's more important for them to leave the grade with the content knowledge we think they need."

He said he wasn't sure if a connection could be made between the projections and the results released this week.

Tim Callahan, spokesman for the Professional Association of Georgia Educators, said state officials should have acted to head off disaster when they saw the warning signs in the pilot test.

"You would not have let the train continue on in the dark and wreck like it has now," he said. A panel Cox is convening to look into what happened will likely do some things that should have been done before, he said.

Superintendents not told

Teachers have complained repeatedly about inadequate training as the curriculum has been revamped, said Callahan — whose group's 72,000 members are mostly teachers.

In recent weeks, as dismal scores trickled in, teachers called the state in alarm, Callahan said. "The initial response was kind of flippant and cold," he said. "They were like, 'Well, you didn't do your job.' "

Gwinnett Superintendent J. Alvin Wilbanks said local superintendents weregiven no direct information that the failure rates would be so high. School leaders did know scores would drop because of the new tests and higher standards.

He said he doesn't know if sharing the projections would have helped.

"With those projections, what you got is what you got and I don't know what knowing about it would have done," Wilbanks said. "But it's always nice to have information and prepare people. I don't know if you'd still administer the test."