A Question About Curves

I'm not a psychometrician, so I don't really get something about standardized tests. So here's a simple question: are standardized tests able to show that every kid is at the proficient or advanced level by 2014?

From what I understand about large-scale standardized assessments, the test developers strive to construct exam questions that will discriminate different levels of competence in the test-taking population. When a test question proves to be skewed (either too easy or too hard), there are statistical processes that can be applied to take that into account in making the ultimate judgment about the score a student receives, which is a way to counteract design problems. Statistical corrections are imposed when a test question does not perform in the way it was hoped when the question was designed. It is quite possible that strong students who answer a question incorrectly on a tested concept could end up with a high score after all is said and done, particularly if the question ends up being answered incorrectly by virtually everyone.

But NCLB says that all children must/can/will be at the proficient or advanced level by 2014.

So which is it? Tests that produce "a normal curve" with "well-designed" questions OR all children can reach grade level by 2014? It can't be both. It simply is not possible. We either get all kids up to grade level by 2014 or we stick with the time-honored tradition of predictable rank ordering through standardized tests.

What's interesting about "a normal curve" is the underlying notion that predictable rank-ordering is the norm. That lining students up and saying, "OK, lil' Johnny, you're Number One because you're the smartest kid in the school. So you stand at the front of the line. As for you, lil' Becky, you're Number 357 because you're the 357th smartest kid in the school. So you stand at the end of the line." Of course, there is nothing "normal" or "natural" about rank-ordering kids on the basis of a single measure, especially a standardized test that pre-supposes such a "normal" distribution of grades and, indeed, guarantees this kind of distribution through its organic, tree-grown, natural, hand-picked "statistical processes" that are created to "counteract design problems." In other words, what we have here is Ye Olde Selfe-Fulfilling Prophece.

This "normal" distribution is said to facilitate the fairest construction possible. As if lining kids up and calling the one with the highest score "smart" and the one with the lowest score "in need of improvement" is fair. What are the odds that the "smart" kid happens to be white and come from affluent, college-educated parents? What are the odds that the "in need of improvement" kid happens to be black or Hispanic and come from poor, uneducated parents or from a single, poor, uneducated parent?

I love the image of an unruly test question that gets out of hand and does not perform in the way it was hoped when it was designed. It reminds me of Dr. Frankenstein and his monster, who ultimately did not perform in the way Frankenstein hoped when he designed it. Like the Frankenstein monster, the unruly test question has order imposed on it. But instead of an angry mob with flaming torches, the test question has "statistical corrections" imposed on it. Either way, the effect is the same: disorder and monstrousness are not tolerated. In the end, we can rest assured that the reality we created ahead of time will be the reality that is re-created on the test. "Strong students" won't be adversely affected by the monster test question. It will be chained down, burned, deleted.

As long as the monster wreaks havoc on everyone, all will be well.

Of course, there's a problem if the monster/unruly test question befriends someone along the way. In the Frankenstein story, "virtually everyone" found the monster to be terrifying and hideous. But the monster made friends with a blind man, an outcast. For the blind man, the monster was not monstrous. Children who take one of these tests and answer the unruly question correctly do not find the question unruly. So what then? What happens to the logic -- the "statistical corrections" -- that would otherwise save "strong students" from being penalized? What if several "weak students" answered the unruly question correctly? And what if these "weak students" happened to be black or Hispanic? Are there certain "statistical corrections" and algorithms that check the race and socioeconomic status of students? And, if so, do these statistical corrections throw out the correct answers of these students on the unruly question, reasoning that these students must have guessed because the strong students did not answer it correctly?