Guest Post: Assumptions that 'Just Seem Obvious' to Steve Brill

By John Thompson

There is a long line of education experts exposing the factual errors in Steve Brill's Class Warfare so, instead, I set out to illuminate the assumptions underneath his brand of "reform." I was overwhelmed, however, by the number of times that Brill, and the non-educators he fawned over, discovered findings that "just seemed obvious." So, I only want to recount Brill's tale of the naive assumptions, opinions, and assertions, based on numbers from one district, that supposedly propelled Bill Gates into the experiment that Gates said was, "the riskiest thing we have ever done."

Brill begins with "Identifying Effective Teaching Using Performance on the Job," by Tom Kane, Robert Gordon, and Douglas Staiger. Kane et. al wrote, "the current credential-centered regime is built upon two questionable premises ..." They then presented evidence that was solid enough to contribute to the minor league debate over teacher certification. But then these economists made another leap, "the second premise is that school districts learn nothing more about teacher effectiveness." Without making any effort to link their data (and that assertion) to reality, Kane and company called for dangerous and revolutionary changes to the entire nation's educational systems. When I first read their opinions, I chalked them up to being academic theory.

If Brill's account is trustworthy, though, I was naive. Either Brill was loose with his words, or he was celebrating a classic "bait and switch." According to Brill, the authors asserted that the need for "paper qualifications" is a "bedrock of public education." Worse, Kane et. al supposedly claimed that the second core assumption of our system is that "districts can learn nothing more of teaching effectiveness after the initial hire," and "this was why almost all teachers initially receive satisfactory evaluations." (emphasis mine.) Even worse, they successfully convinced Bill Gates that a "surprising" lack of volatility in teachers' performances, measured by standardized tests, was grounds for overturning decades of social science research on teaching and learning, and the way that organizations change.

Rereading the paper, I fear that Brill accurately reported the authors' true intentions. Now I see the report as a grab bag of political soundbites. Kane et. al predicted an impending shortage of teachers, which would be most damaging for high-poverty schools. Their solution - based on the presumption that the lowest scoring teachers should be fired using a methodology that is systematically biased against teachers in high-poverty schools - would supposedly attract talent to those schools!?!?! To their credit, Kane and company recognized the need to adjust for circumstances beyond the teachers' control. For example, if construction was going on outside the window on testing day... But, they argued that test score growth models that would be used for evaluations should not be controlled for poverty.

Fortunately, the Gates team has subsequently distanced themselves from their initial overreach, but it still raises a huge question. What did they not know about peer effects and when did they not know it?

If Brill is accurate, having assumed that there is a single simple explanation for the sorry state of evaluations, the theorists did not need to search for other reasons why inner city schools have a hard time attracting and retaining talent. Having reached the shocking discovery that, "some teachers are better than others," Gates and company thus decided that the path to school improvement must go through their version of instruction-driven, standardized test-driven reforms.

According to Brill, Bill Gates bought the hypothesis that the lack of volatility in test scores meant that their version of teacher quality had to be the focus of reform. After a two-hour discussion, supposedly, Gates became increasingly frustrated that educators ignored a solution that "seemed so obvious."

I will leave it to the experts to analyze how stable or volatile test scores are. Subsequent research by Gates' scholars has been used to argue that teacher performance is no more unstable than major league baseball players' performance - as if that has much to do with fixing schools. My point is that their assumption could only have been made by people with no knowledge of the logistics of schooling. As I have explained, if Brill, Kane, or Gates had had a rudimentary knowledge of how schools are actually run, they could not have jumped to such a weird conclusion.

And that gets to other assumptions. When I was in school, Kane would not have begun such a paper with a mischaracterization of his opponents' positions. A reader used to assume that a scholar would start his argument by honestly summarizing what his evidence was and was not able to prove. In polite discourse, a summary of opposing arguments was always presented in a fair manner. Kane would have been honor-bound to summarize the record of longstanding evaluation methods such as the Toledo Plan, the Teacher Advancement Program, and other successful peer review and mentoring efforts, as well as National Board certification. Had Gates been briefed on the strengths and weaknesses of those union-endorsed methods, perhaps he would have chosen to fund efforts to better incorporate data into those holistic systems, or perhaps he would have financed a scaling up on those methods of improving teacher quality.

The bad news is that education, and educators, are held in such low esteem that revolutionary reforms are mandated by non-educators, based on what seems obvious after being exposed to a sliver of information. Too often, it presented only by self-interested parties who represent a small minority of scholars. The good news is that snap judgments can be reversed. Smart people have changed their minds when introduce to better evidence.

This era of primitive test-driven hypotheses reminds me of the rookie mistakes made by first generation econometricians such as Robert Fogel, Stanley Engerman, Alfred Conrad, and John Meyer. One even earned a Nobel Prize, showing that a bone-headed error, born of inexperience need not be a mark of shame. Back then, however, there were more checks and balances in regard to academic assertions. When I was in school, these economists made the daring, and soon-to-be-proven-silly proclamations that the building of the railroad system did not accelerate economic growth and that slavery was profitable. Those mistakes are humorous now, but then again, we did not cite their speculations as reasons to tear apart our railroads or reinstate slavery.

And we should remember a key reason why an econometric model, based on data from one state, reached such inaccurate conclusions about slavery across the nation. To the Harvard economists, it seemed obvious that half of Kentucky's mules must be momma mules and half must be daddy mules. Not knowing that mules are sterile, they assumed the profits due to mule reproduction.

If Gates and Kane would be more open about what they do not know, and listen to actual practitioners, they could still be a powerful force for good. But they need to listen to educators with concrete knowledge of how schools are run, and understand the power of peer pressure.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.