How Much Teachers Affect Student Achievement, and Other Myths

Thanks, largely, to the irresponsible and sloppy messaging by some statisticians and psychometricians who know better, there is an ugly meme circulating that teachers are the most influential factor influencing student achievement, and so, if we can just get rid of the bad teachers and replace them with non-teachers with no experience and no benefits (this is where the logic really breaks down), then we will have the achievement gap on the way to closing up.  

Below are a couple of reminders from the research about what the realities are, despite the obfuscating rhetoric from both sides of the aisle of the corporate policy jet.

The first is an analysis of latest international test results from PISA.  It is offered by Mel Riddile from the  National Association of Secondary School Principals at their research blog, The Principal Difference.

Here's the link to the blog post and the intro below, which is followed by a detailed analysis at the site on how American students compare with students in other countries.    

"There are three kinds of lies; lies, damn lies, and statistics."--Mark Twain

The release of the 2009 PISA results this past week has created quite a stir and has provided ample fodder for public school bashers and doomsayers who further their own philosophical and profit-motivated agendas by painting all public schools as failing. For whatever reason, these so-called experts, many of whom have had little or no actual exposure to public schools, refuse to paint an accurate picture of the state of education.

Secretary of Education, Arne Duncan, should be providing the nation with a proper vision and focus for public education. He knows our challenges all too well. He confirmed that he gets it when he recently wrote me saying, "We must build a culture nationally where great educators ... choose to work with children and communities who need the most help." I believe his message is sincere and heartfelt and it is spot on. However, overstating a problem in order to increase the sense of urgency around school improvement is just as bad as understating the problem.

This week, Duncan had a golden opportunity to use the PISA results to provide focus for our education efforts and to point us in the right direction. Instead, he dug himself deeper into the pseudo-reformers' hole--more charter schools, more reliance on competition and free-market strategies, more testing, more use of test scores to evaluate teachers, more firing of principals and teachers, more closing of low-scoring schools--when he said, "the PISA scores released this past Tuesday were "a massive wake-up call," because the scores show American students holding relatively steady in the middle of the pack of the developed nations taking the international exam.

There is, however, someone who recognizes that the data is being misinterpreted.  NEAToday published remarks from National Association of Secondary School Principals Executive Director, Dr. Gerald N. Tirozzi, that have taken "a closer look at how the U.S. reading scores on PISA compared with the rest of the world’s, overlaying it with the statistics on how many of the tested students are in the government’s free and reduced lunch program for students below the poverty line." Tirozzi pointed out, “Once again, we’re reminded that students in poverty require intensive supports to break past a condition that formal schooling alone cannot overcome.” Tirozzi demonstrates the correlation between socio-economic status and reading by presenting the PISA scores in terms of individual American schools and poverty.  While the overall PISA rankings ignore such differences in the tested schools, when groupings based on the rate of free and reduced lunch are created, a direct relationship is established.

What the PISA and NASSP analysis shows is that, when we take out the bottom quintile of students based on family wealth and income, American students are doing just fine.  More than just fine, in fact--we are right up there with the top scoring nations.  But because we have more children per capita living in poverty than any developed nation on Earth, the effects of poverty take their toll on our national standings in the international test score derbies.

Of course, you never hear this fact mentioned by the corporate education reformers, who now want to double down on another generation of more high stakes tests (how did Einstein define "insanity?") that will continue to label and isolate the poor in remedial testing camps, while blaming teachers and parents for not accomplishing that which is made impossible by 1) poverty and, 2) classist/racist tests .

Which brings me to the second bit of research that puts into context how much teachers, even the best ones, can accomplish alone or even together as long as we continue to ignore the ravaging effects of poverty.

The post below is written by Jim Alexander, and it is based on research by Dan Goldhaber who extends the work of James Coleman into the present day.  

Alexander notes that Goldhaber and his colleagues have concluded is that "that around 9 percent of variation in student achievement is due to teacher characteristics. About 60 percent of variation is explainable by individual student characteristics, family characteristics, and such variables. All school input combined (teacher quality, class variables, etc.) account for approximately 21 percent of student outcomes." So even though teachers are the most important school-based factor in student achievement (however you measure it), a teacher's influence pales in comparison to factors from outside the school.  So now you can explain to me the logic of how student testing and observations should count 100% of a teacher's evaluation for effectiveness when teachers, even the best ones, account for less than 10 percent of student achievement.

Alexander's post (my bold):  

The documentary [Waiting for Superman] and most other commentaries on schools and schooling—as well as laws and funding—are based on the notion that somehow teachers are the root cause of the problems facing U.S. schools. The idea seems to be that we need to get rid of a whole bunch of lazy, incompetent teachers.

I have a problem with that notion. I have trained pre-service teachers for nearly twenty years. When I look at my students and work with them as student teachers, they certainly do not strike me as incompetent or lazy. Instead, the entire standards/ testing/ accountability movement needs to be scraped. It doesn’t need to be tweaked; we’ve been tweaking for at least thirty-five years. We don’t need a different test to use with kids. We have plenty of those right now and already waste considerable instructional time in testing and test preparation. The whole approach has been weighed and measured and found wanting.

At best, student achievement is a mixed bag. It involves both home and school factors. Three variables: socioeconomic status, time spent on homework, and level of parental involvement deal with home variables and are essential variables in student achievement. An Educational Testing Service report stated that the home environment is as important in influencing what goes on in school as in-school factors. A study by the Metropolitan Life Insurance Company found that 97% of students who earned mostly A’s and B’s on their report cards reported that their parents encouraged them to do well in school. Forty-nine percent of students earning “C’s” received little encouragement.

Nor can the impact of media on children be overlooked. Jane Healy has recounted how TV watching adversely impacts children in schools. This is especially true when it comes to a child’s ability to maintain attention—something school requires. A more modern critique related to computer usage, reading, and attention has been offered by Nicholas Carr with similar conclusions. Sherry Turkle addresses the ways that social networking has changed the culture of children and adults. Surely, a discussion of societal change must include the impact of technology.

Approximately 50 percent of our proclivities are genetic in nature. That is not so much of a concern to us—the remaining 50 percent is quite adequate to make a huge difference in student learning. About half of school-to-school variance in achievement relates to out-of-school factors. Dan Goldhaber, senior research associate at the Urban Institute, points out there has been a continuous stream of research indicating the socioeconomic background is the most important factor in student achievement.

The Great Lakes Center for Education Research and Practice is emphatic. Out-of-school factors related to poverty are the major cause of the achievement gap that exists between poor students and the rest of the student population. This contrasts with current educational/government opinions placing student failure squarely on the shoulders of schools and teachers.

It’s true that every child deserves an excellent teacher. Yet, Goldhaber and colleagues have discovered that around 9 percent of variation in student achievement is due to teacher characteristics. About 60 percent of variation is explainable by individual student characteristics, family characteristics, and such variables. All school input combined (teacher quality, class variables, etc.) account for approximately 21 percent of student outcomes.

Judith Harris convincingly demonstrates that a great deal of the outcomes of children’s lives come from the peer group and society. This theory is not without its distracters. It defies the conventional wisdom and annoys those who have long held to the prominence of parental nurture in determining outcomes for children. When it comes to school, in like manner, it is clear that out-of-school variables such as 

the percentage of children living with only one parent, the percentage of eighth graders absent from school at least three times a month, the percentage of children age 5 or younger whose caregivers fail to read to them daily and the percentage of eighth graders who watch five or more hours of television daily are at high risk of school failure.

I propose that the problems in our schools are not predominantly due to lazy, ineffective teachers. Much of what happens in terms of children’s achievement cannot be pinned on what happens in schools. Further, the idea of “cleaning up Dodge” is misguided and foolish. What is needed is a great discussion of where societal and cultural values have taken a wrong turn. In short, the accountability/testing/standards approach is irredeemable. It is broken beyond repair. Educational problems are largely societal in nature. Societies can assess themselves and they can change. Of that I am certain.

Diane Ravitch was long a favorite of the conservatives. She served in the Education Department of the George H. W. Bush administration. She was later a staunch supporter ofNo Child Left Behind. Recently, she has undergone a bit of a conversion as she has reviewed the data of reform. She points out that charter schools are often more hype than reality. Even when they do succeed (usually they do not), much success can be contributed to the dogged determination of students and parents.

So, I repeat, the entire enterprise is flawed. No one can fault standards as the basis of a curriculum guide. Beyond that standards, testing, and accountability form a devastating trio. It simply cannot be decreed that all students will be on grade level by a certain date (2014). It doesn’t work that way. It leaves teachers anxious and demoralized. It does the same for kids. What we need is not more tests and standards and accountability but, rather, a great societal turning.

James Alexander, Ph.D., Professor Kentucky Wesleyan College

So what can we do?  We can continue to improve our teaching in every way we can, even as we must begin to alter the ravaging effects of poverty and to advocate for policies that help to limit the effects the poverty.  Health care, nutrition, housing, transportation, jobs, and integrated and diverse schools that can take take advantage of the power of shared social capital.  

AND we must begin to advocate, as 1,400 of the 4,500 New York principals (nearly a third) have done, and as over 100 New York professor have done, for the end of high stakes standardized testing and for assessment policies that are fair to those who are most in need of the best, enriching curriculums, rather than the inhumane parrot learning camps that children of the poor are subjected to.  Remember the chart, the one I always show?  If you forget everything I have ever said or will say, please remember this chart from Rampell at the NY Times Economix Blog: