KIPP's "Leap in Achievement" Is Misleading

According to Washington Post education reporter Jay Mathews, on average KIPP kids are at about the 30th percentile nationally in 4th grade. But by the end of 4 years at KIPP, they are about the 70th percentile. So it's reasonable to ask, "What caused the leap in achievement?"

Most KIPP supporters argue that KIPP caused the achievement. But I have some problems with this argument.

1) The extent to which any school -- or any single intervention, for that matter -- can "cause" higher achievement is debatable. If you could isolate and control for all other variables, you might be able to test this theory. But you can't, so you can't. At best, there might be a positive correlation between KIPP and higher achievement. But what, exactly, are the factors that are correlated with higher achievement? Better teachers? Enriched curriculum? Longer school day? Better prepared, more academically motivated students? Higher degree of parental involvement? For KIPP, all of these factors are at play.

Indeed, it's certainly possible that one major correlation is parental involvement. In fact, you could easily find strong evidence that parental involvement is much higher at KIPP than at other schools. It is, after all, a requirement for admission to a KIPP school.

As proof, here is the pledge that parents/guardians must sign before their children can be admitted to KIPP Tech Valley, a KIPP school in Albany, NY:

PARENTS’/GUARDIANS’ COMMITMENT

• We will make sure our child arrives every day by 7:30 A.M. (Monday – Friday), or boards a bus at the scheduled time.
• We will make arrangements for our child to remain at KIPP until 5:00 P.M. (Monday – Thursday) and 3:00 P.M. on Friday
• We will make arrangements for our child to come to KIPP on appropriate Saturdays at 9:00 A.M. and remain until 1:05 P.M.
• We will ensure that our child attends KIPP summer school.
• We will always help our child in the best way we know how, and we will do whatever it takes for him/her to learn. This also means that we will check our child’s homework every night, let him call the teacher if there is a problem with homework, and try to read with him/her every night.
• We will always make ourselves available to support our child’s education at KIPP TECH VALLEY. This also means that if our child is going to miss school, we will notify the teacher as soon as possible, and we will read carefully all the papers that the school sends home to us.
• We will allow our children to go on KIPP field trips.
• We will make sure our child follows the KIPP dress code.
• We understand that our child must follow the KIPP rules in order to protect the safety, interests, and rights of all individuals in the classroom. We, not the school, are responsible for the behavior and actions of our child.
• We will always protect the safety, interests and rights of all individuals in the classroom.
• Failure to adhere to these commitments can cause my child to lose various KIPP privileges.

Yes, these are the same parents of children whose achievement was low in the 4th grade, but the parents are more involved by the time the children reach 8th grade. KIPP provides them with a mechanism for how to be more involved (better?) parents. Indeed, it requires them to do so.

So it's possible that KIPP is merely a way to get parents more involved and -- once they are more involved -- students' academic achievement is positively affected. I'm not saying this is "the" reason that "causes" higher achievement. But I would argue that it has a very strong influence. So, my question is, "How many schools can replicate this degree of parental involvement?" Indeed, how many schools can demand it as a prerequisite for enrollment in the same way that KIPP does?

2) As for "leap in achievement," the obvious problem with averaging anything is that the average often does not depict the typical outcome. If there is one outcome that is very far from the rest of the data, then the average will be strongly affected by this outcome. In short, some really high achievers will make the others look pretty good, even if these others are not doing so well. To my knowledge, no one has done a statistical analysis of these data to determine the variance and standard deviation. The variance and standard deviation describe how spread out the data are. If the data all lie close to the average, then the standard deviation will be small, while if the data are spread out over a large range of values, it will be large. Having outliers -- very high achievers that make the KIPP scores seem better than they really are -- will increase the standard deviation. If the standard deviation is small, then the claim about "leap in achievement" would be quite substantive. But if the standard deviation is large, then the claim about "leap in achievement" would be pretty sketchy. In fact, it would be distorted and misleading.

3) KIPP claims that it has a broad-based curriculum and does not shirk on subjects that are not tested under NCLB, e.g., social studies. According to the SRI report on Bay Area KIPP schools, "Students have 90 minutes of (English Language Arts) and math every day. They also have 90 minutes of social studies and science on alternating days." (p. 33) The report does not indicate what is actually taught in the social studies and science blocks, nor how it is taught. However, there is no evidence that KIPP students receive a "broad-based education" other than the fact that science and social studies are taught for 90 minutes every other day. Because students are not tested in these subjects, we have no way of knowing if they are learning anything. More troubling, we have no way of knowing if the instruction they receive is substantive or superficial.