Part 4 of a 3 Part Series

This is the fourth installment of my 3-part series on charter management organizations (part 1, part 2, part 3). Maybe this should be called a post-script - I've only added it because part 3 came out as a dud due to the disappearance of comments from Tom Vander Ark's EdReformer blog.

Let me clarify a bit: I contacted EdReformer to find out what happened to the comments. They're claiming the comments were lost when they changed comment programs (from Disqus to something else), and I'm actually inclined to believe this statement. Could they recover the comments? Maybe. I'm going to act as though the removal wasn't done on purpose. It's an entirely different issue if they intentionally removed the comments. I'll do my best to summarize the comments to the best of my recollection, and provide the actual text of any comments I can track down.

Before reading any further, make sure you're familiar with Tom's brief comments on the Center for Reinventing Public Education report on charter management organizations.

Comments:

The first comment was left by CRPE researcher and Association Director Robin Lake. Lake is very involved in charter/CMO research and directs the National Charter School Research Project (which the CMO report is part of). She visited EdReformer roughly 3 weeks ago and left the first comment. She started off by stating she was a bit confused by TVA's comments and went on to explain that - although she appreciates his support - the CMO movement needs coaches more than cheerleaders (or implied the former was more important). This is a reference to TVA's conclusion, which criticized CRPE (and specifically Lake and Paul Hill) because they "bent over backwards to write a 'balanced' report and missed some stuff that cheerleaders like me find obvious."

TVA responded to Lake's comments by saying they must have looked at "some pretty crappy CMOs" (that's one direct quote I remember) and not the good ones like Achievement First, Success Network, Village Academies, Uncommon Schools, Aspire, Green Dot, PUC, Bright Star, ICEF, Uplift, IDEA, HTH, Harmony, and CharterSchools USA (more below). Additionally, TVA claimed the Hoxby charter school report suggests CMOs are a promising innovation (more below).

The next comment was left by Marc Dean Millot. Millot saved a copy of his comment:

Tom:

Consider this an "open letter."

Yours is one of the strangest summaries of an evaluation I have ever read - particularly where the summary is offered by someone who played such a prominent role in the the subject of the evaluation.

"New philanthropy" grants officers like you, Kim Smith and Jim Shelton offered the CMO as the means of realizing the charter school movement's vision of financially self-sustaining independent public schools, providing consistently high levels of student performance, on a national scale. As the CRPE report points out, the Gates Foundation, New Schools Venture Fund, the Walton Family Foundation, among other new philanthropies - as well as the taxpayers via several substantial US Department of Education charter schools grants, put $500 million behind the decade-long effort to prove this investment thesis. To give your blogs readers some idea of the grants program's audacious scope, when adjusted for inflation it is three times larger than New American Schools prior (and, by comparison at least, successful) ten-year investment to develop mostly nonprofit, fee-driven Comprehensive School Improvement providers. For all practical purposes, the new philanthropy invested the entirety of the discretionary capital available to secure the charter movement future. It "bet the farm" on this one model.

Audacious gambles demand commensurate pay-offs. What are the results?

As you know, Mathematica is responsible for an independent assessment of the educational performance of the CMOs under review. Perhaps that report will find one or more CMO's operating a sufficiently large number of schools at consistently high levels of student performance to prove the educational component of the new philanthropy's investment thesis. Whatever the outcome of that study, the CRPE report provides absolutely no reason for readers to believe that "at current course and speed" the CMOs it reviewed can either become financially self-sustaining or achieve the scale envisioned by its advocates. Indeed the report's detailed look at the most prominent CMOs under review shows not merely the failure of management teams to capture economies of scale, but an impressive capacity to create diseconomies. Worse still, the interim evaluation offers readers no reason to hope that the CMOs under review are making any financial, operational, organizational or managerial changes likely to materially alter the findings of CRPE's final report in 2011, or that while individual organizations may be flawed, the basic business model holds up to scrutiny.

In short, your statement that "the bad news.. is that CMO still require philanthropic support to scale" is an understatement of breathtaking proportions. The truly bad news is that after spending $500 million, CRPE's report of the organizational and financial components of the CMO investment thesis doesn't even warrant what is widely understood to be "the gentleman's C" in k-12 program evaluation - "promising." Indeed, the final set of recommendations in the report, "Other Approaches to Scale," amount to "next time, let's do this (kind of investment program) right."

In the 1990's private sector investors spent roughly $1 bilion to demonstrate that as a breed, the Education Management Organization is a dog that just wont hunt. Over the last ten years, the new philanthropy spent half that proving that changing the breed's name made absolutely no difference. Hunters will tell you that every so often they'll find a terrier that can run with the hounds. Likewise, there are financially viable EMOS, and there will undoubtably be successful CMOs. But like the odd terrier, on examination they will be exceptions that prove the rule. Centrally managed school operations are no more likely to produce quality at scale as non profits or for profits than they have as government agencies.

In an age when those in charge eschew responsibility for decisions that go wrong, readers should not expect the new philanthropy's investment managers to accept responsibiity for the consequences of their decision to rely on the top-down CMO strategy as the means to quality, scale and sustainability; to explain why they did not pursue plausible bottom-up options or how they decided that nonprofit management organizations could succeed where for profits failed, or even recognize just how far their failed decision has set back the charter schools movement - let alone make amends. For one thing, they've all gone on to other jobs. One can only hope that CRPE's financial and operational accounting of CMOs, together with the oral accounts of those CMO's managers that EdSector excised from Tom Toch's original narrative, are enough to take this education reform option off the front burner and into the bottom of that freezer in the basement.

Marc Dean Millot

TVA allowed this comment to be posted (kudos to him for allowing dissenting voices to be heard on his blog), but he dismissed Millot's concerns/suggestions. He added that the CRPE report surely must not have looked at CMOs invested in by the NewSchools Venture Fund or the Charter School Growth Fund (more below). I believe he may have responded that blended learning would help reduce some expenses.

I was the next one to respond. My comment pointed out a few important facts: the CRPE study did, in fact, look quite extensively at NSVF/CSGF CMOs, and the Hoxby study doesn't make any claims about CMOs being more effective than any other type of charter schools (no need to debate the validity/accuracy of the Hoxby report - that's an entirely different topic).

Take a peek at the CMOs funded (in part) by NSVF, and the CMOs funded (in part) by CSGF. Remember that TVA claimed CRPE must have looked at some "crappy CMOs" and not the ones funded by these two organizations. There's not way to put this nicely: TVA is just plain wrong about this, and it's really baffling that he'd make this claim. Here is a list of the only NSVF/CSGF CMOs not included in the CRPE report:

CSGF:

KIPP Austin

KIPP Delta

KIPP NO (too small)

KIPP Philly

KIPP TEAM (too small)

Rocketship

West Denver Prep (too small)

NSVF:

AUSL

Bright Star (too small)

DC Prep (too small)

Education for Change (too small)

LearnNow (for-profit; note that this company was started by OII director Jim Shelton)

Recall that TVA's response to Lake's comment mentioned a number of CMOs that TVA really likes. Here is that list, with notes about funders or whether or not they were included in the study (note: only non-profit CMOs with more than 4 charters were included):

Achievement First (NSVF & CSGF)

Success Network (NSVF)

Village Academies (too small)

Uncommon Schools (NSVF & CSGF)

Aspire (NSVF & CSGF)

Green Dot (NSVF)

PUC (NSVF)

Bright Star

ICEF (NSVF)

Uplift (NSVF

IDEA (CSGF)

HTH (NSVF)

Harmony

CharterSchools USA (for-profit)

To summarize, very few CMOs in the NSVF/CSGF portfolios were not in the study, and all the great CMOs TVA mentions are either supported by NSVF/CSGF or excluded from the study. Further, it's practically impossible that the study didn't look specifically at the management, struggles, human capital, and financial sustainability of at least some of these CMOs. Let me explain why I can make that claim.

Lake and Co. noted that they looked closely at 10 CMOs with particular attention to larger CMOs because the goal of this kind of organization is to go to scale. They're not too interested in small chains - they want the big operators. Of the large charters - which the researchers say were represented in this 10 CMO sample more than their smaller counterparts - only a few were not in the NSVF/CSGF portfolio: Harmony, Constellation Schools, PPEP Tech, KIPP Houston (remember that KIPP as a whole is considered a franchise in this study; their local groups of schools, however, are considered CMOs), and American Quality Schools. All the rest of the big ones are NSVF/CSGF investments: Aspire, Green Dot, Achievement First, ACRPS, Lighthouse, ICEF, Uplift, and PUC (note: I'm using the same definition of "large" as CRPE does - 10 schools or more - although I'm looking at 2010 data when they may have been looking at another data set. In other words, this isn't totally accurate, but it's pretty darn close).

In other words, it'd be almost impossible that CRPE didn't look at NSVF/CSGF CMOs, which flies in the face of TVA's comments that the researchers must have looked at some "crappy CMOs" (unless he consideres NSVF/CSGF CMOs crappy, which I guarantee you he does not).

Secondly, TVA's response to Lake's first comment mentioned the Hoxby study is evidence that CMOs are highly effective. And, since we're not talking about CMO to public school comparisons here but rather CMO to stand alone charters here, I'm going to point out part of the study that TVA either didn't read or disregarded. Specifically, the Hoxby's study says:

When considered either by itself or simultaneously with other characteristics, a charter school's operating agency type (CMO, EMO, or CGO) does not have an association with achievement effects that is statistically different than zero. [p V-4]

This isn't to say CMOs aren't effective (as defined by the Hoxby study) - but it does suggest there's no difference between CMO-operated charters and stand alone charters. Additionally, I mentioned that the Hoxby report said 29% of charters were operated by CMOs, 21% by EMOs, and 49% by CGOs (community growth organizations).

TVA responded to my comment by expressing his surprise that the Hoxby report was so strong given the limited number of "one-off" charter schools (I believe this is how he described stand alone charters). He shrugged off the bit about NSVF/CSGF-funded CMOs. That was about the full extent of his comment.

Conclusion:

I generally wouldn't waste my time putting together a 4-part series on just any blogger making questionable comments - but TVA isn't just any blogger, EdReformer isn't just any blog, the CRPE report isn't just any report.

My guess is that anyone reading this far knows about TVA's experience in education, particularly his role at the Gates Foundation. The foundation made significant donations to NSVF during his tenure there, and these grants were specifically made to expand CMOs. The CRPE report, to a certain degree, is a critique of some of the work TVA did while at the Gates Foundation (his main reform was the small schools experiment).

The EdReformer blog is "incubated by NewSchools Venture Fund with support from a donor committed to innovations in learning." This would be particularly relevant if EdReformer intentionally pulled the comments (which I don't have any evidence they did - and, in fact, they've provided evidence that suggest the comments were not pulled intentionally but rather removed when they switched comment programs). Still, something to keep in mind, particularly if you're a regular reader of EdReformer.

Most importantly, TVA is considered an "opinion leader" in education reform, works for a number of education organizations, sits on the Board of Advisors for Democrats for Education Reform, and undoubtedly played and continues to play a significant role in the CMO movement. It's unclear if TVA didn't read the CRPE report, suffers from the Pollyanna principle, or simply didn't comprehend the material - and it's really not that important. What is important, however, is that TVA is making comments about the growth of CMOs that muddies rather than clarifies the strengths and struggles of CMOs (particularly the struggles), acts surprised when presented with information that contradicts his initial comments, and then seems to dismiss the facts when presented to him.