Duncan's "What Lies Clearinghouse" Adds Its Own Misleading Spin on CREDO Study

Last June Stanford U's CREDO published the first nationwide assessment of charter school test performance. Here is the Press Release from the researchers of that study:

Stanford, CA – A new report issued today by the Center for Research on Education Outcomes (CREDO) at Stanford University found that there is a wide variance in the quality of the nation’s several thousand charter schools with, in the aggregate, students in charter schools not faring as well as students in traditional public schools.

While the report recognized a robust national demand for more charter schools from parents and local communities, it found that 17 percent of charter schools reported academic gains that were significantly better than traditional public schools, while 37 percent of charter schools showed gains that were worse than their traditional public school counterparts, with 46 percent of charter schools demonstrating no significant difference.

The report found that the academic success of students in charter schools was affected by the individual state policy environment. States with caps limiting the number of charter schools reported significantly lower academic results than states without caps limiting charter growth. States that have the presence of multiple charter school authorizers also reported lower academic results than states with fewer authorizers in place. Finally, states with charter legislation allowing for appeals of previously denied charter school applications saw a small but significant increase in student performance.

The Stanford report, entitled, “Multiple Choice: Charter School Performance in 16 States,” is the first detailed national assessment of charter school impacts since its longitudinal, student-level analysis covers more than 70 percent of the nation’s students attending charter schools. The peer- reviewed analysis looks at student achievement growth on state achievement tests in both reading and math with controls for student demographics and eligibility for program support such as free or reduced-price lunch and special education. The analysis includes the most current student achievement data from 15 states and the District of Columbia and gauges whether students who attend charter schools fare better than if they would have attended a traditional public school.

“The issue of quality is the most pressing problem that the charter school movement faces,” said Dr. Margaret Raymond, director of CREDO at Stanford University. “The charter school movement continues to work hard to remove barriers to charter school entry into the market, making notable strides to level the playing field and improve access to facilities funding, but now it needs to equally focus on removing the barriers to exit, which means closing underperforming schools.”

The report found several key positive findings regarding the academic performance of students attending charter schools. For students that are low income, charter schools had a larger and more positive effect than for similar students in traditional public schools. English Language Learner students also reported significantly better gains in charter schools, while special education students showed similar results to their traditional public school peers.

The report also found that students do better in charter schools over time. While first year charter school students on average experienced a decline in learning, students in their second and third years in charter schools saw a significant reversal, experiencing positive achievement gains.

The report found that achievement results varied by states that reported individual data. States with reading and math gains that were significantly higher for charter school students than would have occurred in traditional schools included: Arkansas, Colorado (Denver), Illinois (Chicago), Louisiana and Missouri.

States with reading and math gains that were either mixed or were not different than their peers in the traditional public school system included: California, the District of Columbia, Georgia and North Carolina.

States with reading and math gains that were significantly below their peers in the traditional public school system included: Arizona, Florida, Minnesota, New Mexico, Ohio and Texas.

"If the supporters of charter schools fail to address the quality challenge, they run the risk of having it addressed for them," said Dr. Raymond. "If the charter school movement is to flourish, a deliberate and sustained effort to increase the proportion of high quality schools is essential. The replication of successful charter school models is one important element of this effort. On the other side of the equation, however, authorizers, charter school advocates and policymakers must be willing and able to fulfill their end of the original charter school bargain, which is accountability in exchange for flexibility."

To download a copy of the full report and executive summary, visit: http://credo.stanford.ed

Now 8 months after the release of that ground-breaking report, ED's IES has published this one page misleading propaganda flyer that entirely ignores the main findings of the Report:

WWC Quick Review of the Report “Multiple Choice: Charter School Performance in 16 States”¹

What Two Groups of Students Were Contrasted?

In the pooled analysis, students attending charter schools nationwide were compared to similar students attending traditional public schools that had sent students to those charter schools.

In a more detailed analysis, students attending charter schools were compared only to similar students within the same education market, defined as all of the charter school students and all of their matched comparison records.

What is this study about?

The study examined the effect of charter school attendance on annual student achievement growth in math and reading.

The study analyzed data on a large sample of students in grades 1 through 12 who were attending charter schools and traditional public schools in 16 states.

The study authors matched charter school students to similar students based on grade level, baseline test scores, subsidized lunch status, special education status, and demographic characteristics. The authors were able to match 84 percent of charter school students.

The authors examined changes in standardized reading and math test scores from one school year to the next. They estimated effects by comparing the test score changes of charter school students to those of matched students attending traditional public schools.

---

WWC Rating

The research described in this report is consistent with WWC evidence standards with reservations

Strengths: Matched charter school students to similar students in traditional public schools using demographic and academic characteristics.

Cautions: Although the study matched charter school students to traditional public school students based on demographic characteristics and test scores, it is possible that there were other differences between the two groups that were not accounted for in the analysis, and these differences could have influenced achievement growth.

What did the study authors report?

The study found that charter school students’ reading and math test score growth was slightly lower than the test score growth of similar students attending traditional public schools. These differences were small, equivalent to moving a student from the 50th to the 49th percentile in math and less than that in reading. The study also found substantial variability in charter school performance; students in nearly one-fifth of the charter schools had higher test scores than students in traditional schools in the same education market while students in nearly one-third of the charter schools had lower test scores than students in traditional schools in the same education market.

The WWC has reservations about these results because charter students may have been different from traditional public school students in ways not controlled for in the analysis.

¹Center for Research on Education Outcomes. (June 2009). Multiple choice: Charter school performance in 16 states. Stanford, CA: Author.

WWC quick reviews are based on the evidence published in the report cited and rely on effect sizes and significance levels as reported by study authors. WWC does not confirm study authors’ findings or contact authors for additional information about the study. The WWC rating refers only to the results summarized above and not necessarily to all results presented in the study.

It's just too bad that Duncan's paid dissemblers and liars did not include this little section from the Executive Summary of the CREDO study:

Despite promising results in a number of states and within certain subgroups, the overall findings of this report indicate a disturbing — and far‐reaching — subset of poorly performing charter schools. If the charter school movement is to flourish, or indeed to deliver on promises made by proponents, a deliberate and sustained effort to increase the proportion of high quality schools is essential. The replication of successful school models is one important element of this effort. On the other side of the equation, however, authorizers must be willing and able to fulfill their end of the original charter school bargain: accountability in exchange for flexibility. When schools consistently fail, they should be closed.

Though simple in formulation, this task has proven to be extremely difficult in practice. Simply put, neither market mechanisms nor regulatory oversight been a sufficient force to deal with underperforming schools. At present there appears to be an authorizing crisis in the charter school sector. For a number of reasons — many of them understandable — authorizers find it difficult to close poorly performing schools. Despite low test scores, failing charter schools often have powerful and persuasive supporters in their communities who feel strongly that shutting down this school does not serve the best interests of currently enrolled students. Evidence of financial insolvency or corrupt governance structure, less easy to dispute or defend, is much more likely to lead to school closures than poor academic performance. And yet, as this report demonstrates, the apparent reluctance of authorizers to close underperforming charters ultimately reflects poorly on charter schools as a whole. More importantly, it hurts students (pp. 7-8).