Judge Refuses to Stop Unfair and Unreliable Value Added System

Not in the bolded section of this news story, the federal judge notes that the teachers have a strong case against the unreliable nature of the the Tennessee VAM system known as TVAAS.  He just refuses to do anything about it.

....TVAAS is a complex algorithm that aims to isolate the impact of individual teachers on their students’ learning, as measured by state tests. One of the nation’s first “value-added” formulas, it has inspired similar efforts in other states.

TVAAS scores have been calculated since the 1990s but started being used to help determine ratings, bonuses and tenure status only since 2011, when Tennessee overhauled its teacher evaluation law.

Under state law, TVAAS scores make up 35 percent of teachers’ ratings, with the rest based on in-person observations and “achievement measures,” which can include graduation rates, students’ AP or IB exam scores, or school-wide TVAAS scores.

The two teachers who filed the lawsuit, Lisa Trout and Mark Taylor, had strong ratings from classroom observations but TVAAS scores that were too low to make them eligible for bonuses from Knox County Schools. The district gives bonuses of up to $2,000 a year to teachers with strong ratings. Trout and Taylor charged that those scores should be discounted because only some of their students took the end-of-course exams used to generate the TVAAS scores.

Taylor’s rating was based on scores of just 22 of his 142 students, he said, rendering his TVAAS score meaningless.

Court documents reflect an exchange between William Sanders, the statistician who designed TVAAS, and Taylor’s parents, with whom he is acquainted.

Sanders was asked if a TVAAS score based on test scores of only a small fraction of a teacher’s students reflect a proper use of TVAAS. His answer: “For an overall evaluation of the effectiveness of the teacher to facilitate student academic progress, of course not.”

Mattice said in his ruling that he found the criticism compelling. But ultimately, the court ruled that Knox County had the right to hold back bonuses based on Taylor’s TVAAS. And he said the court did not have authority to tell the state legislature to come up with a different way to factor student learning into teachers’ ratings.

“It bears repeating that Plaintiffs’ concerns about the statistical imprecision of TVAAS are not unfounded,” the opinion reads. “However, this Court’s role is extremely limited. The judiciary is not empowered to second-guess the wisdom of the Tennessee legislature’s approach to solving the problems facing public education.”....