The once-respectable Los Angeles Times is leveraging its dwindling platform to attack individual teachers under the guise of data transparency. The editorial board Times won a court case allowing them to use a highly contentious, self-designed algorithm to rank the best and worst teachers in the Los Angeles Unified School District. Neither the suicide of one of the shamed teachers, nor the widespread criticism of the statistical methods have aroused the editorial board’s editors’ better judgment.
Many school districts, such as the LAUSD, estimate teacher performance based off of their students’ standardized test results. So-called “value-added modeling” attempts to estimate a teacher’s relative abilities based on how they expect students to do given their past performances.
The school district will be forced to release the data on teacher evaluations to The Times for publication. While I’m all for transparency of government data, there are a few glaring problems with value-added scores that the public might not be aware of.
1. Value-added measures are as unstable as a chain-smoker on a flight from LA to Japan. Teacher ratings often swing wildly from year to year and are sensitive to tiny changes in the statistical methods. The University of Colorado at Boulder’s National Education Policy Center found that only about half (46.4 percent) of LAUSD teachers retained their same effectiveness rating under slight tweaks to the model [PDF].
Specifically, the NEPC added measures of school ranking and early elementary grades into their own value-added model to see how it might disrupt the rankings (and it did). There’s many reasons why such variables might not have been originally included: adding in past performance and school transfers make it difficult to know what in the history of a student ultimately led to their current abilities.
Statistical geeks can debate the best models, but if a series of very reasonable decisions leads to radically different rankings, it’s way too unstable to shame a teacher in a national newspaper.
2. Standardize tests suck at measuring the value of a teacher. “Test scores largely reflect whom a teacher teaches, not how well they teach,” notes Stanford Professor of Education, Linda Darling-Hammond. “In particular, teachers show lower gains when they have large numbers of new English-learners and students with disabilities than when they teach other students.”
The LA Times appears oblivious to this well-known fact. In an email, a representative tells me, “Research has repeatedly found that teachers are the single most important school-related factor in a child’s education.”
False. Parenting, motivation, and IQ are at least as important, if not vastly more important, to the success of a student than a teacher. Teachers can bring out the best in a student, but a child from a broken home and with an abusive parent just isn’t going to do as well.
3. It’s not okay to shame everyday citizens. Assume for a moment that the LA Times pulled off a statistical miracle and overcame all of the criticisms of value-added methodology, it’s still awful to shame people.
Here is the Times’ explanation to me:
“The Times is committed to reporting on the issues and events that are important to Southern Californians and education is of primary concern to our community. We published the “Grading the Teachers” series and value-added data analysis because parents and the public have a right to some form of objective evaluation of LAUSD teacher effectiveness. The Times value-added rating, which was based entirely on public-record information, should be considered as only one component in overall teacher assessment. It’s also important to note that all teachers listed in The Times database were given an opportunity to view and comment on their value-added ratings in advance of their publication, and hundreds did so.”
That’s a beautiful theory, but in practice the list paints targets on teachers’ backs for tiger-moms and paparazzi press. When the New York Post publicized the name of the “worst teacher,” reporters hounded her and family.
If we lived in a perfect marketplace of ideas where nuance was currency and readers spent more than 30 seconds on a post, The Times might have a better case. But we don’t, and their sloppy editorial decisions are going to hurt innocent teachers.