Evaluating Teachers: What About Doing it the Old-Fashioned Way?

Photo: kevin dooley

As part of our ongoing obsession with improving public education, we bring you a new study from Jonah E. Rockoff of Columbia Business School and Cecilia Speroni, a former doctoral student at Columbia’s Teachers College, that explores the power of objective and subjective teacher evaluations. While an emphasis on merit pay and test scores can lead to widespread cheating (as covered in this week’s Freakonomics Marketplace podcast), not to mention the occasional Matt Damon outburst, Rockoff and Speroni offer a potential glimmer of hope for the old-fashioned approach: the study finds that subjective teacher evaluations for New York City teachers had strong predictive power for future student performance. Here’s the abstract:

A substantial literature documents large variation in teacher effectiveness at raising student achievement, providing motivation to identify highly effective and ineffective teachers early in their careers. Using data from New York City public schools, we estimate whether subjective evaluations of teacher effectiveness have predictive power for the achievement gains made by teachers’ future students. We find that these subjective evaluations have substantial power, comparable with and complementary to objective measures of teacher effectiveness taken from a teacher’s first year in the classroom.

The authors used subjective evaluations from both applicant interviews for a certification program and mentors who worked with teachers their first year, as well as objective evaluations “based on student achievement data from their first year of teaching.”

Among the many knocks on the new push for objective evaluation measures is that they fail to capture the nuances of teaching, which the authors believe traditional subjective methods do much better.

This is an especially noteworthy finding, considering that variation in subjective evaluations likely also captures facets of teaching skill that may affect outcomes not captured by standardized tests.

But not so fast — the researchers noted two important things about teachers that received high evaluations from their mentors: their students had higher-than-average test scores prior to entering their classrooms, and they taught fewer minority or economically disadvantaged students. This could mean that better teachers go to schools with better students, or that a mentor’s impression of a teacher is influenced simply by their sample of students.

Rockoff and Speroni call for more research into the nuances and possibilities of subjective and objective testing, writing that “policymakers will need to have a better understanding of the power and limitations of the measures they use in establishing incentives and accountability for teachers.” Which seems to sum up the current debate quite well.