Those Cheating Teachers! A New Freakonomics Marketplace Podcast

This year alone has seen teacher-cheating scandals in Washington, D.C., Los Angeles, Atlanta, and elsewhere; in this week’s Times, Sharon Otterman reports how New York State is trying to curtail cheating and offers some specific instances of past cheating:

A charter school teacher warned her third graders that a standardized test question was “tricky,” and they all changed their answers. A high school coach in Brooklyn called a student into the hallway and slipped her a completed answer sheet in a newspaper. In the Bronx, a principal convened Finish Your Lab Days, where biology students ended up copying answers for work they never did.

(Comstock)

This comes as little surprise to Steve Levitt, who several years ago recognized what most legislators and school administrators were unable (or unwilling?) to foresee: that the introduction of high-stakes testing would create incentives that might encourage some teachers (especially bad ones) to cheat on behalf of their students. So he developed an algorithm to catch cheaters, which was so successful that then-Chicago schools chief Arne Duncan brought Levitt in to help identify and fire cheating Chicago teachers.

In the latest Freakonomics Marketplace podcast (you can download/subscribe at iTunes, get the RSS feed, listen live via the media player above, or read the transcript below), Kai Ryssdal, Steve Levitt, and Stephen Dubner talk about the misaligned incentives of standardized testing, and how, when a rule change gives people incentive to behave badly, a small share of them inevitably will.

Here’s where to find Marketplace on the radio near you.

 

Audio Transcript

KAI RYSSDAL: Time now for a little Freakonomics Radio. It's that moment every couple of weeks where we talk to Stephen Dubner, the co-author of the books and the blog of the same name, joined this week by the other guy, the other co-author, Steven Levitt from the University of Chicago. Guys, welcome to the program.

STEPHEN DUBNER: Hey Kai, how are you?

STEVEN LEVITT: Good to be here.

RYSSDAL: OK, so the topic du jour is cheating, specifically teacher cheating. There have been some big scandals lately in Atlanta and I think in Washington, D.C. And Levitt, in Chicago, like eight or 10 years ago, you actually caught some teacher cheating. So the first question is, based on your experience, can we say how many teachers cheat?

LEVITT: Looking at the data, our estimate at the time, we thought that 5 percent of all the elementary school classrooms in Chicago showed evidence that the teachers had cheated on behalf of their students on these exams.

RYSSDAL: Do we know why teachers cheat, Dubner?

DUBNER: Incentives, right? So these days we've just seen the No Child Left Behind law be scaled back quite a bit. What's happened is states have been given more latitude for how they're going to administer tests. But the fact is if you're a teacher and all of a sudden there's a new incentive in place for you to not do poorly in your class, then teachers all of a sudden have the kind of incentive that students used to have. And so there are some teachers -- now granted, it's a very, very small portion of them -- who will cheat on behalf of the student. In this case, literally erasing incorrect answers and filling in the correct ones -- not necessarily to help the kids, but to help themselves not look like they're bad performers.

RYSSDAL: Levitt, getting back to Chicago in 2002 and this scandal that you caught. The then guy running the Chicago school system, Arne Duncan, who's now the U.S. secretary of education, fired a whole bunch of teachers. Is that still currently policy, if you cheat you get fired?

LEVITT: I think there are two things you can do if you don't like cheating. One is what Arne did expose, which is he actually let us really ferret out who the cheaters were and they went to the trouble to hold the hearings and to fire a bunch of tires. The other option that's available to policymakers if they really don't like cheating is just make it harder to cheat. People don't cheat much on the LSAT or the SAT because the companies that provide those tests spend a lot of money and they make it hard to cheat. With school districts now, I think have an ambivalence towards cheating because they really do want higher test scores and so they often carry out these tests in ways that it's not hard at all for teachers to cheat. In fact, maybe even subtly encouraged.

RYSSDAL: But it's that whole spend-a-lot-of-money thing because public schools, as we know, in good parts of this country are out of money. How do you do it in a way that's cost effective?

LEVITT: I think you can even do it without spending money. So for instance, one thing I proposed to Chicago at the time was that instead of having the teachers in a school administer the testing themselves, you would just have the teachers go to a different school that day and have other teachers come in and administer the test.

DUBNER: Also, with as much money as the Department of Education spends now, Kai, and as high as unemployment is, it's not hard to imagine that you could enact a scenario where you could offer a part-time job. You know, we higher lots and lots of census takers, we could higher some exam proctors at a much, much lower cost and if it would help get the schools in the shape we want them to be in, it would be money well spent.

RYSSDAL: Not to end on a downer, but this is really the classic Sisyphean task. Right? People are always going to cheat and you're always going to be behind in ingenuity and ways to combat it. Right?

LEVITT: I don't think so. I think this one problem. Many problems are difficult, this one is easy. The real problem here is that the people who would have to make those choices and would have to spend that money, they actually don't want cheating to go away that badly. So I think it's a failure of incentives, not that we don't know how to fight this. So...

DUBNER: Levitt, you actually tried to start a company offering your catching-cheating-teacher skills to school districts around the country.

RYSSDAL: Is that right? Do tell. How did that go?

LEVITT: Nobody was interested. I mean, who wants to buy our product? One person came forward. In East St. Louis, they had just reconstituted the entire district. And one of the guys on the board said, 'We'd really like you to come and we think there might have been cheating. Could you look at the data?' And they couldn't get the school, even the overseeing board, could not get the school district, they wouldn't give us the data. So of all the business ideas I've ever had, that I think ranks at the very bottom in terms of profitability and likely success.

RYSSDAL: Freakonomics.com is the website. Steven Levitt and Stephen Dubner, guys thank you so much.

LEVITT: Thank you, Kai.

DUBNER: Talk to you soon, Kai.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 18

View All Comments »
  1. Mike B says:

    Hidden due to low comment rating. Click here to see.

    Disliked! Like or Dislike: Thumb up 1 Thumb down 14
    • Tony says:

      People complain about “high-stakes” testing all the time, but let’s be honest – the only metric that is currently used in public schools to measure performance and determine compensation is tenure. That is, the only way we measure teachers is the number of days they have worked in that particular school system.

      Are standardized tests a perfect system of measuring performance? No. Can they be improved/made less susceptible to cheating? Absolutely. Are they a better reflection of student’s education than a the number of years that student’s teacher has taught in that school system? You bet.

      Well-loved. Like or Dislike: Thumb up 14 Thumb down 5
      • Mike B says:

        Parents will either be satisfied or they won’t, students will be college ready or they won’t and students will be employable or they won’t. Life is a far better test than anything ETS can some up with.

        Thumb up 2 Thumb down 6
      • Ed Darrell says:

        Tenure is a false issue. Most teachers in the U.S. don’t get anything that can be counted as tenure — California and Texas, the two largest states, don’t have it at all, for example (Texas is an “employment at will” state that forbids union organizing among teachers). In states where tenure traditionally played a role, procedures for granting tenure have been changed so that no one entering the profession today can expect to get tenure, no matter how good they are.

        The argument demonstrates that critics don’t know what’s going on in schools, however. Especially, they don’t have a clue what motivates teachers to do well.

        Thumb up 0 Thumb down 0
  2. RGJ says:

    Steve algorithm (sp) column was brilliant.

    On the subject of “teach to the test”, I have a few thoughts.

    1. In the current state of public education in our inner cities, teaching successfully to some sort of set of gained knowledge hardly seems to be the worst outcome.

    2. Why are these teachers made aware of the test answers? The system should be like SATS or ACT — a universe of information with a representative group of questions produced at any given test.

    3. Apparently in the Chicago scenario teachers proctored their own tests, then hung around along with them in a room full of erasers afterward. Easy fix. The elelmentary school teachers proctor the high scool tests, the high school teachers proctor the middle school tests. Doesn’t cost a dime.

    The illogic of teacher’s unions saying you can’t measure teachers by tests results makes me bonkers. Kids are measured from the day they step off the kindergarten bus to the day they grab the diploma. Testing is the coin of the realm in education. For teachers to claim exemption throws into question the entire value of their profession. Maybe we should all be sitting under a tree with Plato, dudes.

    The only more maddening thing I’ve seen lately is teacher’s unions claiming that testing is not a fair way to evaluate teachers — because, look! Teachers will cheat!

    Boggles the mind.

    [WORDPRESS HASHCASH] The poster sent us ’0 which is not a hashcash value.

    Thumb up 7 Thumb down 9
  3. Joe says:

    So when we hold the performance of an individual on the test scores of the students, standardize them, and expect all students to do well. Of course the teachers are going to cheat.

    Thumb up 5 Thumb down 1
  4. Quentin says:

    I really want to listen to this episode, but the file is only 3.7MB and stops after 5:18. This is true for the listen option above. Is it possible for you to find the full length file? Thanks!

    Thumb up 1 Thumb down 0
  5. Mike Hunter says:

    That’s one sexy teacher!

    Thumb up 0 Thumb down 1
  6. Penelope says:

    All of the blaming teachers from people who have never done it bothers me. People who know what goes on in the inner city know that every part of the system is fundamentally broken. And they have a right to criticize because they know what they’re talking about. The truth is, the fault-finders have no idea what they would do with 50K of debt for a career that gave them little or none of the satisfaction that should be an inherent part of teaching. Add to that a house note. And add to that how you are sacrificing your own kids’ future by not earning more money doing something else. Add to that an economic environment that means that starting over could very possibly leave your family without health insurance. You really DON”T know what you would do when you’re not in that other person’s shoes.

    I wish every person who wanted to blame teachers for problems deeply rooted in our society’s line between rich and poor (and race, if we’re honest) HAD TO LIVE IT. Then get back to me. This is true for every politician who has great ideas on “how to hold teachers accountable” (like teachers created the problem) and would include Steve Levitt.

    Mr. Levitt, I challenge you to substitute teach in the inner city before you tell me that making more valid standardized test scores will fix education. The business idea was not bad because of the ambivalence of the people who would be spending money for it, it was bad because they knew the tests did not get to what was wrong and they penalized a group of people with partial and varying responsibility for the problem.

    Well-loved. Like or Dislike: Thumb up 10 Thumb down 1
    • Ed Darrell says:

      The “business idea” is bad because education is not a business, first, and what passes for “business ideas” in education would be laughed out of business, if applied.

      What business, for example, requires its employees to bring all of their tools? In most schools in the nation, teachers provide their own computer resources, often their own internet connections (if they want to use internet in the classroom), their own pencils and paper, their own textbooks (especially AP teachers), and more. IRS assumes every teacher will spend at least $600 annually on supplies “other businesses” provide to employees, but most teachers blow past that amount in the first couple of weeks of every year. Here in Texas, since Rick Perry, teachers must provide their own telephones for the classroom. Copy paper. Copies. Printers. Chalk, chalk boards, markers and white boards . . . you pick the classroom resource, and teachers in far too many schools must provide it, if it is to be available to aid the education of students. No business does that.

      What business claims its legal to provide no lunch break or bathroom breaks?

      What business makes employees responsible for inventory loaned out to customers, assuming no normal wear and tear?

      The “business model” doesn’t work in education because students are not widgets, firing all the bad ones is a urine-impoverished business model in the first place, and most people don’t know what makes businesses run, let alone what makes a school work.

      Thumb up 2 Thumb down 0
  7. David B. Cohen says:

    I’m a Freakonomics fan, but on this topic, the treatment of the topic missed some important points. I responded in this blog post:
    http://accomplishedcaliforniateachers.wordpress.com/2011/10/18/freakonomics-cheating/

    Thumb up 0 Thumb down 0
  8. Stepk says:

    So you guys have drunk the Kool-aid. I love many of your pieces because you tear apart at the way we look at things and make us look at what we really know rather than what we think we know. In this case you are asking all the wrong questions. The question you should be asking is whether the test scores of students is a valid way to judge the ability and performance of teachers? When we are building widgets it is easy to set standards because you expect every widget to come out the same; you use the same materials, you use the same molds, etc. Compared with students, it would be like giving your manufacturer different molds and materials every day and expecting them to still produce the same widget. Testing, even over the long term, does not take into account the many factors that influence the outcome, even using the statistical modeling they use. For example, how could the PSAT graders know that the night before I took the test my Grandfather passed away and it deeply affected me (I was an “A” student and didn’t do well even though I received 100% on every regent exam I took.). The reason some teacher’s cheat (and I am sure many others think about cheating) is because they know they have little control over the outcome of their “product” because they have little control of the “material” they work with. The other choice that teachers have is to move to a school where either testing is not such a critical part of the review process or they move to a school that has less of the factors that can impede their students performance. A good Freakonomics discussion would be centered around this not how to keep teachers from cheating by increasing surveillance or removing them from the grading process.

    Thumb up 3 Thumb down 0
    • Ed Darrell says:

      Are test scores valid methods to evaluate student achievement is the first question we should ask. College admissions officers say 80 years of experience with the best students tells them “no.”

      THEN we can ask, since test scores are inappropriate for monitoring student learning in specific, is it appropriate to use a device the student takes to determine whether the teacher has taught well, knowing that the test only measures student performance at that moment, and doesn’t measure well whether the student has learned?

      There’s a difference between learning and performing on a test. We know we’re not measuring learning with tests — is it fair to then assume we can measure teaching, by an inaccurate measure of what students learned?

      Thumb up 1 Thumb down 0