Have D.C.’s “Best Schools” Been Cheating?

A USA Today investigation found what appears to be compelling evidence of teacher cheating in Washington, D.C., schools that were heralded as major successes due to test score increases.  The smoking gun: too many erasures resulting in answers being changed from wrong to right.  The numbers are so extreme that they do seem to indicate that massive cheating occurred. Not surprisingly, the school district is none-too-eager to investigate — especially because teachers at the schools were given big bonuses as rewards for the test-score improvement. Though on Tuesday, acting D.C. schools chancellor Kaya Henderson did request a review.

Photo: iStockphoto

When Brian Jacob and I investigated teacher cheating in Chicago schools, which we described in Freakonomics, we didn’t use erasure analysis.  Rather, we developed new tools for identifying strings of unlikely answers.

You might ask why we didn’t use erasures, when it is such an obvious approach.  The answer: unlike the D.C. schools, the Chicago schools did not farm out grading of the test exams to a third party.  What got the D.C. schools in trouble is that the third party routinely analyzed erasure patterns.  The internal group that scored the exams in Chicago did not routinely look for erasures; that was only done when there were suspicions about particular classrooms.

Conveniently, there was an acute shortage of storage space in the Chicago warehouse where the test forms were graded.  This, of course, necessitated that all the actual test forms be destroyed and disposed of shortly after the test was administered.

Some teachers in D.C. are surely wishing there had been a similar lack of storage space in our nation’s capital.

(HT: David Clayton)

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 10

View All Comments »
  1. Jason says:

    This is another example of Kerr’s Folly in practice. The district likely expected that the teachers would increase their efforts to achieve the financial bonus. It is also likely, however, that the teachers believed that their additional efforts would have little impact on standardized test performance. Therefore, to get the bonus they cheated. Another example of hoping for A and rewarding for B.

    Thumb up 1 Thumb down 0
  2. Doug says:

    This is some great reporting:

    “Noyes is one of 103 public schools here that have had erasure rates that surpassed D.C. averages at least once since 2008. That’s more than half of D.C. schools.”

    The funny thing about averages is that about half of the numbers are usually higher than them.

    Thumb up 0 Thumb down 0
    • Bayoris the Goatherd says:

      Point taken, although in this case you would expect the distribution to be skewed right, with a tail of cheaters, meaning that more than half of the values would be below the mean.

      Thumb up 1 Thumb down 0
  3. gpo says:

    If they would have just have teachers proctor different classrooms and not their own. Teachers who are getting a bonus should never touch their own students’ exam paper. Or just bring in some subs to have a second person in the room. Or have students take the test in an auditorium. So simple to stop cheating.

    Ironically this action only works once or in a single year. Think about it. Student have a baseline of scores from the previous year. Now this year you cheated and bumped up the scores. So now the students next year will really improve less or not at all. Those teachers just hurt the teachers who get those students next year.

    Thumb up 1 Thumb down 0
    • Joshua Northey says:

      I think there is ample evidence from human history that:

      A) People do not pay attention to time horizons that long.
      B) People could care less about their co-workers if there is money involved unless they are extremely close friends.

      Thumb up 1 Thumb down 0
  4. Mike B says:

    Teachers should start to apply fake erasure marks to all of their tests (or encourage students to make them themselves) so that the erasure metric becomes less reliable.

    Thumb up 0 Thumb down 1
  5. Jen says:

    I’m not sure why there’s an assumption that it’s the teachers cheating in this way. This looks more like the cases wherein a principal and a trusted associate (or associates) change answers or demanding that teachers do so, though the more people involved the more likely that someone would blow the whistle. The article says the teachers ALSO thought there was something going on with the scores.

    At least in my state, it would be nearly impossible to have the tests alone long enough to do any sort of large-scale erasing and correcting. You pick them up on the way to administer and they are to never be let out of your sight. The students come in, they take the tests, they stay in the room the whole time. Once you dismiss them, the box is to go back to the locked office where they are kept. If you kept your box for any length of extra time, it would be noticed.

    Principals though (and maybe a “coach” — another reform concept) ? Plenty of time and opportunity to unlock doors that no one else has a key to and do whatever.

    One classroom or two with odd patterns — that’s a teacher (harder to do though, when kids are likely to tell on you one way or another). Whole schools? That’s more likely higher up the food chain.

    Thumb up 2 Thumb down 0
  6. Brent Thompson says:

    My school uses standardized test scores to inform practice and continuously improve our teaching. Using the tests as one measure to help improve my teaching is the only reason I see for using standardized tests. They aren’t the best way to measure student performance and there can be a myriad of factors influencing test scores that are out of a teachers control. No wonder there is such a huge temptation to cheat. I don’t know anyone who goes to work thinking “Today I’m going to do a crappy job”. Maybe you need to spend these bonuses on supporting teachers who are struggling and develop a more collaborative supportive system. Teaching effectively is a hard job and there is always more to learn. We are always helping each other and sharing our work at my school and I don’t know if it would be that way if we were in a high stakes testing atmosphere.

    Thumb up 0 Thumb down 1
  7. Piotr says:

    I am amazed, that in US (proudly the most innovative and developed country in the world) such a simple set of standardised forms is still checked by a live person instead of being automated. Any scanner or camera and a piece of OCR/pattern recognition software (OK, probably customised, but this can be done for really small money today) would allow for marking these tests in hours, not days and without a chance for fraud. Tens of software companies would crave for doing it – probably mostly for fame.

    If someone speaks to S. Jobs, B. Gates, M. Dell or some other rich individual from IT business – they would probably happily finance this project because it means both fame for them as individual and business for their companies.

    But maybe the problem is – it would show the real level of US education?

    Thumb up 0 Thumb down 1
  8. Marsen Sedgwick says:

    I teach in the NYC department of education. I recently learned of a well-regarded principal who instructed teachers proctoring the January Regents exams to read aloud the exams in such a way as to make the correct answer obvious to the observant student. Some new teachers were shocked at this demand, but were forced into complying with it. From what I’ve been told here (and this is my first year), cheating on important test is rampant, and it doesn’t always come in the form of erasures. Finally, there are very few safeguards in place against cheating.

    Thumb up 1 Thumb down 0