Those Cheating Teachers! A New Freakonomics Marketplace Podcast

This year alone has seen teacher-cheating scandals in Washington, D.C., Los Angeles, Atlanta, and elsewhere; in this week’s Times, Sharon Otterman reports how New York State is trying to curtail cheating and offers some specific instances of past cheating:

A charter school teacher warned her third graders that a standardized test question was “tricky,” and they all changed their answers. A high school coach in Brooklyn called a student into the hallway and slipped her a completed answer sheet in a newspaper. In the Bronx, a principal convened Finish Your Lab Days, where biology students ended up copying answers for work they never did.

(Comstock)

This comes as little surprise to Steve Levitt, who several years ago recognized what most legislators and school administrators were unable (or unwilling?) to foresee: that the introduction of high-stakes testing would create incentives that might encourage some teachers (especially bad ones) to cheat on behalf of their students. So he developed an algorithm to catch cheaters, which was so successful that then-Chicago schools chief Arne Duncan brought Levitt in to help identify and fire cheating Chicago teachers.

In the latest Freakonomics Marketplace podcast (you can download/subscribe at iTunes, get the RSS feed, listen live via the media player above, or read the transcript below), Kai Ryssdal, Steve Levitt, and Stephen Dubner talk about the misaligned incentives of standardized testing, and how, when a rule change gives people incentive to behave badly, a small share of them inevitably will.

Here’s where to find Marketplace on the radio near you.

 


Mike B

Since these sorts of high stakes standardized tests are more often than not detrimental to the educational process wouldn't it in fact for teachers and districts to cheat on the tests as a way to get the state off their backs so they can do their job? All schools already have a feedback mechanism, its call the local school board election.

Tony

People complain about "high-stakes" testing all the time, but let's be honest - the only metric that is currently used in public schools to measure performance and determine compensation is tenure. That is, the only way we measure teachers is the number of days they have worked in that particular school system.

Are standardized tests a perfect system of measuring performance? No. Can they be improved/made less susceptible to cheating? Absolutely. Are they a better reflection of student's education than a the number of years that student's teacher has taught in that school system? You bet.

Mike B

Parents will either be satisfied or they won't, students will be college ready or they won't and students will be employable or they won't. Life is a far better test than anything ETS can some up with.

RGJ

Steve algorithm (sp) column was brilliant.

On the subject of "teach to the test", I have a few thoughts.

1. In the current state of public education in our inner cities, teaching successfully to some sort of set of gained knowledge hardly seems to be the worst outcome.

2. Why are these teachers made aware of the test answers? The system should be like SATS or ACT -- a universe of information with a representative group of questions produced at any given test.

3. Apparently in the Chicago scenario teachers proctored their own tests, then hung around along with them in a room full of erasers afterward. Easy fix. The elelmentary school teachers proctor the high scool tests, the high school teachers proctor the middle school tests. Doesn't cost a dime.

The illogic of teacher's unions saying you can't measure teachers by tests results makes me bonkers. Kids are measured from the day they step off the kindergarten bus to the day they grab the diploma. Testing is the coin of the realm in education. For teachers to claim exemption throws into question the entire value of their profession. Maybe we should all be sitting under a tree with Plato, dudes.

The only more maddening thing I've seen lately is teacher's unions claiming that testing is not a fair way to evaluate teachers -- because, look! Teachers will cheat!

Boggles the mind.

[WORDPRESS HASHCASH] The poster sent us '0 which is not a hashcash value.

Read more...

Joe

So when we hold the performance of an individual on the test scores of the students, standardize them, and expect all students to do well. Of course the teachers are going to cheat.

Quentin

I really want to listen to this episode, but the file is only 3.7MB and stops after 5:18. This is true for the listen option above. Is it possible for you to find the full length file? Thanks!

Mike Hunter

That's one sexy teacher!

Penelope

All of the blaming teachers from people who have never done it bothers me. People who know what goes on in the inner city know that every part of the system is fundamentally broken. And they have a right to criticize because they know what they're talking about. The truth is, the fault-finders have no idea what they would do with 50K of debt for a career that gave them little or none of the satisfaction that should be an inherent part of teaching. Add to that a house note. And add to that how you are sacrificing your own kids' future by not earning more money doing something else. Add to that an economic environment that means that starting over could very possibly leave your family without health insurance. You really DON"T know what you would do when you're not in that other person's shoes.

I wish every person who wanted to blame teachers for problems deeply rooted in our society's line between rich and poor (and race, if we're honest) HAD TO LIVE IT. Then get back to me. This is true for every politician who has great ideas on "how to hold teachers accountable" (like teachers created the problem) and would include Steve Levitt.

Mr. Levitt, I challenge you to substitute teach in the inner city before you tell me that making more valid standardized test scores will fix education. The business idea was not bad because of the ambivalence of the people who would be spending money for it, it was bad because they knew the tests did not get to what was wrong and they penalized a group of people with partial and varying responsibility for the problem.

Read more...

Ed Darrell

The "business idea" is bad because education is not a business, first, and what passes for "business ideas" in education would be laughed out of business, if applied.

What business, for example, requires its employees to bring all of their tools? In most schools in the nation, teachers provide their own computer resources, often their own internet connections (if they want to use internet in the classroom), their own pencils and paper, their own textbooks (especially AP teachers), and more. IRS assumes every teacher will spend at least $600 annually on supplies "other businesses" provide to employees, but most teachers blow past that amount in the first couple of weeks of every year. Here in Texas, since Rick Perry, teachers must provide their own telephones for the classroom. Copy paper. Copies. Printers. Chalk, chalk boards, markers and white boards . . . you pick the classroom resource, and teachers in far too many schools must provide it, if it is to be available to aid the education of students. No business does that.

What business claims its legal to provide no lunch break or bathroom breaks?

What business makes employees responsible for inventory loaned out to customers, assuming no normal wear and tear?

The "business model" doesn't work in education because students are not widgets, firing all the bad ones is a urine-impoverished business model in the first place, and most people don't know what makes businesses run, let alone what makes a school work.

Read more...

David B. Cohen

I'm a Freakonomics fan, but on this topic, the treatment of the topic missed some important points. I responded in this blog post:
http://accomplishedcaliforniateachers.wordpress.com/2011/10/18/freakonomics-cheating/

Stepk

So you guys have drunk the Kool-aid. I love many of your pieces because you tear apart at the way we look at things and make us look at what we really know rather than what we think we know. In this case you are asking all the wrong questions. The question you should be asking is whether the test scores of students is a valid way to judge the ability and performance of teachers? When we are building widgets it is easy to set standards because you expect every widget to come out the same; you use the same materials, you use the same molds, etc. Compared with students, it would be like giving your manufacturer different molds and materials every day and expecting them to still produce the same widget. Testing, even over the long term, does not take into account the many factors that influence the outcome, even using the statistical modeling they use. For example, how could the PSAT graders know that the night before I took the test my Grandfather passed away and it deeply affected me (I was an "A" student and didn't do well even though I received 100% on every regent exam I took.). The reason some teacher's cheat (and I am sure many others think about cheating) is because they know they have little control over the outcome of their "product" because they have little control of the "material" they work with. The other choice that teachers have is to move to a school where either testing is not such a critical part of the review process or they move to a school that has less of the factors that can impede their students performance. A good Freakonomics discussion would be centered around this not how to keep teachers from cheating by increasing surveillance or removing them from the grading process.

Read more...

Ed Darrell

Are test scores valid methods to evaluate student achievement is the first question we should ask. College admissions officers say 80 years of experience with the best students tells them "no."

THEN we can ask, since test scores are inappropriate for monitoring student learning in specific, is it appropriate to use a device the student takes to determine whether the teacher has taught well, knowing that the test only measures student performance at that moment, and doesn't measure well whether the student has learned?

There's a difference between learning and performing on a test. We know we're not measuring learning with tests -- is it fair to then assume we can measure teaching, by an inaccurate measure of what students learned?

James Briggs

One of the many breakthroughs for me with Freakonomics was in understanding that some people will cheat if there are enough incentives. Politicians are the most functional people in the world and their primary goal is to keep their jobs or get a better one. An objective test done one a year makes it easy to show where a school stands. That kind of test the one that is the most useful to them. I am sure they knew that a test like that would encourage cheating. If the cheaters weren’t caught the higher scores would make the politicians look good and if the cheaters were caught the blame would go elsewhere. Unless the incentives change for the politicians the same types of tests will continue to be used.

James Murray

Widgets are the wrong analogy for exactly the reason you mention - the raw material, the kids, change dramatically - with the catchment area, the stream the child is in, and so on.

Schools add value by taking whatever kids they are given and making them more 'educated' by the end of the year.

So, find out where and how the the level of the concept 'educated' can be measured.

Then,measure this as the kids go into the school year and then when they end it.

That difference is taken as a result of the schooling and becomes the measurement of how well that teacher has done.

Doing the same for schools and their overall effect on the year whole and that tells you how well the principle and their team are doing.

James Briggs

One big problem in our society is we misapply the production line model. It certainly does not belong in education. Teachers little control over the education they give. There is a lesson plan for everyday of the school year determined by some expert who doesn’t know that children are individuals and have individual differences. Some children learn best through memorization while other will benefit by teaching concepts. I graduated at the bottom of my class and I was one of the best educated but I suffered from depression and ADHD. And I hung out with other kids like me. We were reading college level books and discussing them while ignoring our home-work. One reform I would like to see is to give the teachers the resources that they need to teach and let the teacher teach their students the way they think is best and then test their students fairly.

Don Kamp

Links to Washington Post blog reports of teacher cheating in DC school:

http://www.washingtonpost.com/blogs/class-struggle/post/dc-principal-slammed-for-reporting-cheating/2013/01/10/24e7b47a-5ac7-11e2-9fa9-5fbdc9530eb9_blog.html

http://www.washingtonpost.com/blogs/class-struggle/post/dc-schools-cheating-report-thin-and-biased/2012/08/11/569d3f5c-e40d-11e1-a25e-15067bb31849_blog.html

Cody

Standardized testing is the worst way to acquire information about a school, you give a test to a bunch of kids that don't care if the majority scores low then the school gets less funding, if the school has less funding classes are bigger it's harder to learn and grades go down for more students. but the schools that score well get more money and can continue making their students smarter. Why does it make any sense to give less money to the schools that obviously need more attention and learning because of a lack of budget? It's a vicious cycle that I have witnessed first hand at my own high school.

Daniel Dreyer

I just wanted to toss on here props to the freakonomics team. You're getting a lot of weird flack for off-subject stuff in this comment thread. Thanks for the concise and interesting look at teachers cheating on standardized testing and ways to improve the current system WITHOUT over-complicating things with broader systemic issues. Well done. Enjoyed it.

Linda

The Atlanta Journal reported the average bonus of the charged Atlanta teachers was, $2600. The bonus of the superintendent, over 10 years was $345,000. The Guardian reported, 90% of the schools' principals were replaced, because shaded in bubbles didn't match a target.

Expectations of adherence to professional standards, by people who have no power nor autonomy and who face firing for lack of obedience, was, in the case of the Atlanta teachers, IMO, prosecutorial overreach and the punishment was judicial abuse.
If there is justice, the teachers will win on appeal.

For comparison, not one executive from the financial sector has been brought up on charges for the fraud related to the 2008 financial crisis.