How to Game a Grading Curve

Students in three of Professor Peter Fröhlich‘s computer programming classes at Johns Hopkins University recently devised a method to game their final grades.  Frolich grades exams on a curve — the highest grade in the class, whatever it may be, becomes 100 percent, and “everybody else gets a percentage relative to it.”  So students collectively planned a boycott:

Because they all did, a zero was the highest score in each of the three classes, which, by the rules of Fröhlich’s curve, meant every student received an A.

“The students refused to come into the room and take the exam, so we sat there for a while: me on the inside, they on the outside,” Fröhlich said. “After about 20-30 minutes I would give up…. Then we all left.” The students waited outside the rooms to make sure that others honored the boycott, and were poised to go in if someone had. No one did, though.

Catherine Rampell discusses the strategy:

This is an amazing game theory outcome, and not one that economists would likely predict…

In this one-off final exam, there are at least two Bayesian Nash equilibria (a stable outcome, where no student has an incentive to change his strategy after considering the other students’ strategies). Equilibrium #1 is that no one takes the test, and equilibrium #2 is that everyone takes the test. Both equilibria depend on what all the students believe their peers will do.

If all students believe that everyone will boycott with 100 percent certainty, then everyone should boycott (#1). But if anyone suspects that even one person will break the boycott, then at least someone will break the boycott, and everyone else will update their choices and decide to take the exam (#2).

The problem is that Nash equilibrium theory alone doesn’t tell us what the students are more likely to do. Economists would say that the first equilibrium, where no one takes the exam, is unlikely to result because it is not “trembling hand perfect,” an idea that helped win Reinhard Selten win the Nobel Memorial Prize in Economics.

Fröhlich was impressed by the students’ scheme. “The students learned that by coming together, they can achieve something that individually they could never have done,” he wrote in an e-mail. “At a school that is known (perhaps unjustly) for competitiveness I didn’t expect that reaching such an agreement was possible.”  He has, however, revised his grading policy to prevent future gaming.

(HT: Sarah Martin)


Garam

I wonder, if I had done this in my experimental econ class, would we have deserved an A+ for putting our knowledge of game theory into action?

gpadugan@ymail.com

this makes no sense. the professor is still held accountable to the rules of the school. so the school could (and should over rule). besides, if we go by the numbers (which is what the students did) then go all the way. 0% has no value. It can not be 100% and 0% so technically speaking the test is null it and should be required to be re taken. you can say everyone passed with 100 or that everyone failed with 0. The professor could say that they all get a 0 instead of 100. he's just doing this for the press. I'm an adjunct professor and a sysllubus is not a contract, not a legal document, it's a guide. to take it literal as the students did is quite alarming indeed. they shall have plenty of time to continue this line of thinking in the occupy camps when they can't get a job.

Et

You are mistaken. The professors devise the grading mechanics, not the school (within reason). This was a flaw present in Professor Fröhlich‘s syllabus and was made apparent in the syllabus reviewed at the start of every semester. I've taken a number of his courses and it was discussed semi-jokingly a number of times in the past. Nobody had actually taken advantage of it before, perhaps in part because of the difficulty of guaranteeing collaboration of all students in previous years.

The key difference this year was that the class started using a discussion board (Piazza, specifically) to ask and answer questions related to course material throughout the semester, and participation was mandatory (graded), so all students in the course were active on the discussion board to some degree. It was on this board, as well as a google document, that the class organized (and debated, quite lengthily) the plan to not take the final.

The professor was aware of this discussion, but could not simply change his grading policies mid-semester. It was not until after the semester ended that he was able to modify the policy. Thus, the plan went through and was successful on several occasions in 3 different courses throughout the semester.

Source: I was in one of the courses that participated.

Read more...

Hopkins Student

Actual Hopkins student here with friends who were in that class and witnessed the whole thing from beginning to end. As rosy and clever as this article makes the whole thing sound, it wasn't an all smiles "we gamed the system!" situation.

It was my understanding that this curve for the final had been a long-standing policy of the Professor. There was one student who outright refused to boycott the test (at least the only one who publicly declared his intentions) and was adamant about his taking the final. He had all kinds of moral reasoning about it being the right thing to do and how he had to do it for himself.

Given the nature of the curve, the other students in the class knew that the only way to ace the final and not have to take it was to ensure that not a single student took it. The situation quickly got nasty as the class tried to coerce the student into not taking the exam. The Professor had to step in and expressed his disappointment. Ultimately, the class's repeated statements making him out to be an "asshole" caused him to give in to the pressure.

The Professor's policy changed not because the students figured out a way to game the system, but that this was the first time a student adamantly refused to boycott the final and was nearly physically impeded from doing so.

Read more...

Et

This is correct: the policy was long-standing. There was quite a bit of hostility regarding the ethics of skipping the exam as well. There was a small book's worth of discussion on the topic, some of which was removed from the discussion board due to hostility.

curveBreaker

"...using non-traditional techniques and collaborative learning to surmount the obstacles teachers had put in their way" - (wiki - academic dishonesty)

I respect John Hopkins as an establishment- I hope these student will become leaders one day for their talents, not their politics, their antics.

fair?

"Hey look! I skipped this test and got an A!"

um? bravo?

Andrew Vaughan

Equilibrium #1 is not unstable in this scenario. If a student were to "break the picket line", his or her peers would shortly follow him, reverting the expected relative score of each student to its default value. Given that there is zero expected gain to be had by walking into the room (and probably a bad time with his friends the next day), and positive expected gain if all remain outside, this seems like a stable Nash equilibrium to me.

Timepiece

The very first time that grading on a curve was explained to me (8th grade history), the teacher acknowledged that if no one filled in any answers, we would all get 100%. He then immediately added something to the effect of "but you know that one 'sniveling little weasel' will fill them in, and everyone else will get a 0". This being a fairly competitive private school, I'm sure we all immediately concluded that he was correct, and chose to fill in the answers.

Though I still wish I'd tried a campaign to leave the essay blank. That was also my first time using a blue book - in 8th grade!

Fritz Schenk

As I understand the story, no one scored a zero. No one even walked into the class to take the test. I was a high school teacher (I know...not college) and when a student did not show up for the final, I could not give a zero as a score. I had to assign an incomplete and the final grade was incomplete until the score was made up. Unless there was some school rule or class rule that said missed tests were treated as if it was taken and that no points were earned, the test was not taken so no one could have gotten a zero.

Alex

Nash Equilibrium problems are applied only to non-cooperative (ie: no collusion) situations. Since the students are able to discuss and plan their decisions, Nash Equilibrium cannot be analysed. Traditional game theory would, in fact, predict that the students behave this way.

Professor with a clue

What this demonstrates is the low quality of the course, the professor, the students and the institution.

At least in this instance.

If you are simply using grades, marks or other metrics without any grounding in what a course is designed to teach students, or having any criteria which such learning can be assessed, then the whole exercise is a waste of time.

Does the course have a syllabus?
Does the course have clearly specified learning outcomes or objectives?
Does the course clearly indicate what (if anything) is being taught?
What skills, abilities or knowledge are students given the opportunity to acquire and demonstrate?

If students (and professors) took these issues more seriously, then perhaps we could all improve the quality of higher education.

After all, from my perspective, knowing that a student got 100 percent for a course because they gamed the system does not satisfy my desire to know "can this student do/know/demonstrate X"?

In addition, it makes me question the professors ability to actually teach X.

Read more...