# How to Game a Grading Curve

Students in three of Professor Peter Fröhlich‘s computer programming classes at Johns Hopkins University recently devised a method to game their final grades.  Frolich grades exams on a curve — the highest grade in the class, whatever it may be, becomes 100 percent, and “everybody else gets a percentage relative to it.”  So students collectively planned a boycott:

Because they all did, a zero was the highest score in each of the three classes, which, by the rules of Fröhlich’s curve, meant every student received an A.

“The students refused to come into the room and take the exam, so we sat there for a while: me on the inside, they on the outside,” Fröhlich said. “After about 20-30 minutes I would give up…. Then we all left.” The students waited outside the rooms to make sure that others honored the boycott, and were poised to go in if someone had. No one did, though.

Catherine Rampell discusses the strategy:

This is an amazing game theory outcome, and not one that economists would likely predict…

In this one-off final exam, there are at least two Bayesian Nash equilibria (a stable outcome, where no student has an incentive to change his strategy after considering the other students’ strategies). Equilibrium #1 is that no one takes the test, and equilibrium #2 is that everyone takes the test. Both equilibria depend on what all the students believe their peers will do.

If all students believe that everyone will boycott with 100 percent certainty, then everyone should boycott (#1). But if anyone suspects that even one person will break the boycott, then at least someone will break the boycott, and everyone else will update their choices and decide to take the exam (#2).

The problem is that Nash equilibrium theory alone doesn’t tell us what the students are more likely to do. Economists would say that the first equilibrium, where no one takes the exam, is unlikely to result because it is not “trembling hand perfect,” an idea that helped win Reinhard Selten win the Nobel Memorial Prize in Economics.

Fröhlich was impressed by the students’ scheme. “The students learned that by coming together, they can achieve something that individually they could never have done,” he wrote in an e-mail. “At a school that is known (perhaps unjustly) for competitiveness I didn’t expect that reaching such an agreement was possible.”  He has, however, revised his grading policy to prevent future gaming.

(HT: Sarah Martin)

#### tung bo

The students are also betting that their cohesiveness and game theoretical thinking will impress Prof. Frohlich. They stood to lose if the Professor took a legalistic approach: since none of the student participated in the exam by going into the room, the Professor can treat that as a forfeit. That means no grades for any student or failling grades for forfeits.

This risk should force some and then most student to defect given a large enough class. With a small class, it was possible to enforce nonparticipation.
Yet, the Professor can also offer a make-up take home test to replace the in class test. Without the public mutual monitoring, almost certainly some will defect.

Ultimately, the students were betting that Prof. Frolich was a 'nice' person and would not follow these other alternatives.

#### Matt

In high school I had a math teacher who had a similar grading curve policy for the final. A bunch of hands went up after she finished explaining it, at which point she amended it by saying that if everyone did very poorly (e.g., scored 0s) she would nullify the test and make us take a new one (and half the hands went down).

It all comes down to how much one thinks the teacher will appreciate the effort.

#### Jaime

It seems to me that this result was in all likelihood triggered by some previous action (on behalf of the teachers, university, etc) that removed any incentive for competition in the class as this is one of two mechanisms I see that would lead to this equilibrium. Alternatively it is possible some group of students with little chance of getting a high grade coerced their peers to take part in the plan.

This latter mechanism is less likely and not stable, as students actually have an incentive (that increases as more students are coerced into getting a 0%) to stand up to those that want to attempt the 0% grade strategy as the resisting student's grade is bound to see a greater difference compared to those other students in their class and hence end up with a more outstanding degree than his/her peers.

The former mechanism could be, for example, that the given class would not count for the final grade of the course/degree but a pass was still required to successfully complete the course/grade. This situation creates a stable equilibrium where all students are highly incentived to carry out such a strategy as there is no comparable gain between the "0%, 0 effort strategy" and "each student individually studying to get a passing grade" strategy, while there is a clear cost difference (in effort) that pushes the group towards the consensus arrived at in the anecdote.

Would be nice to get some background on this.

#### Joe

Should they have all gotten 100%? I think that they should have gotten an undefined 0/0 if that were the maximum grade.

#### Ana Bee

One other thing to take under consideration the "dude, don't be an asshole" cause, which is sufficient to persuade a person not to break contract, when uttered near the classroom door.

#### Lou

A more sophisticated curve would raise the median or mean to a desired value (say 75 or 80%). Thus everyone would get the same lower grade and remove the incentive for the top students to participate.

#### Peter

Lou, I was going to do the same thing. If the curve were based on the median score there should be, by definition, 50% of the class than can expect to receive a better grade. This would restore the separating equilibrium, by ensuring one student has an incentive to defect.

#### Khurram Makhdumi

If nobody entered the exam hall then they don't get zeroes, they were absent. To get zero, one must enter the exam hall and at least write name on the answer sheet. Given every student has an individual answer sheet, observing the divergent attitude directly is harder, thus the incentive to diverge and the more persistent equilibrium, i.e. everybody takes an exam.

The students took a significant risk, because the instructor had another option: to refuse to give them grades and instead assign them "incompletes".

#### Tom Fox

It seems to me like they all should have received an incomplete rather than a zero. Did he take attendance?

If this was my class trying to "game" my system I would have gamed them right back and said that there is a difference between not taking and not passing the exam. That would force them to all fill in "A" or something like that. Then we would see how much they trust each other.

#### Nate Vack

In my high school US History class, we actually did this same trick on our last weekly test. We all took the test, but had everyone score 0% by choosing "E" on an "A-D" multiple choice test. Though we had no way to detect defectors, everyone in our 25-person class cooperated. It was pretty great.

However: this was only one of many, many tests and we fully expected to be caught and made to re-take the test (which is exactly what happened), so the stakes were significantly lower for us.

#### Joe

A more interesting scenario would have been if the test was take home. Students would not be able to know if others were, in fact, staying true to their agreement. By standing outside of the classroom united, this uncertainty goes away.

#### Dave

That was their grade on the final but was it their final grade? Assuming the Professor issues final grades on the same type of curve based on other course work, midterms, etc. By getting identical grades on the final exam, all the students accomplished was to remove the final exam from the equation.

#### Andrew Kelly

Hello Freakonomics, it's the student from Rampell's article.

I wanted to clear up a few things about what really went down before things get out of hand in the comments section, like they have on many other websites as the story itself gets further from the truth.

I was in the Introduction to Programming course, which had students from seniors to freshmen, across many engineering and science majors. I'm a MechE senior who wanted to have a little programming under my belt before beginning my Masters program at Hopkins, and Python seemed like an interesting language.

Although an introductory class, the assignments were challenging and required a good chunk of time weekly. There was a midterm with a pretty mediocre average. It was only until the class of the year that we found about this 'boycott' potential. Fröhlich was holding a Q&A, and a classmate asked if it was true that the Intermediate Programming course did not take their midterm. Fröhlich explained that if no one took the exam, we would all receive A's 'based on a lousy Python script' he uses for his grading. So instead of some scheming, lazy students who were bent on gaming the system and the curve, we were merely handed an interesting to a 40 minute exam. Also, since there wasn't certainty that absolutely no student was going to walk in those doors meant we all studied for the exam. Did I study as efficiently as I would have without this looming over my head? No, but it did open up more time to study for my other exams and the lengths I went to organize the 'boycott' broke up the monotony of my 7th exam period at Hopkins. We used a public Google Drive spreadsheet to actively make sure everyone was on board, we reached out to those that weren't posting on our course website to see if they were still enrolled in the course and knew the terms of the final. Other classes were not as fortunate, but we were able to get 100% committal to the boycott from the students we knew were still in the class.

So I never really thought about it as a "curve" until these articles started putting it in that light. The final itself was only 10-15% of the final grade, so really your performance in those 10 assignments, midterm, group project, and participation defined your course grade.

As a senior in engineering at Hopkins, I've seem some crazy competitive behavior (but not like stereotypical pre-med behavior). It was extremely refreshing to hang out with a wide cross section of students outside the exam room the morning of that exam, having donuts and enjoying the whole situation. We pulled off someone I didn't think was possible, but it put more happy faces on Johns Hopkins students than I've ever seen as the result of a final exam.

tl;dr
We didn't set out to 'game the system'. Our class was told by Fröhlich on the last lecture of the semester that if no one took the exam, we would get an A. If one person took it, game on. We made sure everyone was on board, studied anyways, went outside the lecture hall on the day of the exam, had donuts, went home.

#### PricklyPete

This would be much more interesting if the students did this without waiting outside the classroom ready to go inside in case of any defections.

#### SPENCER

Except they should have all gotten an incomplete for not taking the test. Hard to give a grade on a test never taken. More plausible, im assuming, would be if everyone wrote their name on the test and turned it in for a zero. Although zero, especially in a computer programming where zero divided by zero to yield a percentage should result in a logic error... hmmm

#### Aries

The common game theory example of prisoners dilemma, usually involve a situation where the participants do not know the other person action/testimony. In the case here, the participants have an almost perfect information of all the participants and a whole semester to work out and discuss the system. They also seem tohave a. relatively high degree of confidence in the teacher. predicted behavior. There seems to be little impediment to this plan and obviously they seem to have all be prepared for plan B. If anyone defect. Seems to me they are playing the game of mutually assured destruction deterrence to the fullest.

#### Kreyg

We did this once for the first year exams in grad school (Econ PhD). After studying the entire summer, a large group of us were in the office of the Macro professor who asked us if we looked at any of the models that dealt with money from a previous professor. We said of course not, that was not what we learned in class.

Clearly, he was going to ask about one of these models and none of us had ever seen one. The exams are curved so we all agreed to not look at them so that no one had to freak out and try to learn an entire set of models within a week.

I'll never forget how much the one girl bragged about getting the scholarship for achieving the best score on the macro exam after she stabbed us all in the back and looked up the money models. I could say I don't get satisfaction when I see that she took seven years to graduate and did not obtain a permanent academic position...but I'd be lying. Schadenfreude for the win.