The Art of SATergy

My son took the SSAT exam this past Saturday. And while I was sitting in the Choate athletic facility waiting for him to finish, I remembered that Avinash Dixit and Barry Nalebuff‘s new book, The Art of Strategy, has a great example concerning standardized testing. Game theory is so powerful it can help you figure out the correct answer without even knowing what the question is.

Consider the following question for the GMAT (the test given to MBA applicants). Unfortunately, issues of copyright clearance have prevented us from reproducing the question, but that shouldn’t stop us.

Which of the following is the correct answer?

a) 4p sq. inches

b) 8p sq. inches

c) 16 sq. inches

d) 16p sq. inches

e) 32p sq. inches

O.K., we recognize that you’re at a bit of a disadvantage not having the question. Still, we think that by putting on your game-theory hat you can still figure it out.

Before reading their analysis, take a shot at trying to reason your way to the correct answer.

Here’s what they said:

The odd answer in the series is c. Since it is so different from the other answers, it is probably not right. The fact that the units are in square inches suggests an answer that has a perfect square in it, such as 4p or 16p.

This is a fine start and demonstrates good test-taking skills, but we haven’t really started to use game theory. Think of the game being played by the person writing the question. What is that person’s objective?

He or she wants people who understand the problem to get the answer right and those who don’t to get it wrong. Thus wrong answers have to be chosen carefully so as to be appealing to folks who don’t quite know the answer. For example, in response to the question: “How many feet are in a mile?” an answer of “Giraffe,” or even 16p, is unlikely to attract any takers.

Turning this around, imagine that 16 square inches really is the right answer. What kind of question might have 16 square inches as the answer but would lead someone to think 32p is right? Not many. People don’t often go around adding p to answers for the fun of it. “Did you see my new car — it gets 10p miles to the gallon.” We think not. Hence we can truly rule out 16 as being the correct solution.

Let’s now turn to the two perfect squares, 4p and 16p. Assume for a moment that 16p square inches is the correct solution. The problem might have been: “What is the area of a circle with a radius of 4?” The correct formula for the area of a circle is pr2. However, the person who didn’t quite remember the formula might have mixed it up with the formula for the circumference of a circle, 2pr. (Yes, we know that the circumference is in inches, not square inches, but the person making this mistake would be unlikely to recognize this issue.)

Note that if r = 4, then 2pr is 8p, and that would lead the person to the wrong answer of b. The person could also mix and match and use the formula 2pr2, and hence believe that 32p or e was the right answer. The person could leave off the p and come up with 16 or c, or the person could forget to square the radius and simply use pr as the area, leading to 4p or a. In summary, if 16p is the correct answer, then we can tell a plausible story about how each of the other answers might be chosen. They are all good wrong answers for the test maker.

What if 4p is the correct solution (so that r = 2)? Think now about the most common mistake: mixing up circumference with area. If the student used the wrong formula, 2pr, he or she would still get 4p, albeit with incorrect units. There is nothing worse, from a test maker’s perspective, than allowing the person to get the right answer for the wrong reason. Hence 4p would be a terrible right answer, as it would allow too many people who didn’t know what they were doing to get full credit.

At this point, we are done. We are confident that the right answer is 16p. And we are right. By thinking about the objective of the person writing the test, we can suss out the right answer, often without even seeing the question.

Now, we don’t recommend that you go about taking the GMAT and other tests without bothering to even look at the questions. We appreciate that if you are smart enough to go through this logic, you most likely know the formula for the area of a circle. But you never know. There will be cases where you don’t know the meaning of one of the answers or the material for the question wasn’t covered in your course. In those cases, thinking about the testing game may lead you to the right answer.

If you want a fun way to learn a ton of useful game theory, this is the book for you. How good is it? Steve Levitt has a blurb on the book saying it’s so good, he read it twice.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.



View All Comments »
  1. Rachael says:

    Yep, that’s the answer I got.

    A simpler version of the reasoning, which isn’t as game-theoretically watertight as that given here but which used to serve me well at school, is to pick the answer which has most in common with the other answers, so all the other answers deviate from it in minimal ways. So, in this example, all the answers have a 16 or a pi in them, so the correct answer is the one with both a 16 and a pi.

    Similarly with spelling questions: the choice of answers will often be something like “necessary, necesary, neccessary” and even if you don’t know the correct answer, you’d pick the first, because it’s equal in edit distance from the other two. Whereas if the last one were right, the other two would be suboptimal decoy answers, since one would be some distance from the correct answer and the other would be even further from it in the same direction.

    Thumb up 1 Thumb down 0
  2. Adam Bee says:

    It didn’t seem fair to me that they allowed themselves to use their correct knowledge of the formula when figuring out the answer. They also shouldn’t have spent so much time reverse engineering what the question is.

    In any realistic situation, the test-taker will know what the question is and not know what the answer is.

    The only part of this that I found useful was the ruling out of c. because it is so obviously much different. The rest is just BS filler.

    Thumb up 0 Thumb down 0
  3. Kevin says:

    Unfortunately this would have no use on the exam since you used the formulas that would probably be needed to answer the original question to root out the potentially wrong ones…Besides the fact that using such logic would take far too long.

    Thumb up 0 Thumb down 0
  4. Anna says:

    You didn’t even need to go through all that analysis. It’s just a matter of picking the most common of the different elements. Most of the choices have Pi. Rule out C. The only number that is a choice more than once is 16. 16pi.

    You don’t even have to know the first thing about circle formulas, and I would bet that most people reading this and guessing the right answer went about it my way.

    Thumb up 0 Thumb down 0
  5. chappy says:

    Well I arrived at the correct answer, but I think my logic was not as involved. I agreed with the logic that 16 sq. inches seemed to odd, but then I thought why would they leave it in…because 16(pi) square inches is the right answer and they are trying to ‘catch’ people that aren’t looking at the units.

    Thumb up 0 Thumb down 0
  6. jz says:

    so, I’m not as smart as I’d prefer to think.

    Thumb up 0 Thumb down 0
  7. adriano says:

    Is either 4pi or 16pi depending on the radius, 2 or 4 respectively

    adriano, Italy

    Thumb up 0 Thumb down 0
  8. Matt C says:

    It’s even easier than that. One thing the authors missed: the only answer without a pi is 16. Since each wrong answer usually represents the inclusion of a single common mistake, and we’re pretty sure the answer has pi after it, 16pi makes the most sense. If the answer were 16pi and the only number without a pi were 8, that number would only appeal to people who made TWO mistakes (using the wrong formula AND dropping the pi). Unless they’re saying the pi-less answer was a throwaway, ignoring the fact that the choices include both 16 and 16pi is ignoring a very relevant and helpful piece of information.

    Thumb up 1 Thumb down 0