A Better Way to Rank Colleges?

Amidst another scandal surrounding U.S. News and World Report’s college rankings, economists Christopher N. Avery, Mark E. Glickman, Caroline M. Hoxby, and Andrew Metrick have proposed another option: rankings based on students’ revealed preferences. Here’s the abstract:

We present a method of ranking U.S. undergraduate programs based on students’ revealed preferences. When a student chooses a college among those that have admitted him, that college “wins” his “tournament.” Our method efficiently integrates the information from thousands of such tournaments. We implement the method using data from a national sample of high-achieving students. We demonstrate that this ranking method has strong theoretical properties, eliminating incentives for colleges to adopt strategic, inefficient admissions policies to improve their rankings. We also show empirically that our ranking is (1) not vulnerable to strategic manipulation; (2) similar regardless of whether we control for variables, such as net cost, that vary among a college’s admits; (3) similar regardless of whether we account for students selecting where to apply, including Early Decision. We exemplify multiple rankings for different types of students who have preferences that vary systematically.

Eric Hoover of the The Chronicle of Higher Education comments on another interesting finding in the paper: 

The authors also challenge the assumption that an admission rate is an indicator of desirability. Half of the top 20 colleges in the revealed-preferences list, they found, would fall outside the top 20 if one ranked them only according to their admission rates (the lower the rate, the better, conventional wisdom holds). Notre Dame, for instance, placed 13th on the desirability list, but its admission rate was only the 58th lowest. The University of Virginia placed 20th on the desirability list, but it had only the 76th lowest admission rate.

(HT: Seth Matthew Fishman)


Right... this is like trying to hang our hats on the Coaches Poll for NCAA basketball rankings... check out their pre-season rankings from this year... now how robust is that?

Multivariate models built off of empirical data will always trump crowd-sourced, aggregated guess-timations.

Gin Song

Wouldn't this suffer the same under-coverage as the official employment rate?

I would wager with many fine folk that a significant amount of prospective students opt not to apply to schools for the following reasons: chance of acceptance, chance of financial aid/scholarship, distance from home. (The latter would be an interesting study to see what the ideal distance one's home should be from the university of their choice. [~200 miles would be my guess])

The key issue is the assumption that the available universities to accept from includes the most desirable school for the student.

Mike Jones

full paper


That seems like poor logic to me: I grew up near a state school with a good football team and a lively night scene. Everybody wanted to go there, but not because it was a good school, simply because it was a good time.

Does a rampant drinking and football culture make a school or just ?

Enter your name...

They restricted their study to high-achieving students, who are more likely to care about the potential for academic achievement than the football team.


Yes, because 17-18 year old boys and girls who get good grades hate to have a good time, or even really know anything about college other than what they hear about from random people they meet.

Seminymous Coward

This would produce a weird feedback loop where students choose to go to the places ranked higher because those are the places students last year chose to go.

Enter your name...

That's no different from any other good in the market. People buy clothes, houses, books, and food by considering what's popular with people who are believed to be knowledgeable about these things.


With one minor exception... the "people who are supposedly knowledgeable" in our case is "a cohort of high-achieving high school students who have never been to college in the first place!"

John B. Chilton

Link to Chronicle article:



Isn't this just begging the question? Won't many 'high achieving' students preference for a college be based on the exact rankings that these ones are trying to replace?

tung bo

I think it's pretty clear that this study is showing the 'desirability' of the colleges and NOT the 'quality' of the colleges. Thus, external opinions such as magazine rankings would affect people's perception and the desirability.

I think this part is much more important: " We exemplify multiple rankings for different types of students who have preferences that vary systematically."

Trying to have 1 single list of ranking is ridiculous given the variation in college applicants.
There should be separate lists for: "math/science", "health care", "liberal arts", "sports", "business", "politics", etc.


This is just as stupid as ranking based on admissions denied. Who is ranking on the quality of education? The reality is, outside of some dedicated head hunters, the real ranking that matters is your school's name recognition. These rankings are apparently so bogus that all anyone in the real world is concerned with is that the school actually exists.

Basing things on applicant preference is kind of stupid really. This assumes people who apply (17 year olds) know going in which is the best school ( "best" is highly subjective anyway) and use the complete information to pick the best school. Look at the real problem. The rankings the students have been subjected to this far are being calculated wrongly, so how are we to believe that a mass of high school students will, en masse, be able to weed through the losers and go after that school that offers the best educational experience?

Why not see how many schools will submit to some kind of test on incoming students, and then test outgoing students? You could use some kind of algorithm to factor in drop out/transfer rate/night lie/football team and roll from there. I know I got a 98% on my MFAF leaving my college, I'd be interested to see where I was when I went in. I could tell you if the money was worth it then.



And I apologize for not proof reading that. Auto correct went crazy.