What Should Be Done About Standardized Tests? A Freakonomics Quorum

What should be done about the quality and quantity of standardized testing in U.S. schools? We touched on the subject in Freakonomics, but only insofar as the introduction of high-stakes testing altered the incentives at play — including the incentives for some teachers, who were found to cheat in order to cover up the poor performance of their students (which, obviously, also indicates the poor performance of the teachers).

Personally, I used to love taking standardized tests. To me, they represented the big ballgame that you spent all season preparing for, practicing for; they were easily my strongest incentive for paying attention during the school year. I realize, however, that this may not be a common view. Tests have increasingly come to be seen as a ritualized burden that encourages rote learning at the expense of good thinking.

So what should be done? We gathered a group of testing afficionados — W. James Popham, Robert Zemsky, Thomas Toch, Monty Neill, and Gaston Caperton — and put to them the following questions:

Should there be less standardized testing in the current school system, or more? Should all schools, including colleges, institute exit exams?

Here are their responses. Many thanks to all of them for their participation. I have to admit, I never saw the parallel between tests and French fries before, but now that I’ve seen it, I won’t soon forget it.

W. James Popham, author of The Truth About Testing: An Educator’s Call to Action and America’s Failing Schools:

Standardized tests have much in common with French fries. Both of them differ in composition as well as quality. French fries are available in numerous incarnations, including straight, curly, skins-on, skins-off, and, in recent years, with sweet potatoes. Regarding quality, of course, the taste of French fries can range substantially – from sublime to soggy. It’s really the same with standardized tests.

Certain standardized tests (called achievement tests) are intended to show us what skills or knowledge students have mastered. Other standardized tests (called aptitude tests) are designed to predict how well test-takers will perform in future settings, such as when they get to college. Some standardized tests are designed to differentiate among test-takers so we can say that Kevin scored at the 82nd percentile, while Melanie’s performance puts her at the 96th percentile. Some standardized tests are supposed to let us know how well a particular group of students, such as those in a given school, have been taught. But, just as is true with French fries, standardized tests can vary dramatically in their quality. Some standardized tests perform their measurement mission marvelously; others do a dismal job of it.

Thus, if we’re asked whether there should be more or fewer standardized tests in our school system, the only defensible answer is, “It depends.” It depends on whether the right kinds of tests are being used and whether those tests are good ones. Given the kinds and caliber of the standardized tests currently being used in our schools, I come down on the “less” side of the argument. But that’s chiefly because the wrong sorts of standardized tests are frequently being used. Take the No Child Left Behind Act, for instance, a federal accountability law requiring scads of standardized tests to be used in evaluating schools. Do you know that almost all of the standardized tests now being employed to judge school quality are unable to distinguish between well taught and badly taught students?

We surely don’t need more of those sorts of misleading tests. But we definitely do need more standardized tests that are sufficiently sensitive to instructional quality, so we can accurately tell which schools are truly successful and which ones aren’t. Standardized tests can be written that accurately measure a school’s instructional effectiveness, yet also stimulate teachers to do a better job of teaching.

Turning to the exit-exam question, all schools – kindergarten through college – should employ exit exams allowing us to determine what students have actually learned. We owe it to our students to make sure that they’ve been properly taught. But when I hear, as I recently have, of a proposal for colleges to start using end-of-course tests as exit exams, I become altogether apprehensive. I was a college professor for more than 30 years, and I assure you that most professors know no more about making exit exams than they do about making French fries.

Robert Zemsky, professor and chair of the Learning Alliance at the University of Pennsylvania, and former member of the Spellings Commission:

Discussing testing is roughly akin to planning a visit to the dentist – it’s all about remembered pain. No one really likes to be tested. And yet high-stakes testing — already a key element in the reform of primary and secondary education – has become a standard feature of the “let’s reform higher education” industry.

Testing raises a host of problematic questions. Who is being tested: the student, or the teacher? What is being tested: what the student knows, or what the student has learned? Should the tests focus on specific knowledge – like the ability to read a complex text or solve a standard physics problem – or should the test focus on more general attributes, like creative thinking and problem solving? Can a test in which the test-taker – that is, the student – does not have a direct stake in the outcome actually command the test-taker to do his or her very best?

Then there are the questions of what to do with the results. I have actually sat through an extended discussion of how we could use regression analysis to parse out the contribution different teachers made to a group of students’ performance on a set of standardized tests. The answer was, yes it was possible, and could in fact be used to award merit pay increases. But nobody left the room feeling very comfortable that there would be any gain in what we knew made for good teaching.

What we know – and what makes those of us in higher education particularly leery of generalized tests designed to capture how well an institution teaches attributes like creating thinking and problem solving – is that the best predictor of how well a group of college students will do on such a test is how well they did on the SAT or the ACT. Those instruments may not be perfect, or even good at identifying scholastic aptitudes, but boy are they good at telling us who the best test-takers are.

Thomas Toch, co-director of Education Sector, a Washington, D.C., think tank:

There’s a lot of standardized testing in public education. Elementary, middle, and high school students are taking some 56 million reading, math, and science tests this year just to comply with the demands of the No Child Left Behind Act, and many states and school systems layer a lot of other standardized tests on top of that.

This testing is valuable. Without it, parents, taxpayers, and policymakers would have a tough time knowing how well schools were performing. That was the case prior to the advent of the “standards movement” in public education in the 1990s, when states began setting standards, testing students, and publicizing the results. Students could fall through the cracks, and many did, but educators didn’t have strong incentives to help them because without tests that measured students’ performance against clear standards, there was no way of holding teachers and principals accountable for their students’ success.

NCLB took the standards movement to its logical next step, requiring standards and testing systems in every state, and creating consequences for schools that failed to make adequate progress with specific groups of students that public schools hadn’t educated very well in the past – the poor, students of color, English language learners, and the disabled. That has been the law’s most valuable contribution.

But we need much better tests. For a variety of reasons, including the need to produce vast numbers of tests quickly and cheaply, the majority of today’s state-level standardized tests are multiple-choice measures of mostly low-level skills, such as the recalling of facts in a reading passage. They largely sidestep higher-level skills, such as having students compare and contrast two reading passages, and the open-ended questions that are best suited to measuring such skills. Roughly half of the nation’s students are taking tests under NCLB that are completely free of open-ended questions.

This presents a problem, because when tests are high-stakes events, as they are under NCLB (teachers and principals can eventually lose their jobs if their students flunk NCLB tests for several consecutive years), educators have a strong incentive to “teach to the test.” In this case, that means teaching low level skills at the expense of the more demanding material that everyone says students need to master in today’s complicated world.

Exit exams, which students must pass to graduate, make sense. “Social promotion,” or advancing unprepared students, has been commonplace in schools and colleges for a long time.

But such tests pose tough questions. Two-thirds of the nation’s public high school students currently must pass exit exams in reading and math in order to graduate. But the majority of the tests measure ninth- or tenth-grade-level basic skills; passing them doesn’t mean students are ready for the workplace, much less prepared for college. Yet many state lawmakers have been wary of setting the bar higher for fear of large numbers of students failing.

But is it fair to give students what amounts to a counterfeit passport to college or work? And do such tests spur high school teachers and principals to aim high with their students? To both questions, the answer is, “No.” In most states today, high school exit tests serve the same role as the standardized tests mandated by NCLB: they try to jack up the floor of student achievement in the nation’s schools. The best high school exit tests would be end-of-course exams akin to the “comprehensive” exams that many colleges and universities require students to pass in their majors before graduation – tests, that is, that would raise the ceiling of student achievement.

Monty Neill, executive director of FairTest:

The No Child Left Behind law has had one clear accomplishment: it has given a black eye to education policies based on the overuse of standardized testing.

NCLB’s testing mandates have flooded American classrooms with millions of additional tests. At the same time, the rate of learning improvement has actually slowed, according to the National Assessment of Educational Progress (NAEP).

A mounting pile of surveys and reports document the negative consequences of testing overuse and abuse, as well as growing public opposition to the test-and-punish approach. For more evidence, just listen to the roars of approval when any of the presidential candidates criticizes the law. No wonder more than 140 national education, civil rights, religious, disability, parenting, and civic groups have called for its comprehensive overhaul.

Having long tracked the misuse and abuse of such tests, FairTest predicted a range of negative consequences from NCLB. Most have now been documented by independent researchers. The problems are compounded by high school graduation tests, and by pressure to score high on college admissions exams.

High-stakes testing has narrowed and dumbed down curricula; eliminated time spent on untested subjects like social studies, art, and even recess; turned classrooms into little more than test preparation centers; reduced high school graduation rates; and driven good teachers from the profession. Those are all reasons why FairTest and other experts advocate a sharp reduction in public school standardized testing and a halt to exit exams.

One-size-fits-all testing schemes make even less sense for colleges and universities. How could one exam ever accurately assess the learning of students majoring in subjects as diverse as art history, biomedical engineering, and political science?

As such, the politicians blindly mandating such exams are the ones outside the mainstream, not assessment reformers like us. Indeed, the testing industry’s own standards state that no single exam should be used as the sole or primary criterion to make high-stakes educational decisions such as promotion, retention, graduation, college admission, or scholarship awards.

There are better ways to assess student learning. Classroom-based information, such as grades, provides richer evidence of performance. High school grade point average is a better predictor of college success than either the SAT or the ACT.

The nation does need better assessments and more training for educators to get the most out of them. FairTest has long promoted high-quality, classroom-based assessments that can be used to improve student learning and teaching. We also support the more than 760 colleges that do not require admissions test scores for many or all of their applicants.

High quality assessment is an educational necessity. But high-stakes standardized tests harm educational quality and promote inequity.

Gaston Caperton, president of The College Board:

The quantity of testing is less important than the quality of testing. This is where the SAT excels. In an era of rampant grade inflation, the SAT offers students the most level playing field available to demonstrate their knowledge of core material. The SAT, in combination with the grade point average, provides students, parents and admissions counselors with the best predictor of academic success in college.

As for the question about exit exams, I think they largely exist already in the form of final exams in various subject areas, both in high school and in college. The SAT is unique in that it provides a focused look back at a student’s accomplishments in high school while offering a glimpse into that student’s potential in a college environment.


As an education researcher, I love standardized tests for their usefulness as a heuretic for comparing school or classroom outcomes. My job would be much harder without them.

But, as a former student (and now as a parent) I loathed them. Not because I couldn't do well on them, but because I hated wasting my time. They were pointless in terms of their usefulness in the classroom and did nothing to advance my learning of the subject matter. In fact, they may have harmed my learning.

There is a certain math curriculum now in vogue for which my high school graduation class was the first to utilize for algebra, trig, and calculus. I excelled in those classes, and I got a 5 on the AP calculus exam. These grades, plus my high SAT scores, got me into MIT. But once there, I realized I had no conceptual knowledge of math and had no reasoning skills. I could memorize and regurgitate, but I struggled mightily with deduction and proofs. Nothing was intuitive for me. Organic chemistry, which in those days required no computation, was my saving grace.

I have chosen my kids' schools based on their avoidance of this math curriculum, as well as their avoidance of a similar reading curriculum.



I have two thoughts:

One is that most people don't know what it means for a test to be standardized. Standardization simply means that everyone, regardless of race or economic status, takes the same test, at the same time, in the same way, under the same conditions. A rational, non-racist person cannot possibly be opposed to the concept of standardization.

The other is that some schools are inappropriately transmitting their anxiety about AYP problems under NCLB to students. I recently dealt with an elementary school student who was totally freaked out about her school's tests. She thought that if she didn't do well enough on the test, that federal law would automatically force her to flunk fifth grade. If you've got kids in elementary or middle school, please explain to them that nothing bad will ever happen to them as a result of these tests.

Einar Tangen

Much of the discourse seems to assume that tests are necessary to measure the effectiveness of our educational infrastructure at producing the human machinery of our society. What a sad but apt commentary on our society's approach to education. Unfortunately like a lot of outmoded post war production models which emphasized quantity over quality, these tests serve only to measure the systems failures rather than preventing them.

It is unfortunate that we have been unable to adjust our thinking about educational models as we have business models. The irony is that while we seem to acknowledge the information age requires creative processing of complex knowledge; our education system emphasizes memorized regurgitation. It is even stranger that these efforts persist even as the numbers of American born children abandon the building blocks of creativity, the hard sciences and humanities. Business has recognized that creating failures is a waste of resources, that it is better improve the process rather than try to weed out the rejects after they have been produced, but this is not widely applied to our education system, even in the face of consistent policy failures.

While education is not a simple issue, valuing it is. Do we as a society really value education? While a naturally gifted mind can discover the value of education, for most of us it is imparted through our environment, beginning at home. It is interesting that many of those who claim that they value education are the first to cite faith and tests as replacements for reason and process. With such unapologetic illiterates standing at the helm of our society is it a wonder that the environment necessary for true education is lacking.



As a high school student, I feel that all the standardized testing is good for the administrators, but not good for the students or teachers. For the teachers, there is no incentive not to "teach by the test", since their "performance" is judged by these tests. That in and of itself is folly to me, since the best teacher in the world can't successfully teach a student who has no desire to learn. For the students, there is no incentive to go above and beyond the test, since the test is all that counts. The SAT is, to me, a silly test. An 8th grader could successfully take the SAT in my opinion. The SAT, to those who have taken it year after year in high school like I have, and sat through the 3 and a half grueling hours, know that the SAT doesn't test what you know, it tests how you know how to take the SAT. The questions are deliberately designed not to test your math skills or reading skills, but how well you can read through the trickery that is woven into the problems. The new essay section is a farce, too. Like any other test, you have to know what to write, in what format, which buzzwords you have to use, etc. The A.P. tests in this regard are superior, since they test your knowledge, not if you read the "Secrets of the SAT" or not.


Susan Meier

Mr. Popham has it right. It's critically important to get tests that respond to instruction. It's not realistic to get rid of tests. You are harkening back tio a day that never exists - and I've lived in schools for 45 yeasr. I was a principal back in the days of no testing, and it's a mistake. No accountability is a mistake; it leads to too much variation and the losers are the most vulnerable kids.

I like AP and IB tests. I like the New York State Regents Examinations. AP and IB courses and Regents are NOT flat, empty courses aimed at low level thinking; and they are absolutely aimed at tests.
We need thinking tests. We know how to do this. It's important and it should be a national demand. Or we can just ignore it, or fight it with some mythology about free and wonderful academicians - and the world will rely on the tests it has, as it is right now: the AP, the IB, the SAT and the ACT.



Testing is a necessity, and should be utilized to help plan the next step in a student's education. In my 20 years in an elementary classroom setting, however, I have seen the quality of instruction decline tremendously, due in part to the high-stakes testing frenzy.

Children spend countless hours learning strategies for passing tests. They are demeaned and/or punished for failing to use these strategies on every worksheet even if they have completed the work correctly. At my current campus, students in grades 3-5 take at least one day-long "benchmark" each week beginning in October. This continues through March or April, depending upon the grade level and when the actual TAKS test is given. Beginning this year, the principal instituted practice TAKS tests in reading and writing for first and second grade as well.

Administrators are surprised that by fifth grade students exhibit behavior problems and refuse to use the "strategies" they've been taught. They are equally surprised that students who struggled in third grade have not been helped more by the constant testing.

Granted, this is the extreme to which high-stakes testing can go. However, this extreme is occurring in more than one Texas school district. Unfortunately, for students in these districts, they're not getting an education in anything save how to take the current state-mandated test.



The big question about standardized tests is "what do they really measure"?

I personally test well, I know other smart people who also test well, but I also know other smart and educated people who don't.

A question that needs to be answered about giving a test is "Why are you giving it?" If you are measuring schools and teachers, instead of making students spend days testing, why not give an hour test, and give different people different tests, and get a statistical model?


Actually, John, when I graduated from college last year all of my peers applying to the finance/consulting jobs had to put their SAT scores on their resumes. If your score was below 1350 then you could forget about working at Goldman.

I don't love the SATs, but a good score can help you get a job.


Does Mr. Caperton honestly think that knowing the relationship between obscure English terminology is core material for High School? Or being able to judge whether Sue, Sam, and Sarah are older/younger/or just plain dead?

Interesting that the SAT doesn't test whether a person can read a train schedule or how to balance a checkbook. I guess that's why so many Americans are consistently late and neck-deep in debt.


I've been a classroom teacher for 30 years. The biggest change that I can see that NCLB has brought into the schools is a dramatic increase in the levels of tension experienced by both teachers and administrators. As a consequence, there has been a decrease in the empathy staff feel for students that come from troubled backgrounds, the very children who will be left behind.

J. Greene

The suggestion that the students be tested to find out what they actually *learned* is, I think, far more useful to everyone. If a child goes into a class reading poorly, and the final test shows that while he didn't necessarily master the required material, his reading skills improved drastically, that tells us the teacher did an excellent job. Those skills will serve the student for the rest of his life and help him catch up on the things he's missed, even if takes him longer.

One of the things that bugs me about the way we're doing it now is that we seem to forget that things can also be learned later! In fact, ideally, one will continue to learn for a lifetime. I wonder how many lifetime learners all this pressure to perform is going to produce?

Gunther ST

Standardized test have considerable value. I thought that the regents examinations I had to take at the of a course (math, language,civics) in New York in the 1940s were good. You either knew your stuff or you did not and took the course again.
But now, no one is kept back and promotion is assured/required so as not to inhibit a child's mentality. The result is a consistent lowering of standard, a poorly educated work force, and a lack of skilled workers required by industry. - It is a sure way of pacing the downfall of the nation.


Are standardized test used to provide specific feedback to students? In the market system price is more than an abstract number, it conveys information to the seller of goods and services as well as the buyer.
With testing the buyers, colleges, can determine how to best distribute their scarce resources with test scores in the same way that buyers can use price in deciding what to buy. Sellers use marketing studies to determine how to position themselves in the market, what products support higher prices. Beyond the gross score that the student gets do any tests provide specific guidance or feedback to the student or her teacher? Do they pinpoint the weaknesses the student could focus on? Are they used to help the student identify her strengths?


I am a high school English teacher in an urban high school. The majority of my students are African-American. When students enter my classroom in the fall, I pre-test them on an ACT, the test the state uses for NCLB. I regularly have students who score 12s and 13s and even 2s and 3s. By the time the March testing roles around, my class averages have moved from around a 10 to a 16. They have improved, but to an average dismal score, which makes me look bad. I do not get credit for improving a student from a 2 and a 3 to a 12 or 13, but only get the shame of having students scoring well below the national average. And what resources do I have for my students reading at a 2nd grade level in 11th grade? None but what I develop through earnest research and diligent tutoring. Not all teachers do this, and even the ones who teach to the test end up with the same numbers. I find it interesting that none of the Quorum members mentioned the persistent achievement gap nor the the fact that no one has adequately determined why it exists or how to effectively eradicate it.


Stanton J

I have no problem with standardized tests and tell my son that they are just part of his education. But what I really resent is the incredible amount of time that schools spend prepping students for these tests instead of actually teaching and engaging them. This is the very poor result of accountability in the field of education.

Giving regular standardized tests to kids would be fine as long as the school simply makes them part of the curriculum, instead of what goes on in my son's school where they devote unacceptable amounts of time talking about how to respond to test questions or teaching kids how test questions can trick you. If schools are doing their job, they have no need to be concerned about performance on tests. They would have covered everything that is probed on the standardized instrument. What is clear about schools that teach test prep is that the favor form over substance, never a good thing in education.

My son's school has even assigned tests as homework!!! This is outrageous. When I went to the school's principal with my concerns, her response was: it's only one month of the school year. One month of learning is a precious thing to a growing child.

Politicians ought to be held to account for whipping up such test-crazy sentiments in schools.



Justin @3 got close to what I was thinking. What, exactly, are we trying to measure? Clearly, the standardized tests measure rote memorization. So how do you measure a student's capacity for reasoning, which is, I believe, far more important? SAT? ACT? Speaking from my experience in the workplace, people who can think critically are rare, even those with college degrees. It may be that the only true way to measure how well a teacher or a school actually prepares a student for the working world is by following up on those students at certain intervals after they graduate and assess their progress in the workplace.

Secondly, several posters here have noted the need for "better" teachers. Nobody, however, has defined what constitutes "better". IMHO, the best teachers are the ones that have a mastery of and passion for their subject. I don't think that a teacher with a degree in education can teach mathematics better than a teacher with a degree in mathematics can teach mathematics.

Finally, I wish that Economics was a required course in all high schools all across this country as well as a course in life skills, like balancing a checkbook and managing your money.

And thanks, Mr. Caperton, for the ad. That was most enlightening.



It seems that we should be able to work out a regression analysis that determines the portion of a score attributed to home life, school life, teacher experience, teacher quality, and student effort. When I look at broad historical data of the students in my class room I clearly see trends for those with college educated parents, stable home lives and middle incomes vs. those with relatively under-educated parents, disruptive home lives and near poverty incomes.

Lewis Orans

In high school in the early 1960s, New York State mandated standard testing for all students in major subjects. These "Regents Exams" were administered after the regular course final examination (during "Regent Week" each semester). I believe they leveled the playing field to determine the quality of learning at all high schools in the state and gave a fair representation of the quality of education and achievement in each school. These scores became part of transcripts and so entered the college application scene.
I attended a tough high school (Roslyn) which still ranks very high in national ratings (and in college placement in leading institutions). I know that my Regents grades generally outpaced my course grades. I attribute this to the challenging education I received ... and education that prepared me for the real challenge of an Ivy League College where, as a public high school graduate, I had to compete with graduates of many top tier private high schools.

This type of testing is certainly worthwhile today. However, some of the teaching done in public schools (I have seen this in both Texas and California) seems to be "teaching to the test" as opposed to teaching to promote real learning. This is a cautionary tale, and we must be careful to keep the focus on learning and not just the scores.


Mr O

I feel your pain. I teach Earth Science at an alternative inner-city middle school for at-risk students. They have all been kicked out of their homeschools (for drugs, weapons, fighting, failing grades, etc). I have only 5 weeks to prepare them for the MEAP (Michigan's standardized test), but I manage to track their progress using the previous two year's exams. Only about 2% of my students can pass the MEAP the first day of class, but for two years straight I have helped boost that number to 30% or better by test time. My method? I blitz them with daily experiments. It's all hands-on learning everyday. And while I have impressed my principal, my fellow teachers, and even my district's central administration, being able to prove that my teaching is having an impact and raising the scores of students whom most people (let alone teachers) would not come within five miles of, as far as NCLB is concerned, my program is an abysmal failure. I've actually considered suing my state depo. of ed. because they keep putting out reportcards that say my school's science program deserves D's and E's.

What a crock! No?



“which, obviously, also indicates the poor performance of the teachers” — no, not obviously. Some teachers aren't ever going to get good test performance out of their classes for reasons that are totally out of a teacher's hands.

— Posted by KC in Lubbock

I completely disagree with this statement, and have to admit am entirely too curious and a little terrified to hear what some of these reasons could be.

In my experience (and I do work in education), I have yet to see a student who does not have the potential to achieve "good test performance" and have seen many, many cases where the problems are entirely the teachers themselves.