Search the Site

Posts Tagged ‘statistics’

What's the Real Crime Rate in China?

Official statistics would certainly suggest that crime in China is extremely low.  Murder rates in China are roughly one-fifth as high as in the United States.  According to the official crime statistics there, all crimes are rare.  China certainly feels safe. We walked the streets in rich areas and poor and not for a moment did I ever feel threatened.  Graffiti was completely absent.  The one instance where I thought I finally found some graffiti near a train station in the city of Shangrao, the spray painted message on a bridge turned out not to be graffiti, but rather a government warning that anyone caught defecating under the bridge would be severely punished.

Yet, there were all sorts of odd behaviors that made it seem like some crimes were a big problem. 

First, there seemed to be an obsession with the risk of counterfeit money.  Our tour guides felt the need to teach us how to identify fake money.  Whenever I bought something with currency, the shopkeeper went through a variety of tricks to validate the legitimacy of the bills. 



Is the Analytics Revolution Coming to Football?

In the New Republic, Nate Cohn explores the small but growing role of advanced statistics in football. Projects like Football Freakonomics notwithstanding, the NFL isn’t usually thought of as a realm where stats hold all that much sway, in part because the game is so much more of a complex-dynamic system than, say, baseball. Here’s Cohn on one big change fans might notice if more coaches start relying on statistics:

The one place where fans could see analytics at work is in play calling, which also happens to be the place where analytics could impact the average fan’s experience of the game. The numbers suggest, for instance, that teams should be aggressive on fourth down, and that it’s better to go for first down with a lead in a game’s final minutes than to run the ball on third down to run out the clock. Yet even the teams with well-regarded analytics departments, including San Francisco and Baltimore, largely adhere to a conservative and traditional play calling approach: the coaches “just aren’t listening to them yet,” [Brian] Burke says. And the few coaches with a reputation for following the statistics, like New England Patriots coach Bill Belichick, aren’t even close to as aggressive as the numbers would advise.  



Calling All Data Memoirists

The statistician Andrew Gelman has asked us to publicize what sounds like a nifty project: a Year-in-the-Life look at what data hounds and statisticians actually do:

So here’s the plan. 365 of you write vignettes about your statistical lives. Get into the nitty gritty—tell me what you do, and why you’re doing it. I’ll collect these and then post them at the Statistics Forum, one a day for a year. I think that could be great, truly a unique resource into what statistics and quantitative research is really like. Also it will be perfect for the Statistics Forum: people will want to tune in everyday to see what comes next.

In an e-mail, he adds:

I think it would be a great service to the professions of quantitative research to get vignettes from a wide variety of statistical practitioners.  (I’d be interested in hearing what empirical economists do during their days too!)  So I’d like to spread the net wide and get lots of stories from people.

And yes, for those of you who read the agate type, this post goes in the Bygones Being Bygones file.



FiveThirtyEighter Nate Silver Answers Your Questions About Politics, Baseball, and The Signal and the Noise

We recently solicited your questions for Nate Silver regarding his new book The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. Not too surprisingly, a lot of the questions were about politics and baseball. Below are Nate’s answers to some of them. Thanks to him for playing along and to all of you (as always) for sending in the excellent questions.

Q. Under what circumstances will a voter actually change his/her mind about whom to vote for? I understand that this rarely happens (this study for example), and that most of the action involves undecided voters deciding whom to vote for.

Also, if political scientist are right that voters rarely change their minds, how can a large swing in the polls ever occur? A classic example that your briefly mention in your book is that of Michael Dukakis, who was ahead of GHW Bush by 10% at one point in 1988. –Alan T

A. We see more big shifts in the primaries, when voters don’t have that much information about the candidates. Dukakis was a relative unknown at the start of the 1988 race, before the two parties could advance their own narratives. You rarely see big swings in voter conversion in late stage presidential races, though. If I knew how to cause such a swing, I’d be drawing a big salary from one of the campaigns right now.



Beware the Weasel Word "Statistical" in Statistical Significance!

As Justin Wolfers pointed out in his post on income inequality last week, the Census Bureau was talking statistical nonsense. I blame the whole idea of statistical significance. For its weasel adjective “statistical” concedes that the significance might not be the kind about which you care. Here, I’ll explain what statistical significance is, and how its use is harmful to society.

To evaluate the statistical significance of an effect, you calculate the so-called p value; if the p value is small enough, the effect is declared statistically significant. For an example to illustrate the calculations, imagine that your two children Alice and Bob play 30 rounds of the card game “War,” and that the results are 20-10 in favor of Bob. Was he cheating?

To calculate the p value, you need an assumption, called the null (or no-effect) hypothesis: here, that the game results are due to chance (i.e. no cheating). The p value is the probability of getting results at least as extreme as the actual results of 20-10. Here, the probability of Bob’s winning at least 20 games is 0.049. (Try it out at Daniel Sloper’s “Cumulative Binomial Probability Calculator.”)



Is Income Inequality Rising, and Are a Lot of Feathers Heavy?

New data on income inequality in the United States were just released.  And they provide a useful teaching moment. The graph below, which comes from the Census Bureau, shows the evolution of the Gini coefficient since 1967.  It’s pretty clear that this measure of inequality has been rising pretty much through this whole period.

p>But here’s how the Census Bureau chose to describe these data:

Based on the Gini index, income inequality increased by 1.6 percent between 2010 and 2011; this represents the first time the Gini index has shown an annual increase since 1993, the earliest year available for comparable measures of income inequality.

Say what?



Is There Another Side to the "Hurricane Death Toll"?

Miguel Sancho, a senior producer with ABC’s 20/20, writes in with a question I’ve often wondered myself but cannot answer. Can you?

A thought – every hurricane season we see headlines ascribing blame for lives lost on a given storm. “Hurricane Irene Blamed for Five Deaths in North Carolina,” etc. Certainly when people drown, are killed by floating debris, or die because they can’t make it to the hospital, the statistic sounds logical. But it occurred to me that perhaps, in the interests of fairness and accuracy, we should also give Hurricanes “credit” for lives not lost thanks to the interruption of normal human activity. How many homicides, vehicular fatalities, or drug overdoses didn’t happen [last] week in New Orleans, for example, because people were otherwise occupied protecting themselves from Hurricane Isaac? Just wondering if anyone has ever studied this, comparing average morbidity rates in hurricane zones to the stats during the times when hurricanes roll through.
 
This is not to suggest that overall, hurricanes are a social good. Bastiat’s broken-windows fallacy and all that. But perhaps in this one particular metric, we aren’t seeing the whole picture.

Please don’t judge Sancho’s observation as insensitive to the death and destruction caused by the hurricane itself. I can assure you he is not.



Is "Statistically Significant" Really Significant?

A new paper by psychologists E.J. Masicampo and David Lalande finds that an uncanny number of psychology findings just barely qualify as statistically significant.  From the abstract:

We examined a large subset of papers from three highly regarded journals. Distributions of p were found to be similar across the different journals. Moreover, p values were much more common immediately below .05 than would be expected based on the number of p values occurring in other ranges. This prevalence of p values just below the arbitrary criterion for significance was observed in all three journals.

The BPS Research Digest explains the likely causes:

The pattern of results could be indicative of dubious research practices, in which researchers nudge their results towards significance, for example by excluding troublesome outliers or adding new participants. Or it could reflect a selective publication bias in the discipline – an obsession with reporting results that have the magic stamp of statistical significance. Most likely it reflects a combination of both these influences. 

“[T]he field may benefit from practices aimed at counteracting the single-minded drive toward achieving statistical significance,” say Masicampo and Lalande.



Time for a "Brave New Math"?

Channeling some of the logic in our “Health of Nations” podcast, Peter Marber argues in World Policy Journal that it’s time for a “brave new math.” Marber takes issue with economists’ ongoing reliance on old measures of economic health — GDP, inflation, and unemployment:

Traditional measures point to an American economy that’s up even when Americans are feeling down. Across Europe and in Japan, there is also a sense of confusion over current economic directions—a universal sense that the numbers that have been our staples are increasingly meaningless to everyday people.

Newspapers, radio, and television routinely spout headlines about key statistics on GDP, inflation, and employment—astonishingly influential indicators computed in the United States by the government’s Bureau of Labor Statistics and in capitals around the world. Most seem to have little correlation with the realities on the street. 



The Human-Rights Statistician

A Foreign Policy article by Tina Rosenberg profiles Patrick Ball, a human rights statistician. Rosenberg describes Ball’s testimony at the trial of  Slobodan Milosevic:

 “We find evidence consistent with the hypothesis that Yugoslav forces forced people from their homes, forced Albanian Kosovars from their homes, and killed people,” Ball said…

Could the movements of refugees have been random? No, Ball said. He had also plotted killings of Kosovars and found that both phenomena occurred at the same times and in the same places — flight and death, hand in hand. “I remember well the moment of astonishment that I felt when I saw the killing graph for the first time,” Ball replied to Milosevic. “I assumed I had made an error, because the correlation was so close.”



The Life of the Number-Crunching Analyst

Thousands of economics majors head off to industry each year to work as analysts. They’re lured by the promise that they’ll learn a lot, work hard, play hard and get ahead.  But is it true?  Who better to ask than the brilliant young analyst Elisabeth Fosslien.  And as a good young analyst, she’s distilled her portrait of life as an analyst into charts.  Having once lived the analyst life—my first job out of college was at the Reserve Bank of Australia, crunching numbers and making charts—all of these resonated with me.



The Statistical Significance of Beer

According to a new paper by Stephen T. Ziliak, it was a brewer at the famed Guinness beer company, William Sealy Gosset, who first began to explore the concept of statistical significance:

Gosset (1876–1937) aka “Student” – he of Student’s t-table and test of statistical significance – rejected artificial rules about sample size, experimental design, and the level of significance, and took instead an economic approach to the logic of decisions made under uncertainty. In his job as Apprentice Brewer, Head Experimental Brewer, and finally Head Brewer of Guinness, Student produced small samples of experimental barley, malt, and hops, seeking guidance for industrial quality control and maximum expected profit at the large-scale brewery. In the process Student invented or inspired half of modern statistics.



Where Murder Is Falling, and Rising

Encouraging news via the Associated Press: For the first time in almost half a century, homicide has fallen off the list of the nation’s top 15 causes of death. In Mexico, meanwhile, the murder trend continues to move in the opposite direction: During the first nine months of 2011, some 12,903 people were killed in drug-related violence—11% more than the . . .



The Perils of Drunk Walking (Ep. 55)

In our latest Freakonomics Radio on Marketplace podcast, Stephen Dubner looks at why the first decision you make in 2012 can be riskier than you think. (Download/subscribe at iTunes, get the RSS feed, listen via the media player above, or read the transcript.)

The risks of driving drunk are well-established; it’s an incredibly dangerous thing to do, and produces massive collateral damage as well. So if you have a bit too much to drink over the holiday and think you’ll do the smart thing and walk home instead — well, that’s not so smart after all. Steve Levitt has compared the risk of drunk walking with drunk driving and found that the former can potentially pose a greater risk:



"Football Freakonomics": When Good Stats Go Bad

In the third segment of “Football Freakonomics,” Dubner examines how impressive stats in the NFL are often indicative of bad results. For example, we all want a quarterback who throws for big yardage. But for all the times a quarterback threw for 400 yards or more last season, how many of those games did his team actually win?



Nazis, Sunken Ships, and a 67 Year-Old Game of Telephone

This is a guest post by Jeff Mosenkis, a freelance producer with Freakonomics Radio who holds a Ph.D. in psychology and comparative human development.
Nazis, Sunken Ships, And a 60 Year-Old Game of Telephone
By Jeff Mosenkis
Did you hear the one about the two statisticians who go deer hunting? The first one misses his shot ten feet to the right of the deer; the second one misses ten feet to the left of the deer. They then high five each other and shout “Got him!”
While the quantitative method might not work for hunting, it apparently does for finding sunken warships. NPR’s Alix Spiegel reported this remarkable story about two Australian cognitive psychologists who used a statistical distribution to find two sunken World War II ships, 67 years after they were lost.
On the evening of November 19, 1941, the HMAS Sydney was off the coast of Western Australia when it exchanged fire with the German HSK Kormoran, and sunk with all 645 crewmen aboard. It was a national tragedy, particularly because nobody knew exactly what happened to the ship and why it sunk. The German crew scuttled their damaged ship, and 317 surviving German sailors were picked up in lifeboats at sea or on shore and interrogated.



A Lottery Loophole (Sorry, Now Closed) in Massachusetts

In the Boston Globe, Andrea Estes and Scott Allen write about how people have been taking advantage of a statistical quirk in the rules of an obscure Massachusetts Lottery game called Cash WinFall. A Michigan couple in their 70s, Marjorie and Gerald Selbee, spent three days buying more than $600,000 in Cash WinFall tickets from two convenience stores in Sunderland, Mass. Their timing was purposeful:

For a few days about every three months, Cash WinFall may be the most reliably lucrative lottery game in the country. Because of a quirk in the rules, when the jackpot reaches roughly $2 million and no one wins, payoffs for smaller prizes swell dramatically, which statisticians say practically assures a profit to anyone who buys at least $100,000 worth of tickets. During these brief periods – “rolldown weeks’’ in gambling parlance – a tiny group of savvy bettors, among them highly trained computer scientists from MIT and Northeastern University, virtually take over the game. Just three groups, including the Selbees, claimed 1,105 of the 1,605 winning Cash WinFall tickets statewide after the rolldown week in May, according to lottery records. They also appear to have purchased about half the tickets, based on reports from the stores that the top gamblers frequent most.



A Strange Sentence About Grand Slams

From today’s Times, an article by David Waldstein called “Mets’ Stretch Without a Slam? Gone. Gone“:

The Mets had gone 299 games and 280 plate appearances with the bases loaded since their last grand slam, while their opponents had hit 18 during that span. So when the opportunity arose in the fourth inning Tuesday night — with Jason Bay at the plate, no less — the chance of a Mets grand slam was slim.

Was the chance of a grand slam really so slim?



A Young Reader Asks: Is There an Elitist Oligarchy in the Underworld of Knitters?

A reader named Sarah Johnson, who is passionate about crocheting, noticed something curious about the demographics of a user-rated knitting-and-crocheting website called Ravelry. Sarah is graduating from high school in June, and plans to major in psychology and pre-med. She writes: “I joined a free site for knitters and crocheters called Ravelry. As far as a little Internet research goes, it’s one of the biggest knitting and crochet sites out there, with over 1 million members. CrochetMe, another big site with comparable features, has 224,000 users. Crochetville, a large crochet forum, has only 46,000 members as of today. 414,974 people like ‘knitting’ on Facebook. By comparison, only 5,560 people like ‘crochet.'”



Cracking the Lottery Code

In Wired, Jonah Lehrer profiles Mohan Srivastava, a Toronto statistician who seemingly cracked the scratch-lottery ticket code. “The tic-tac-toe lottery was seriously flawed,” writes Lehrer. “It took a few hours of studying his tickets and some statistical sleuthing, but he discovered a defect in the game: The visible numbers turned out to reveal essential information about the digits hidden under the latex coating. Nothing needed to be scratched off-the ticket could be cracked if you knew the secret code.”



The Authors of Scorecasting Answer Your Questions

Last week, we solicited your questions for Tobias J. Moskowitz and L. Jon Wertheim, the authors of Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won.
You shot them a lot of good questions — heavy on the NFL, to be sure.



Bring Your Hidden-Side-of-Sports Questions to the Scorecasting Authors

Earlier this week, Tobias J. Moskowitz (a University of Chicago finance professor) and L. Jon Wertheim (a Sports Illustrated writer) contributed a guest post on black NFL coaches, which was an adaptation of their new book Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won. You may recall this as the book Levitt described as “[t]he closest thing to Freakonomics I’ve seen since the original,” much to his wife’s chagrin.



Scorecasting: A Guest Post

When my wife saw the cover of the new book Scorecasting by Tobias J. Moskowitz and L. Jon Wertheim, which was sitting on my bedside table, all she could do was shake her head.



A Reading List for Stats Fans

Andrew Gelman, a statistician at Columbia University, offers some reading suggestions for fans of statistics (no, they are not as numerous as fans of, say, Harry Potter, but still …).



Are NFL Coaches Starting to Listen to Economists?

Are NFL coaches starting to listen to economists?
My gut feeling is that the answer to that question is almost certainly a resounding “no.” There are at least three pieces of data that hint at the possibility that economists might be making some headway.




John Brenkus Answers Your Peak-Performance Questions

We recently solicited your questions for John Brenkus, author of The Perfection Point, a book about the limits of athletic achievements. Read on for Brenkus’s thoughts on those special swimsuits at the Beijing Olympics and the most surprising perfection point he encountered in his research. Thanks to everyone, especially John, for participating.




Detecting Political Momentum Is Harder Than You Think

Over at FiveThirtyEight.com, Nate Silver has a post attempting to debunk the idea that there is momentum in political campaigns. But I think he’s wrong. And his post provides a fun opportunity for a simple statistics lesson on the difficulty of discovering momentum.