Free Super-Crunching Software

I probably have an unhealthy attraction to the powers of Excel. I taught my daughter how to use it when she was 7. When I teach corporate finance, I try to make sure that my law students come away from the course knowing how to crunch in Excel.

It would be embarrassing to teach students how to use Microsoft Word in a law-school course; but one of the goals of my corporate finance course is to make sure that they can comfortably manipulate its numerical cousin.

A middle-school math teacher recently told me that there are some things you can do on a graphing calculator that you just can’t do in Excel. I’m pretty sure (like 99 percent sure) that this is not true. In fact, Microsoft has expanded the functionality of Excel so that it’s starting to invade the domain of statistical packages.

The just-published (shameless plug) paperback edition of Super Crunchers has a new chapter that describes several different free tools that make it easier and easier to crunch numbers.

1. Microsoft has a new data-mining add-in that lets you run all kinds of cool statistical procedures inside Excel. Taking a page from the Google playbook, Microsoft is just giving this add-in away (but it only works if you’ve purchased the Office 2007 version of Excel).

2. Google (taking a page from its own playbook) is giving away its Website Optimizer, which will let you run randomized experiments on your own web page.

Any webmaster who is not running randomized trials on different page content is making a serious mistake.

Here’s an explanatory video. I’ve used the Website Optimizer myself and it is a joy to use.

3. I’ve created and assembled links to a bunch of cool “prediction tools” that let you plug in a few numbers and predict how long you’ll live, predict your due date (if you’re pregnant), rate the quality of a book title, or even predict political or sporting contests.

One of the cool things about these tools is that they provide feedback on the precision of predictions that is easy to digest. When you see the results of an experiment like this one below, you have a pretty clear idea of not only the winner, but of how confident you should be in the results.

INSERT DESCRIPTION

(As with all other statistical tests, you should not just blindly accept the p-values in the print out, but these graphics are still a huge leap forward.)

A fourth freebie is the open-source statistical package called “R.” While most members of the Freakonomics crowd tend to use Stata as their statistical package of choice (and businesses tend to run SAS or SPSS), R is the Linux of statistical software. It lets you do an awful lot for free.

Of course, having mastered the commands of Stata and SAS, I have poor incentives to learn the commands of a new (GNU) software. And R is probably not kept up to speed on the cutting-edge empirical methods as quickly as the traditional packages. (I should disclose that SPSS and SAS have paid me handsomely to give Super Crunching talks, so I may not be the most objective observer.)

But then again, R has plenty of power to run the vast majority of statistical techniques. There is still a huge discrepancy between the techniques that are used by academics and those used in business.

In fact, here’s a Super Crunchers bleg: Can anyone identify an instance where a business has run an instrumental-variables regression?

The I.V. approach has been around for decades and is a standard (if misused) technique in hundreds, if not thousands, of academic articles. But provocatively, I’d almost bet that it has never yet been used by a corporation to help make a business decision. We’ll send some Freakonomics schwag to the first person who can prove me wrong.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 59

View All Comments »
  1. B Reilly says:

    One thing that Excel can’t do very well is statistics:

    http://pages.stern.nyu.edu/~jsimonof/classes/1305/pdf/excelreg.pdf

    Thumb up 0 Thumb down 0
  2. Matt B says:

    Can anyone tell me what software package that screenshot that Ian posted is from? Is it Google Analytics?

    Thumb up 0 Thumb down 0
  3. Jason B says:

    I work in business (defense analysis) and we do have a license for SAS, but a lot of people around here prefer R. The reason is that the SAS business model relies on selling training. We often don’t have time for formal training, and R gives you lots of free reference and learning resources.

    Unfortunately I can’t say I’ve ever heard of anyone around here doing an instrumental variables regression.

    Thumb up 0 Thumb down 0
  4. Jason B says:

    One more note. Excel cannot solve complex expressions in terms of your choice of variables. Excel cannot do derivatives and integrals. A good TI or HP graphing calculator can do these things.

    Some of them are banned from schools or testing situations because of this.

    Thumb up 0 Thumb down 0
  5. Quin says:

    Reminds me of the review of “Advanced Excel for Scientific Data Analysis” that I ran across recently:

    http://books.slashdot.org/article.pl?sid=08/10/01/1329243

    Thumb up 0 Thumb down 0
  6. colin says:

    Excel can solve complex expressions with any variables, and it can perform derivatives and integrals. Its not in as easy a format to use with math textbook type questions, but it can be done.

    Thumb up 0 Thumb down 0
  7. tudza says:

    Beware of Excel automatic formatting. Here’s one article on the subject:

    http://www.biomedcentral.com/1471-2105/5/80

    Thumb up 0 Thumb down 0
  8. David says:

    HR departments use multi-variate regression all the time to set values on jobs that are difficult to match to available market data.

    Contact me offline for REALLY specific examples if you need.

    Thumb up 0 Thumb down 0