If It's Raining, You Might Want to Reschedule That Interview

It is no secret that weather affects mood, and even behavior. The Bagel Man we wrote about in Freakonomics, who ran an honor-system business, received lower payments during foul weather. Now along come Donald Redelmeier and Simon D. Baxter from the University of Toronto with an interesting question: do applicants to medical school suffer if they happen to be interviewed on a rainy day? Redelmeier and Baxter looked at the data for nearly 3,000 applicants over a six-year period. The result:

Overall, those interviewed on rainy days received about a 1 percent lower score than those interviewed on sunny days (average score 16.31 v. 16.49, p = 0.042). This pattern was consistent for both senior interviewers (16.39 v. 16.55, p = 0.08) and junior interviewers (16.23 v. 16.42, p = 0.041). We next used logistic regression to analyze subsequent admission decisions. The difference in scores was equivalent to about a 10 percent lower total mark on the Medical College Admission Test.


DanielJ

Classic freakonomics.

Caliphilosopher

That isn't helpful. It's also not enough to just test the interviewers; perhaps the interviewees were worse at their interview because the weather affected their moods.

This study doesn't do enough to get rid of confounding factors. We should probably leave studies about external factors and moods to social psychologists, NOT economists.

DrS

Does that hold within each month that they are interviewing?

It could be that those graduating in 4 years, on time, are more likely to be interviewed during a sunny May day, rather than in the winter after graduating in December, or after taking time off to get their head together, or retake the MCATs multiple times.

Phil

Is it more likely the interviewer or interviewee who is affected by the weather?

But it's the University of Toronto, and there is a high correlation between weather on consecutive days. So maybe the crappy weather the day before influenced the Maple Leafs to lose, and that put everyone in a bad mood?

Nah. The Leafs suck in all kinds of weather, so that's probably not it.

JH

Oh, please, Caliphilosopher. It's not like there aren't holes in this material in general. But studies are what they are -- if you wait during study design to eliminate every confounding factor, nothing will ever be studied.

Of course the results shouldn't be taken as more than slightly intriguing and anecdotal (pretty much a universal for anything in their writing), but why impute a conclusion greater than the one given?

And by the way, what makes social psychologists more able to perform (or analyze) basic statistical research than economists?

Michael Giberson

Did they analyze the effects on ratings for applicants that reschedule at the last minute due to rain? I think your conclusion is not, strictly speaking, supported by the analysis.

And if a little rain discourages the applicant from showing up at a job interview, how much confidence will the interviewer have that the applicant would show up for work under rainy or snowy conditions?

noah

@Caliphilosopher

It doesn't matter whether the interviewer or interviewee or both performed worse. Rain correlates to lower score, period, so you're better off interviewing on a sunny day. It doesn't matter if you'll do better because it improves your mood or because it improves the interviewer's mood. Something is improved on a sunny day that gets you a better score, and that is useful information.

Marton

Caliphilosopher: the study was done by a doctor from a Toronto hospital and one of his med school students, not by economists. It was published in this blog by a journalist, not an economist.

While you do have a point in your comment, you do seem to have some preconceived notions against economists that might impair your neutrality...

Zach

In terms of results though, it doesn't really matter which party is affected more by the weather. There's no real need to establish causation here.

volatilitysmile

But then, given Toronto's weather patterns, do you call a minus 25 celsius a hiring freeze?

Kristi

This problem is easy to fix , and this time it is the social psychologists who figured it out. If the interview is on a rainy day, the interviewee should start by making some small talk about the weather. Norbert Schwarz and Gerald Clore did a study in 1983 where they called people at home on a rainy or sunny day and had them rate their overall life satisfaction. People reported being significantly less satisfied with their lives overall when they happened to be called on a rainy day compared to on a sunny day. But if the caller started by saying, "By the way, how's the weather down there?" then the effect went away and people reported being just as happy on rainy days as on sunny days.

Everything seems a little worse on a rainy day, but we don't think about the effect of the rain on our judgments, so we attribute the negativity to something else, like ourselves or the person we're interviewing. But if you're made aware of the rain, even in an indirect way, then you can compensate for it when evaluating yourself or another person.

I'm looking out my window right now at the rain in Bellingham, WA (just north of Seattle), and I think my life is pretty great.

Read more...

blue92

"[H]ow much confidence will the interviewer have that the applicant would show up for work under rainy or snowy conditions?"

I don't know about you, but my [imaginary] cat always seems to need emergency surgery whenever the sky turns the slightest hint of grey...

Doug Spencer

I have often wondered about a related phenomenon in the reverse direction: do law and business schools fail to attract top students if their campus visit day is scheduled during rainy/snowy weather?

Danielle

How about snow? Since the majority of interviews are done in fall and winter, I had more snowy interviews than rainy ones...

Kevin H

@Caliphilosopher

Your mistaking mechanism for effect. Yes, we don't know why exactly this happens, but the study is still very valuable in supporting the main thesis contained in the article.

Caliphilosopher

@ Kevin H - My point is that if this is just about correlation, then fine, but if this is about causation (which seems to be the more important topic), then we need more than just descriptive basic statistical analysis.

The study says the following: "We did not examine more complex combinations with time lags, such as when a sunny day followed multiple rainy days."

This is an important detail (I would argue a potential major confounding factor) that wasn't analyzed; doing so might make their evidence less supportive of their thesis.

I think that they are correct in saying that there are external factors that are correlated to the psychological states of the interviewers/interviewees, but at the end of their paper they say that "Calling attention to these issues may diminish their impact on judgment." - however, calling attention to the fact that psychological states may be correlated with moods doesn't help diminish these effects if they don't have a causal explanation.

I can run a study that calls attention to the fact that two phenomena are correlated, but if I'm actually trying to diminish the occurrence of one of the effects, me placing a signpost that alerts people to the correlation doesn't necessarily help in the diminishing department.

@ Noah - so, what exactly is that something that is improved on a sunny day?

@ JH - The above claim made in the paper would be one confound that could have easily been eliminated. I didn't say that they would be able to get rid of ALL confounding factors (no matter how many studies they carry out), but it's not unreasonable to analyze the time lag between sunny days and a priori rainy days.

Read more...

John

I think it would still be interesting and useful to know the causal effect, i.e. whether it is the interviewer or the interviewee, or both, who are affected. Because if you are the one being interviewed and you know about this correlation, you might want to do something about this. So if it is you yourself who gets put down by the weather, you can make yourself aware of this and hopefully that will improve your mood. If is the interviewer, then you probably have to resort to the sort of tactics that were suggested by Kristi.

j

@ a few commenters:

the interviews for canadian medical schools are not rolling as they are at some state schools. interviews are entirely conducted in the late winter-early spring with no date-chances correlation and in fact with interviewee self scheduling in some cases. therefore it is probably not a confound to say that those people who have worse applications are more likely to have interviews in the winter months etc etc

Tony

My son is interviewing at the University of Washington med school in Seattle next week. To have a good chance of a sunny day he'll have to reschedule in July.

StatsProf

The authors and commenters are missing another important factor. With data from almost 3000 people, a 1% difference, and p values like .04 and .08, the "effect size" of the weather must have been very small.

In other words, this is not "classic freakonomics"; rather, this study was potentially over-powered. If you have a data set with that many people in it, it is "easy" to find significant results for almost anything, even meaningless variables like participant ID number.