Discriminating Software

The Economist takes a look at the software that big companies are using to sort through job applicants. It finds that people who use Chrome and Firefox browsers are better employees, and people with criminal records are suited to work in call centers. One drawback to having a computer sort potential employees is that its algorithms may treat some variables as proxies for race, as discussed in our “How Much Does Your Name Matter?” podcast, in which the Harvard computer scientist Latanya Sweeney found that distinctively black names are more likely to draw ads that offer arrest records. 

(HT: Louis Henwood)


So... a computer algorithm that was generically designed to correlate people's demographic characteristics and the ads they click determined that one demographic was more likely to click on a certain type of ad. Because it was "black" and "arrest records" rather than "male" and "sports tickets" or "elderly" and "medical alerts" a kerfuffle ensues.

I guess one could try to specifically design an algorithm that would weed out any possible correlation to any proxy for race, but the assumptions ("stereotypes," if you will) required to do so would be far more insidious than just letting the vanilla algorithm run its course.


Imagine a benign characteristic. (e.g. drives model X car.) Imagine 100% of demographic D who drive model X car are horrible employees, and 100% of D who don't are good employees. Imagine that among minority group M, driving model X car has no relation to employee quality. However, imagine if the majority of M drives model X.
Also, imagine a majority of the applicants are in D. Therefore, driving a model X has a strongly positive correlation across the applicant pool, but has the effect of discriminating against minority M (even though it doesn't actually relate for them).

(Yes, I know this is contrived, and yet I bet you could find examples that this match.)

[WORDPRESS HASHCASH] The poster sent us '0 which is not a hashcash value.

Enter your name...

Yes, you're right: an algorithm that is designed to reject X will reject X, even if X shouldn't be rejected in every case. For example, if you want to find someone who is rich, then you might reject everyone who drives an old car, but you'd be screwing up if you rejected the person driving a Rolls Royce Phantom.

Of course, it also works the other way around: an algorithm designed to accept X will accept X, even if X shouldn't be accepted. After all, that "rich person's" car might be rented, borrowed, or stolen.


"having a computer sort potential employees is that its algorithms may treat some variables as proxies for race"

So these proxies for race might include IQ, criminality, etc. and that would be a problem for an employer, how? Arent employers to use actual data even if the truth is politically incorrect?

Enter your name...

Depends on their goals. If their goals include not getting sued into the ground for unintentional-but-unreasonable discrimination, then they do need to worry about this. It's called "disparate impact", and it's why places with smart HR departments don't require four-year university degrees for receptionists and janitors, even when the job market is so weak that people with these degrees are desperate.


It will disriminate between a good candidate and a bad one. Why does the word "discriminate" have to mean bigotry?


When you apply for a job online, which is the only way to do it now at most companies, and you have to spend all day filling out various surveys, skills tests, and personality tests only to be rejected by the system because of some seemingly meaningless correlations in your data, it can be very disheartening. It's no wonder so many of the unemployed have stopped looking for work. Just having that label "unemployed" is an automatic disqualifier at most places, and unemployed people are well aware of that. Of course the algorithms are selecting better employees "on average", but meanwhile thousands of good people are slipping through the cracks. Maybe they should turn to a life of crime -- it might at least help them get a call center job down the road.


The issue here is not with discrimination. All hiring is discrimination to separate 'who fits' from 'who doesn't'. The issue here is that of risk. Algorithms may contain unknown systemic biases particularly with incomplete or wrong feedback mechanisms. And by the time the biases are uncovered, significant loss would have occurred.


'whom fits'. Sorry, you're disqualified from further consideration for the position.


Guys, I love your blog, but PLEASE do not make your videos auto-play. I tend to open multiple tabs and read them one by one, so if one of the background tabs suddenly starts speaking it can be disturbing and leads to a "which one is it" search... Thanks!


What's next? Discriminating against younger people by charging them with a higher car insurance rate?