Google and Click Fraud: Behind the Numbers

Last week, we cited a study finding that 16.6 percent of all pay-per-clicks on the Internet were fraudulent in the fourth quarter of 2007, up from 14.2 percent for the same quarter in 2006. The statistic, as reported by MediaPost, was compiled by Click Forensics, an independent auditor that has created the Click Fraud Index. Given what must be billions of total ad-clicks on the Internet, as well as the constant development of click fraud detection programs by online ad providers, the number seemed surprisingly high.

Enter Google, which maintains that its internal click fraud algorithms tell a different story. According to the Google AdWords blog, the company maintains a “3 part system for invalid click detection” that involves filtering out anything funny-looking, analyzing the filtered clicks to determine how many are actually fraudulent, and then investing accordingly. Freakonomics spoke to a Google spokesperson, who explained the discrepancy between the first and second steps in the Google process:

Not all invalid clicks are click fraud — for example, the second click of a double click is not one with malicious intent. But since our goal is to filter out as much click fraud as possible, we cast the net of invalid clicks wide enough to effectively minimize that proportion and ensure that advertisers are only charged for valid clicks. The invalid click rate [i.e., the total number of clicks filtered out by Part 1] has remained in the range of less than 10 percent of all clicks every quarter since we launched AdWords in 2002.

After these figures are analyzed offline, Google maintains that the number of fraudulent ad-clicks not caught by the filters is less than .02 percent of the total clicks on Google ads. So it looks like, on Google’s end anyway, the number of click fraudsters may not be quite so huge, and that the metrics used by third party auditors such as Click Forensics may be flawed. A Google spokesman wouldn’t speculate as to how many people that .02 percent represents (though whatever the number is, it has likely declined, given that the company’s total ad-clicks were reportedly down 7 percent in January). Anyone care to do the math?


Brian

Interesting topic for a search marketer! Lots of folks have pointed out that engines certainly have a vested interest in understating click fraud numbers, but don't forget that auditing firms like Click Forensics have a similar interest in overstating them.

The major difference is that an auditing firm like CF can actually show a 3rd party exactly how they've come up with their numbers. Search engine click fraud routines are completely black-box, and advertisers must just take them at their word. (This is by necessity to an extent, as disclosing the methods could actually help fraudsters figure out how to better game the system.) There was much talk in the industry a few years ago about establishing an auditing/certification process for click fraud prevention efforts, but this seems to have fizzled out for some reason.

Just to put a little context around this, Google's 2007 revenue from PPC advertising was about 16.5B. Even using a conservative 1-3% fraud estimate the figure is fairly large, so it's surprises me the issue has been getting such little attention.

Read more...

edel

As some had stated here, it is interesting to read what Google's most profitable branch thinks about its own job performance. Nevertheless the reality is likely to be way different; If one teenager can write a code that can easily bypass the best Antivirus software available (were even there are many competitors), I can only guess what a corporation with huge economical incentives could do to fool 1 single branch of Google (that shadows all competitors)...

My guess is that Google do really try to minimize those false clicks, at least till the point where the competitors could prove it, but after that point... everything under the rug will work just fine and it's much cheaper too.

mmm

Come on now. So what if Google claims that its click fraud rate is like .02%? Their entire business is run on ads and click through ads in particular. They have every reason to engage in puffery regarding the less fraudulent nature of its clickers.

And, I doubt very highly that Google would ever let a third party validate its numbers.

That said, the truth is probably somewhere in between, say, 1 in 10 clicks is fraudulent?

Also, is there a standard definition of "fraudulent" clicks?

gcruse

"16.6 percent of all pay-per-clicks..."
"up from 14.2..."

16.6% is really 1/6 while 14.2% is really 1/7. Taking the numbers to one decimal place instead of simple fractions casts the penumbra of hyperbole over the point being made.

MS

Asking Google about click fraud is like asking George W. Bush about torture... Whatdayathink they're gonna say?

Scott

Maybe this is a matter of semantics, but... it seems that there's a really big discrepancy in how Google describe fraudulent clicks and how Click Forensics does so.

Is it possible that Click Forensics is reporting on total fraudulent clicks, while Google is reporting only on fraudulent clicks that get billed to their customers without being caught? From reading the post above, I can't tell what the definition of a fraudulent click is, exactly.

denis bider

As a Google advertiser who used to advertise heavily on their content network and subsequently spent a week analyzing the click fraud I got, I would distrust *everything* Google has to say on this topic. It is my strong impression that, if they don't engage in outright lying, then they intentionally shut their ears and ignore evidence, as it is in their interest to claim that click fraud is low when actual reality is far from it.

They know it's hard for you to prove that the phenomenon you're seeing is click fraud, so they internally promote an attitude which discourages acknowledging the problem.

Literally, I would trust *none* of what they have to say on this topic.

Venkat Kolluri

The percent click fraud rates across the board are much lower than what the original MediaPost article suggests. In fact, over the past few years lots of ad networks have improved the internal click filtering process to be more pro active towards click fraud just like Google. At Chitika, we have several steps in place to get better analysis of clicks.

For instance, detailed conversions data (i.e., the clicks that actually resulted in sales or sign-ups etc) from the merchants/advertisers side is fed back into our click monitoring systems and we are able to track the quality of clicks from various sources. We are no longer relying on guesstimates based purely on click activity and click patterns.

The processes that we put in place help us quickly hone in on the top click traffic partners and we are able to tighten up the click acceptance metrics at the publisher level. The click monitoring systems might not be that effective for small publisher clients, but the accuracy is pretty high for large publisher accounts. So we can expect to see higher rates for smaller accounts, but the click acceptance rates are pretty much on target for those accounts that contribute to the majority of the click traffic.

Therefore, the percent of click fraud is far lower than 16.6% measured in the MediaPost study.

Read more...

Justin James

Google? No click fraud? You beleive that? Let me tell you, I have managed a few ad campaigns through Google. Every client I managed, universally dumped their "AdSense" system because nearly 100% of the clicks were fraudulent! How else do you explain how dinky sites in India and Russia that seem to consist of nothing but stolen blog entries and archived mailing lists blow out a campaigns budget before the sun rises in the US every day? C'mon, Google is the Internet's biggest huckster, just look at their market valuation if you don't beleive me. Eric Schmidt ran Novell into the ground (even if Steve Ballmer claims to have buried him), and their two principle people need him to referee arguements as to what kind of beds to put in the corporate jet. And people thing the loony from Overstock (Byrne) is wacky... no, sorry, Google is a sham of a company build on ripping people off. Ever since they went public, Google has done nothing positive, and their ad systems are the worst of it.

J.Ja

Read more...

andy

As an alternative to payment derived from number of clicks there is a service called Project Wonderful (http://www.projectwonderful.com/ ) that allows people to obtain ad space using a never-ending auction avoiding clicks and click fraud. it seems like the right mix between newspaper style advertising and internet technology and seems more efficient.

Ido

Good post!
I'm sure Google advertisers are going in a direction of 'checking' what they are getting from Google in their systems. This way they could tell if they numbers (traffic/CTR/CPM) are making sense or not.

Bob

If Google is that good at filtering out bad clicks (.02), they should share the formula with Fair Isaac and other companies that detect fraud. To date they have not been able to build a system to detect and filter out fraud to this low rate. Google also does not allow ad networks to filter out traffic (per their contracts with the ad networks) because Google wants to be the sole source of determining what is fraud or not. They penalize ad networks for sending click fraud/spam into Google by cutting their commissions. On the other hand, they tell advertisers there is not a click fraud problem but you cannot audit Google data. The bottom line is Google keeps ad networks at arm's length by not providing them any insight into what traffic was bad. They keep advertisers at arm's length by telling them they cannot be audited. I don't know of any other business that you can spend as much as you do with online advertising and it cannot be reconciled. With a closed model like this, there has to be fraud and the only winner is Google.

Read more...

Googly

Google also has an Ad format called referals. The publisher gets paid only upon sale or registration or such predetermined action.

Microsoft is also talking about similar concepts.

http://guessworker.wordpress.com/2008/03/03/desktop/

John

In Google we trust.

DIS

google has a vested interest in minimizing both the number of fraudulent clicks and the perception of people of the number of fraudulent clicks.

Bill Hughes

As a web master of several sites with varying degrees of reliance on ASense, I can tell you that over the past year, Google's payout per click has dropped dramatically. So I suspect more web masters will risk click-fraud vs. closing up shop.

I've bought AdWords, and I've used AdSense. There is a huge disparity between what Google takes in and what they payout. Unless they loosen the purse-strings, I predict the whole contextual adwords model will collapse.

DB

I have to agree with 10. I am a lawyer and was once approached by someone wanting to sue their "business partner" - when I asked what the business was, this person, without any sense of shame, told me that it was based on Google Adsense and that her business partner was supposed to have set up 100s of websites so that they could put Google Adsense's ads on them... but this person had taken her money and not delivered - so a heartwarming story all round. I did, however, refuse the engagement.

Morley

A lot of people are casting doubt on Google's assurances on the basis that they have the most to gain from lowering the click fraud total.

They obviously did not read Google's rebuttal, and the paper they produced to discuss it in detail. They state that on an accounting and engineering level, the click accounting firms like ClickFacts or Click Forensics are miscounting. Most notably, they're counting page reloads as fraudulent clicks, and they're counting hits from other sites like Yahoo as google fraudulent clicks.

While Google has the most to gain from this analysis, it is close-minded and ignorant to dismiss their claims just because they stand to make money from it.

Caleb Mardini

The cost of click fraud shows up directly in my return per click/dollar spent on programs like Adwords. The greater the fraud the less money I am likely to devote to a particular campaign.

It's in Google's interest to mitigate fraud. As and advertiser however I don't have to worry about fraud as much as I have to watch my conversions and returns.

Drazan

I would say we should take a look into other adwords/adsense providers too. Most recently I was retained by several small businesses to do SEO of their websites and assist them in obtaining better web position. Some of them were using adwords campaigns which were not cost effective, producing little or no business.
After reviewing and analyzing numerous adwords campaigns, the majority were found to be well conducted.
I would say we are suffering from search engine industry wide schemes to inflate profits illegally. Even worse, this exploitation of public domain has resulted in a market place for a few privately owned companies rewarded by billion dollars earnings and profits.
More about at http://adwordsfraud.com/