If you go to Google and type ‘Amazon’ you’ll probably see a result labelled ‘paid’. It was these kinds of ads - known as paid search ads -that economist Steve Tadelis grew suspicious of after he was hired by eBay in 2012.
During his time there, an external consulting firm claimed that paid searches for the keyword eBay on Google received the highest return on investment. However Steve couldn’t make sense of it. In order to see the ad, someone was typing eBay - so where else were they going?
Along with Thomas Blake and Chris Nosko, Steve conducted a controlled experiment which halted the paid search ads for three months. The results showed no measurable impact to eBay’s sales.
“The chief financial officer who owns the pockets of the company realised - wait a minute, we may be putting, you know, bundles of money on trucks sending over to our so called advertisers and those trucks are coming back pretty empty.”
The findings ruffled some feathers among the external consultants who Steve says encourage data to be treated “more like religion than science”.
He likens it to medicine. “300 years ago, they would say well, take a feather from a young chick, and mix it with a leaf of asparagus and put it under your pillow for three nights and you should get better. Thankfully, medicine has evolved a long way since then. But the consumers of the medical advice are still pretty much clueless about medicine. The same thing happens with data science and analytics.”
Steve is now at Berkeley, where he teaches business leaders the fundamental aspects of data science and business analytics.
“It's really to give people enough confidence to be able to ask questions, and then know that if the answers are unclear, it's not your problem. It's the people who're answering you who don't know how to deliver a good answer. “
