Insight crucial to good data
In the mid-2000s, a man went to his local Target supermarket to complain to the manager. His teenage daughter, who was still at school, had received a personally addressed Target flyer advertising maternity products, babywear, baby furniture, nappies and infant formula. ‘‘Are you trying to encourage my daughter to get pregnant?’’
The manager was very apologetic and could not really explain why it happened.
Some weeks later the store manager rang back to apologise once again. It was then that the man admitted: ‘‘Actually, there were some things happening at home that I was not aware of’’.
It turns out his teenage daughter was in fact pregnant and she had not told her parents yet.
Did the supermarket know the girl was pregnant before her own parents?
The answer is yes it did, and it was because of a man named Andrew Pole.
He is a data scientist and economist who started working for Target in 2002. His job was to use the masses of data Target collected from its customers via its loyalty programme and use it to increase sales.
Target knew that customers’ buying habits and routines were very consistent. Once a person bought a brand or product they tended to stick with it. The same was true for the shops they frequented.
But there are a few times in people’s lives when they do change their habits. This is usually around significant events such as a marriage, divorce, moving house or changing jobs.
But the biggest event that prompts people to change their buying habits is when they have a baby.
Target’s marketing department knew this and it wanted to get marketing material into the hands of these expectant mothers as early as possible so they would make additional purchases at Target stores.
The marketers asked Pole if he could identify when a woman was in her second trimester.
He came up with a list of 25 purchasing changes that a woman makes in the various stages of her pregnancy.
For instance, at around the third month women switch from scented soap to unscented soap.
In the fourth month, they start to buy supplements such as calcium, magnesium and zinc.
In the fifth month, they start buying cotton buds and hand sanitiser. When Target’s marketing department started identifying these buying changes, it sent out the mailers with baby and maternity products.
Clearly, that was a bit blatant and crude. So it started sending mailers that still contained the baby and maternity products but included other products like lawnmowers, plants and appliances. That way it was not so obvious.
Targets annual sales increased by 50 per cent between 2002 and 2010.
With results like this, organisations around the world have embraced data and invested heavily in data analysts.
The great thing about all this data is we can make really good graphs and convince ourselves that we are making better decisions because we have more data.
Except all this data does not mean we make better decisions. We can be blinded by data too. Hillary Clinton had a data guy called Robby Mook and the Clintons had a lot of faith in Mook and his data.
Bill Clinton had a hunch that they needed to connect with white middle-class voters in the Midwest. I would say Bill has a track record as being quite an insightful politician.
But Bill’s opinion was dismissed because Mook’s mathematical models showed they would win easily there.
Hillary did not visit Wisconsin once during the campaign because it was a safe Democratic seat.
Except the data was wrong and it was not a safe seat and Trump won it. The last time a Republican had won Wisconsin was Ronald Reagan in 1984.
Trump’s win is often explained as a result of under-educated, emotional and clueless middle voters. Maybe the Clinton loss was partly due to the reliance on hyper-educated and overly logical advisers?
All data comes from the same place – the past. It is not good at predicting anomalies or nontypical events.
The data did not predict a Trump win or Brexit. The data did not predict that Red Bull, a traditional Thai drink that tasted kind of gross, would become Coke’s biggest competitor.
The data did not predict that northern English mining towns would vote for an Eton-educated Boris Johnson but last month they did just that.
It seems to me data is only good if it is combined with insight. At which point it becomes smart data.
Unusual things happen when smart data is involved.