A friend of mine suggested that I read Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t. If you don’t know Nate Silver, he’s a statistician who studied economics at the University of Chicago and has an uncanny ability for predicting the outcomes of elections. In 2012 he correctly predicted the Presidential voting outcome on the day of election in all 50 states and the District of Columbia; and he correctly predicted the outcomes of 31 out of 33 US State Senate races.
While Nate is obviously a fabulous statistician, the title of his book acknowledges that many predictions fail. If you’ve read his book or reviews of the book, you’ll know there are many takeaways to be had. Most deal with the art and science of prediction and whether it is possible to predict outcomes. As marketers, including myself, increasingly look towards big data to answer the question of who is most likely to purchase we should consider the advice provided by Nate. Big data probability modeling can steer you in the right direction, but it can also steer you in the wrong direction.
Google Understands How to Use Big Data
My big takeaway or conclusion if you will from Nate’s book was an eye opening moment that gave me greater insights into the application of Bayes’ theorem for probability modeling. The theorem provides a mathematical formula for calculating probabilities. If the math is absolutely correct, then why are probability models so often wrong? In the last few pages of the book Nate provided his opinion as to why Google is successful with big data as a company.
Probability outcomes are limited by the inputs into the models. The very reason that we can’t predict earthquakes is that we don’t have the “right” data inputs to support probability models for earthquakes with significant accuracy. However, we can predict human behavior including whether someone is inclined to make a purchase if we have the “right” inputs into the probability models. Bayes’ advocated that you start with what you know and adjust the inputs into the probably models as new facts are learned.
According to Nate:
Companies that really get “Big Data” aren’t spending their time in model land. They are running thousands of tests each year and experimenting on real customers.
Google is good because they are willing to try new things with the risk of failure. Failure is not only accepted, but appears to be embraced. By supporting the practice of repeated experimentation with the risk of successes and failures, they are able to learn what works and what does not more quickly. Through this repetitive process of testing and learning, you can throw out what is known to fail and build upon the success with additional new testing and trials of what might work and what might fail. The repetitive process can provide new insights to refine the inputs and improve the accuracy of probably models.
Applying Bayes’ Theorem
In executing marketing campaigns, I’ve come to realize that the expected outcomes and the realized outcomes do not always align all that well. Marketers need to become better at analyzing the gaps and understanding the reasons for why expected and actual outcomes vary. This will enable us to make adjustments and to test and evaluate again and again in a repetitive process that closes the gap between expected and actual outcomes of the marketing mix. Through this refinement of testing and learning, the effectiveness of your marketing campaigns will improve at a rate faster than your competitors which ignore this analysis for the sake of not wanting to acknowledge their failures.
Time to adjust and embrace failures
Not “winning” meant failing to me and was always a tough pill to swallow. I grew up in a family of good athletes, so if I finished second or third in a race with 50 or 60 competitors, my grandmother would ask, “Why didn’t you finish first?” First place was always the expectation, yet rarely the outcome. Although, I never embraced failure, I leveraged it to push me harder and farther. To some extent this has helped to define who I am today.
Although I don’t seek failures, I’ve come to learn that I must embrace them as an opportunity to learn what not to repeat in the future. The lessons of both, successes and failures can improve the inputs into probability modeling as advocated by Bayes. Without accepting the risk of failure you’ll have less of an opportunity to improve the analysis and accuracy of big data. With so many new technologies being introduced to marketing organizations today, it’s a great time to test, fail and succeed. Those who embrace calculated risk taking and are willing to embrace failures as well as successes are going to learn faster as to what will succeed in the marketplace.
What is your tolerance of failure? Willing to not only accept failure but rather embrace failure as an opportunity to learn and refine your go to market execution? If you’re willing and in an environment where failure is accepted, your ability to refine and adjust your marketing mix and execution based upon the testing of probable outcomes will allow for accelerated improvements to outpace your competition.