The Razor’s Edge of Analytics

    1024 681 Tu Nguyen-Brown

    Occam’s Razor is one of the most well-known blogs and resources in the digital analytics industry. It’s probably because its author, Avinash Kaushik, is the godfather of digital analytics. When he began his career, there were no phones, tablets or cross-channel marketing. Fast forward to 2016, and digital analytics now encompasses mobile devices and households that average eight different devices. Tracking and measuring your consumer isn’t so clear-cut anymore, and the amount of data that is out there is immeasurable. How do you approach that?

    the Cliff’s Notes 

    In his blog post from April 4, “A Great Analysts’ Best Friends: Skepticism & Wisdom”, Kaushik talks about this surge in data and the resulting “explosion of experts” in the field. His post is a plea to be better analysts by virtue of skepticism and wisdom, as the post title would imply. Kaushik’s posts are typically lengthy, and this one is no exception, so I’ve summarized his key points for you below.

    • Be skeptical of the data that is presented to you. Question the data source and whether or not these data points are really correlated. To make his point, he cites a report by The Economist studying the link between ice-cream consumption and PISA reading scores by country. Upon first glance, the data suggests causation: the more ice cream you eat, the better your reading score (how I so wish this to be true!). However, a range of other socio-economic factors that, when considered in relation to the data points, would imply that correlation is much more likely. As does common sense.

    The Economist data chart studying the link between ice cream consumption and PISA reading scores by country.

    • When reviewing data like in the example above, look for sample bias. Was there an assumption made in doing the analysis?
    • Statistically, did the data collected and used have all the right boundaries and controls? In other words, is it reproducible?
    • Play the devil’s advocate and look for the alternative, or try to test against the hypothesis.
    • Don’t get paralyzed by the data. Once you start looking behind the curtain, it’s easy to be terrified by the way data is collected. An educated mistake is better than no action at all.
    • When you have made a conclusion, deliver the insights and recommendations in a timely manner so that you can impact business
    • Solve for wisdom – give more than just data. Drive business decisions and change minds.

    a panda’s thoughts

    As I mentioned above, this was a long blog post with a lot of pieces—in talking about too much data, Kaushik may have provided a lot of words! While I agree with many of his points, I have some skepticism about others. After all, he encourages skepticism!

    On his points about data paralysis, I wholeheartedly agree. As someone who designs analytics implementations for our clients, I find myself in a personal struggle sometimes. How should I set this up so that it will be useful? Sometimes we have the luxury of implementing a tool like Adobe Analytics from scratch for a client. Having a blank canvas to work on is an exciting anomaly in the analytics world. At previous companies, I’ve had to work with implementations that were messy, complicated and at times useless, resulting in data we couldn’t trust.

    His emphasis on the timely delivery of analysis and insights, however, gives me pause. I challenge his quote, “An educated mistake is better than no action at all”—this depends on your audience, what exactly that data is and how it will be perceived and acted upon. In college, I had a similar motto of “done is better than good,” and I have my degree as a result…but with data and analytics, this motto doesn’t hold water.

    Kaushik gives a 100/80/60/40/20 percent certainty example of the different recommendations you would give depending on the percentage of skepticism you have over the data. The problem with this rule is the repercussions of the insights or recommendations given with any level of certainty under 100%. It depends on how it will be used to influence decisions, and how any uncertainty you have may harm the trust the client has established in your expertise.

    Kaushik suggests, “Avoid being paralyzed by perfection to make a decision that reflects your certainty of the data because the business needs timely decision making.” This works for some business decisions, but not for all of them. Again, know your audience and the potential impact of your insights. When talking about data, it’s hard to recover from mistakes, so be certain it’s right. With analytics, I think the preferred motto to follow is “quality over quantity.”

    how we make sense of data

    In our effort to help clients make great business decisions based on data, here are some of the questions my team and I ask ourselves before, during and after collecting it.

    1. Is the goal of this data collection to merely report, or can it help in making business decisions? How can we make it valuable?
    2. Will my client understand how to use this information if I’m no longer there supporting them?
    3. Is the data able to answer our clients’ frequently asked questions? What are their expectations for data, reporting, analysis and insights?
    4. What do I want to dig into more deeply? We’re hired for our expertise, and sometimes clients don’t know what they can or should look at. It’s our job to help them think about the bigger picture that data can provide.

    newslettersignup-promo

    AUTHOR

    Tu Nguyen-Brown

    Tu manages analytical design implementation to meet our client’s digital analytics + intelligence needs. She is an analytics professional with more than five years of experience using data-driven analysis to guide clients in optimizing their site and increasing revenue. Previous to smart panda labs, Tu was a Senior Analyst for YP (formerly AT&T Interactive), where she was responsible for deep-dive analysis across multiple platforms and analytics project management on yellowpages.com, m.yp.com and YPMobile apps (phones, tablets) and at Walt Disney Parks and Resorts Online, where she was lead analyst for several Disney sites including disneyaulani.com, and hongkongdisneyland.com. She is an Adobe Analytics: Reporting and Analytics Certified Expert and is Google Analytics Certified.

    All stories by: Tu Nguyen-Brown
    1 comment

    Leave a Reply

    Your email address will not be published.