4 Data Analytics Concepts Every Leader Should Understand

Data analytics is ultimately about making good decisions. It doesn’t matter what business you are in or what your role is at your company, we all want to — need to, really — make smart, informed, evidence-based decisions. Following are four concepts every leader should understand:

RANDOMIZED CONTROLLED EXPERIMENTS.

One of the first steps in any analysis is data gathering. This often happens via a spectrum of experiments that companies do — from quick, informal surveys, to pilot studies, field experiments and lab research. One of the more structured types is the randomized controlled experiment. Many people, when they hear this term, immediately think of costly clinical trials. But randomized controlled experiments don’t have to be costly or time consuming. They can be used to gather data on things like whether a particular customer service intervention improved customer retention or whether a new, more expensive piece of equipment is more effective than a less costly one.

A/B TESTING.

One of the more common experiments companies use these days is the A/B test (which is a type of randomized controlled experiment). At their most basic, these tests are a way to compare two versions of something to figure out which performs better. Companies use it to answer questions like, ‘What is most likely to make people click? Or buy our product? Or register with our site?’ A/B testing is used to evaluate everything from website design to online offers to headlines to product descriptions. It’s critical to understand how to interpret the results and to avoid common mistakes, like ending the experiment too soon before you have valid results or trying to look at a dashboard of metrics when you really should be focusing on a few.

REGRESSION ANALYSIS.

Once you have the data, regression analysis helps you make sense of it. Of course, there are many ways to analyze the data, but linear regression is one of the most important. It’s a way of mathematically sorting out whether there’s a relationship between two or more variables. For example, if you are in the business of selling umbrellas, you might want to know how many more items you sell on rainy days. Regression analysis can help you determine whether and how rain impacts sales. It answers the questions: Which factors matter most? Which can we ignore? How do those factors interact with each other? And, perhaps most importantly, how certain are we about all of these factors?

Fortunately, regression is not something you typically do on your own. There are statistics programs for that! But it’s still important to understand the math behind it and the types of mistakes to avoid. In this refresher, I explain how regression works and share a common — but often misunderstood — warning against confusing correlation with causation.

STATISTICAL SIGNIFICANCE.

Once you’ve done the analysis, you need to figure out what your results mean, if anything. This is where statistical significance comes in. This is a concept that is also often misunderstood and misused. And yet because more and more companies are relying on data to make critical business decisions, it’s an essential concept to understand. Statistical significance helps you quantify whether a result from an experiment is likely due to chance or from the factors you were measuring.