The Measurement Minute

The Measurement Minute by Gary Angel

 

Data doesn’t always create new understanding and new understanding doesn’t always generate new action. It’s all too common to get interesting data but simply not see a straightforward path to using it to change anything.

 

 

In-Store Display Analytics is a rare exception. We’ve found in display analytics that the path from data to understanding to action is remarkably straightforward and this Measurement Minute highlights a real-world case study to show exactly how easy it can be to get significant results.

 

 

The Measurement Minute by Gary Angel

 

Adding product interaction and take-away metrics for display measurement creates something close a full-funnel analysis. When you know Traffic (Opportunity), Time (Engagement), Interactions (Interest), and Take-Aways (Conversion), you have pretty much everything you need to drive effective display and visual merchandising.

 

 

 

The Measurement Minute by Gary Angel

 

Using PoS to measure display performance is surprisingly hard. Beginning of season rollouts make comparability a challenge. And the goal isn’t so much to measure product lift as comparative lift – meaning comparisons often need to be made across years of essentially non-comparable data. That just doesn’t work well. Measuring shopper behavior at display turns out be a better method in almost every respect.

 

 

 

The Measurement Minute with Gary Angel

 

2019 was a busy year for new product development at Digital Mortar and we have three new products coming out this quarter. The most interesting analytically is targeted toward display analytics – helping merchants understand in-store display effectiveness. It’s one of those case where basic measurement is both interesting and immediately actionable.

 

 

The Measurement Minute with Gary Angel

 

The easiest way to create a biased model is often to just use the data you have. Census data, in particular, is a common culprit. It’s free, well organized and incredibly robust. But neighborhood turns out to be a nearly perfect proxy for race and age (not so much for gender). But the obligation to find screening variables is muddy and hard to clarify. How much effort is required? What’s acceptable and what isn’t? Damn if I know…

 

 

The Measurement Minute by Gary Angel

 

You can’t use a protected variable in a model and you shouldn’t use a proxy for it either. But how do you decide what’s a proxy and what isn’t given that age, race and gender are massively correlated to a wide range of likely predictive variables? The answer lies in a generally useful concept in data science: screening off.

 

 

 

 

The Measurement Minute by Gary Angel

 

If a model didn’t use gender as a variable, it can’t be gender-biased…right? Totally wrong. Because correlations to protected variables are everywhere in our world, it’s not only easy for an analyst to duck around restrictions on using protected variables, it’s actually pretty hard to avoid doing it.

 

 

The Measurement Minute by Gary Angel

 

Most people probably think it’s a slam dunk to decide if a model is biased against a protected class. It isn’t. Because age, gender and race are deeply embedded in our culture (and at least with the first two) our biology, they tend to correlate with all sorts of variables. Given those ubiquitous correlations, deciding what variable you can or can’t use is far from trivial.

 

 

The Measurement Minute by Gary Angel

 

If Views is overly tactical and encourages click-bait content, what’s the right metric to replace it? Here are three strategies (all with relevance in other areas) that can help: measure consumption by target audience (which implies measuring audience), focus on subsequent consumption lift, and use target audience as a denominator to create share metrics.

 

 

The Measurement Minute by Gary Angel

 

Game-ability isn’t the only way metrics can go wrong. Sometimes, a metric’s problems are revealed when it accomplishes exactly what you intended with it. Content Consumption metrics are a good example of a metric that – while seeming to drive the intended behavior – can actually steer a business wildly off course.