Why Retail Sales Models Still Require a Human Touch

American retail is back in growth mode. But as executives make critical real estate decisions, they often face an uneasy question — whether to trust their instincts or go with the recommendations of today’s increasingly prevalent computer models.

The dilemma is more pointed now that so many tech vendors offer modeling that is essentially a mystery-cloaked, “black box” process. Typically, the vendor leans heavily on terms such as “AI,” “machine-learning,” “sophisticated algorithms” or some combination of the three. Once the store recommendations are unveiled, the vendor cannot say precisely how the model arrived at these results.

Examples from the field illustrate the enormous benefits of relying on human expertise to ask hard questions, and thereby dramatically improve, retail sales models.

A strange result
In running a regression model on behalf of one of our retail clients, our analyst noticed an amusing correlation: Stores in areas with growing populations appeared to be associated with lower sales. It was a nonsensical result, and the analyst immediately suspected the model was picking up a proxy for something else.

The retailer had tentatively expanded outside its home base in the Northeast, where its stores performed well despite shrinking populations. The scattered stores in the retailer’s new high-growth regions lacked the name-recognition, operational consistency, and distributional support of stores in its more penetrated Northeast markets.

It was these latter factors that were responsible for those lower sales volumes. Lacking the intuition of an analyst or real estate executive, a black box could miss this distinction and encourage the retailer to steer clear of high-growth markets —terrible advice.

Missing information could be just as important. Many retailers now operate an array of store prototypes with different sizes and configurations, which certainly affects sales volumes from store to store. Likewise, a specialty retailer may sell popular merchandise lines at some stores but not at others. And the presence or absence of drive-through windows or catering will affect performance at different restaurant locations, too. On multiple occasions, we have seen retailers rely on legacy models that omitted such important store classifiers. Once we corrected for that, these clients were able to pursue previously overlooked growth opportunities.

The ‘overfitting’ problem
Analysts typically feed square footages, sales volumes, and various market attributes into their real estate models. With enough data about current and past performance, they can easily produce models that describe the factors driving sales variance among these locations with high accuracy.

This does not mean the model can predict performance at wholly new locations — not yet.

Good analysts use a raft of methodologies to stress-test their projections. For example, in creating the model the analyst may deliberately leave out data from a subset of stores in the portfolio. For sake of explication, call it a subset of 200 out of a total of 800 stores.

Having created the model with data from only those 600 stores, the analyst can now apply it to that excluded subset of 200 locations. If the model is accurate, it’s a great sign.

But analysts and their clients can stress-test models in other ways as well. Let’s say that the bottom 10 percent of the retailer’s stores post $1.5 million in average annual sales, the top 10% do $4 million, and the other locations fall somewhere in between.

If a sales-projection model for prospective sites yields a distribution that is wildly different from this — something like 15% of those potential stores yielding $6 million in annual sales—the model clearly lacks predictive power. That means it’s time to either reexamine every component of it or scrap it altogether.

When it comes to retail sales projections, details matter. Retail executives should interrogate modelers about the stress-testing methodologies they employ. Don’t be satisfied if the answer is something like “We’re highly confident in the accuracy of our analytics engine. It is cutting edge.”  

Human intel
Store decisions are high stakes. Creating the model, applying it, producing the pro forma and then making specific store decisions are only the first steps. Next comes negotiating and signing leases, completing the buildout, conducting the grand opening, and then waiting several months for sales to normalize after the “honeymoon phase” in that new market. All told, it could take up to a year until the company is finally able to judge whether the model truly was predictive.

The best way to ferret out bias in retail sales projections is to collaborate closely with those model-builders from the outset. It’s not about blindly feeding as much data as possible into an analytics engine; it’s about using an intelligent and informed conversation as the foundation for further analytical work.

A good analytical team will want to meet with c-suite executives to gain a solid understanding of their real estate goals, ideal demographics, target markets and more. That conversation should then continue with detailed drilldowns involving the finance and real estate groups.

The team should ask for the retailer’s past analyses and seek to understand the assumptions that went into them. And just as retail executives should be willing to question their analysts’ assumptions, the analytical team should ask hard questions of the chain’s operating assumptions. A deep dive could show that the company needs to rethink longstanding views about its best customers and markets.  

When both sides ask the right questions, the model is far likelier to be powerfully predictive. It’s a balanced approach that honors retail executives’ gut instincts—born of years of experience—while taking advantage of insights arising out of properly conducted data analytics.

Such transparent approaches can cast light on distortions that otherwise would have stayed hidden in that proverbial black box.