The march of the machines: predictive models have broken out of the supply chain and are heading straight into marketing territory
Every brand and vendor performs some kind of demand forecasting.
In the last few years, data science has gone massively mainstream, and new machine-learning techniques have dramatically improved the accuracy of forecasting in the supply chain. The impact has been pretty stark in retail, where a three per cent improvement in forecast accuracy translates to an average two per cent increase in profit margin for the retailer.
By putting groups of algorithms to work on forecasting’s knottiest problems, some of the most notoriously unpredictable demand patterns have been cracked – and that’s had an unexpected side-effect: some companies have started applying these forecasting techniques in novel and unexpected ways, particularly product marketing.
To understand why, we need to look at what’s previously been out of reach to forecasters. Until fairly recently, there were three major ‘unpredictables’ in the consumer packaged goods (CPG) market. These scenarios eluded statistical modelling techniques: new product introductions; price promotions; and extreme seasonality. New products are hard to model because there’s no historical sales data to work with- the underlying demand for products with complex and variable promotions history is hard to discern because of all the ‘noise’ in the data. Highly seasonal products (things like limited editions or products associated with a particular calendar event) have no knowable underlying demand pattern – sales are driven almost entirely by external factors. These things were the ‘known unknowns’, and crude demand estimation methods based on assumptions, heuristics and rules-of-thumb were the only ‘forecasting’ methods available for these when this happened.
As supply chains got slicker and data became more abundant, forecasters whittled their error rates down year-after-year by using ever-more sophisticated techniques. Suddenly educated guesswork just wasn’t good enough to meet the needs of these streamlined, optimised operations. At the same time contextual data – the raw fuel of algorithmic modelling – became publicly available via the web. It was this ambient, time-specific data – weather, important calendar and cultural events, and social media activity – which enabled the prediction of the unpredictable.
Looking back, it might seem obvious that the three problems cracked using data science are all marketing problems too. For the first time historical data can be used to build accurate computerised models of product launches, promotions planning, and campaigns… but it has taken time for data-driven practices to cross the chasm between business silos. In North America, one major big-box retailer has struggled for the past two years to develop its forecasting algorithm into a tool that their sales and marketing teams will accept and trust to drive decision-making.
In Asia, however, the culture shift is happening much more quickly, and the crucial difference is that progress is agency-driven. One Japanese full-service agency Hakuhodo has started implementing new product launch models into its offering for a major beer client, using clustering and regression algorithms on data from 20 years’ of product launches to simulate sales in the first few weeks after the new beer enters the market. The model incorporates promotional media placements across channels so the client can understand the impact of their advertising spend, and how to spend more effectively. The same agency is building a drag-and-drop web interface to help clients plan their media spending throughout the year. It won’t be long before the two halves meet in the middle - models driving client spending on agency services through campaign planning across product launches, seasonal marketing and price promotions.
Forward-thinking agencies like this one have realised two things: Firstly, predictive modelling isn’t really about forecasting the future – it’s about making decisions in the face of uncertainty. Secondly, agencies are uniquely well-placed to offer these models, because they own such a massive chunk of the data needed to build them.
The machines are coming, and the march of progress is unstoppable. But agencies and marketers should not be afraid! Demand models are designed to answer the question, ‘what’s really driving my sales?’. It makes sense to take those models and put them firmly in the hands of the people who shape consumer demand for the product, and firmly establish the relationship between marketing spend and product sales.
Steve King is CEO and Co-Founder of Black Swan.