Improved financial planning thanks to new Demand Forecast model

Financial planning enables us to check the viability of our strategy and set realistic targets. Based on this, we decide, for example, when new warehouse space is needed, how much in fixed costs we can build up or how much liquidity we require. In our efforts to continuously improve our processes, we’ve developed the Demand Forecast from the ground up to be the most important driver of financial planning, contributing to an increase in our planning quality and efficiency.

Our Demand Forecast predicts daily customer demand in units per product group, customer type, business model and country of sale. From this, our financial model derives metrics such as turnover, contribution margin, number of deliveries or returns. In logistics, these daily quantities are an indicator for personnel planning. All of these drivers depend directly or indirectly on the Demand Forecast, which revealed known weaknesses. For example, Saturdays were systematically overestimated and Black Friday Week was underestimated. Long-term trends in the development of product groups deviated from the expectations of our category management, in some cases by double-digit percentages. Furthermore, demand shocks couldn’t be easily recognised and excluded from model training. This led, for example, to the unnecessary growth in disinfectant stock (after the outbreak of the Covid pandemic) as well as power storage systems and drones (after the outbreak of the war in Ukraine).

We’ve made numerous adjustments to avoid these known flaws. While we originally estimated all time scales with the same Meta-developed model (Prophet), we’re now deciding in a first step which type of model will be best suited for predicting an underlying scale. Based on the distribution of the data, we estimate the trend and seasonalities separately or allow the model to include days on which not a single item is sold. We currently distinguish between five model types and can extend these as required in the future as soon as we realise that a data pattern previously unknown to us can’t be correctly modelled by any of the existing types. Given the variety of different time series, this is also urgently necessary. For example, we don’t sell Advent calendars in summer, but we do sell garden lounges and car accessories. We sell a steadily growing number of diapers and Oatly oat milk. Corporate customers don’t usually order at the weekend, but for private customers Sunday is often the day of the week with the strongest demand.

Numerous innovations lead to significantly improved demand model

Separately, the question arose as to which granularity we can best estimate demand at and which is most suitable for our financial model. In one extreme case, at product level, we’d have to generate a prediction for several million timescales. In the other, aggregated across all products, we’d lose many product-specific properties. After various tests, we’ve found a middle ground. We can both remain sufficiently granular to not lose the seasonal characteristics of product groups, while simultaneously having enough data available for a reliable forecast. This led to a significantly smaller number of time scales for which demand had previously been forecast. Thanks to this reduction, our monitoring can also be carried out with less effort. This means we can now implement individual adjustments, such as a shorter training period or a model parameter configuration that differs from the model type, with reasonable effort.

While in the past it was always a major challenge to make an accurate forecast for Black Friday Week as well as the week before and after Christmas, these additional models can now forecast both periods with comparable accuracy compared to the rest of the year. On the one hand, the effect of this interplay between weekday and holiday during Christmas was difficult for a model to learn (if 23 December falls on a Saturday, no parcels are delivered on 24 December), since this specific event only occurs once or twice at most in the historical data. On the other hand, Black Friday Week was introduced two years ago to better distribute the peak of Black Friday over a week and thus relieve the burden on logistics. We solved the first problem by training separate models for the holidays, which learn from data across several time scales simultaneously and thus see the combination of weekday and holiday several times in the training data. We were able to fix the latter by training another additional model to make a forecast for all the demand during Black Friday Week, then distributing it sensibly over the individual days of the week.

Was the effort worth it? The chart above provides some insight – it shows the aggregated forecasts for Switzerland. To enable a fair comparison, we trained the previous and the new model with the same data until the beginning of November 2023 and generated a forecast for the following 12 months. This corresponds exactly to the budget planning process. A comparison shows how the blue line (the new forecast) follows the black line (the actual values) almost exactly. Weekly seasonality, Black Friday week, Christmas and the general trend have all improved. While Saturdays were still constantly overestimated by the previous prediction model (recognisable by the green, positive vertical bars at regular intervals), no systematic error can be seen in the blue, positive vertical bars (the residuals).

This improvement is also reflected in the metrics: the average percentage error improved from 16 per cent to seven per cent across the entire one-year evaluation period. On a more technical point, the R-Squared metric, which describes what proportion of the variance for the target variable can be explained, was improved from 34 per cent to over 90 per cent.

Better predictions on all product hierarchies and their limits

Ultimately, as in the chart above, the aggregated forecast is made up of numerous forecasts at lower hierarchical levels. As a result, the accuracy of the prediction at the highest level depends directly on the accuracy of predictions lower down. The chart on the left below, for example, shows demand for the computing product group in Switzerland. Weekly seasonality is also modelled much more precisely here than before.

Nevertheless, the new model shouldn’t be regarded as an oracle. There are always things that a long-term forecasting model can’t know a year in advance. A good example of this is the forecast for the water sports product group. As can be seen in the graph on the right above, the actual demand for water sports products in July and August was underestimated quite a bit. The reason for this is fluctuating weather, which we can’t predict in advance. Our model assumes an average value over the last few years, which is clearly too low. Last but not least, marketing activities such as a voucher campaign or short-term adjustments to pricing can lead to the reality turning out differently than we had planned. Our demand model can’t know these factors a year in advance.

Overall, we can clearly confirm that the development of a new Demand Forecast model for financial planning has paid off. From today’s perspective, we can estimate the development of customer demand with a higher degree of reliability. This gives us confidence in our ambitious goals and the courage to continue taking «piratesque» advantage of opportunities. We don’t know everything yet, but we’re positioned to react quickly to changing conditions.

These articles might also interest you