Understanding ‘forecastability’ and how to generate better forecasts

Getting better statistical forecasts involves choosing the right technology and applying multiple forecast optimization techniques and a continuous improvement approach. Has your demand planning organization adopted these methods?

This blog post is the second in a five-part series on improving statistical forecasting. This post will cover how to assess the forecastability of your product or customer portfolio and use different techniques to improve forecast quality.

What's “forecastability,” and why is it essential for statistical forecasts? 

It’s crucial to determine where to focus your limited forecasting capacity among the multiple products and customers in your portfolio — particularly your demand planning and sales/marketing resources. Understanding the forecastability of your portfolio makes this easier.

Forecastability is a quantitative metric computed to assess:

• The importance of revenue (depending on your portfolio, you can start with volume instead)
• The ease of the forecasting (demand characteristics like variability, lifecycle stage, intermittency and pattern) of your portfolio

At NTT DATA, we often use a 2×2 matrix to assess the forecastability of a portfolio.

However, we're intentionally cautious when measuring variability in this context; we exclude “explainable” variability (for example, seasonality/trend) and only include non-explainable variability when calculating forecastability.

Once you calculate your portfolio’s forecastability, you can use it to make important decisions that help with statistical forecasting.

1. Segment your portfolio by forecastability to maximize your return on invested effort in statistical forecasting. For example, for quadrant 1 (Q1) and Q2 in the forecastability matrix (quantity vs. timing predictability) above. However, other segments call for an alternative planning strategy. For example, Q4 is where you should require your demand planners to provide enrichment basis inputs from sales/marketing/product management and attempt to manage Q3 through replenishment/inventory policies.
2. Establish the lowest level in the hierarchy at which you should forecast. For example, what level of product or customer gives the best balance of forecast quality versus planning processes into which you want to feed the forecast. So, for instance, define your forecast hierarchy to support discussions about product personalization — or customer-specific promotional activities — where relevant. Elsewhere, use a higher level of aggregation for your forecasts.
3. Determine what the time granularity of your forecasts should be and the proper forecast offset (lag). For example, for Q3 products or customers, is it necessary to generate weekly forecasts — especially when the timing of demand is difficult to predict? Or is it better to do monthly or even quarterly rolling forecasts? Similarly, determine the offset of your forecasts by the frozen period of the supply chain process you’re trying to forecast.
4. Choose statistical forecasting algorithms correctly based on the forecast profile. For example, exponential smoothing methods may perform better earlier in the yearly cycle. In contrast, you can manage a Q3 product more pragmatically using a combination of simple forecast assumptions (for example, moving averages, the Croston model) and inventory policies.

So, how do you create better statistical forecasts? 

Better statistical forecasting often equates to better algorithm selection. But that’s not always the case. Better algorithm selection is only one of multiple strategies you can use to improve your forecasts.

Here are some helpful strategies for improving your forecast quality:

1. Find the right planning level to run your stat forecasts. Companies often choose to create statistical forecasts at the lowest level (for example, SKU-customer-week). The rationale is, “I need to generate my forecast at the same level where the forecast needs to be consumed.” Alas, this isn’t true. Running statistical forecasts at a higher level of aggregation often reduces noise and improves your ability to detect trends and seasonality. Then, using other techniques, you can decompose the aggregate forecast to the lowest level needed. Having the flexibility to create statistical forecasts at different levels of hierarchy, or even based on specific product attributes, can likely improve forecast quality.
2. Improve the data quality of time series. Statistical forecasts rely mainly on historical data. This data could be actual orders complemented by internal (historical promotion, historical supply, and so on) and external (weather or socio-economic) indicators. The quality of historical data bears directly on the quality of the forecast. It’s critical to identify historical data issues, such as data holes and unexplained upsides and downsides, and address those data issues. For example, classifying bulk orders into a separate time series for the same SKU/customer is effective in better forecasting the run-rate orders’ time series. Selecting the right outlier clean-up approach before running a statistical forecast algorithm on that time series is also key to improving data quality.
3. Leverage multiple back-testing periods. Companies often test statistical forecasts generated through various algorithms in a single back-test period. However, if you use a best-fit approach to pick the best algorithm for use, it can cause over-fitting. A better approach is to test each of your statistical forecasting approaches over multiple simulation periods. For example, if you’re generating a one-month lag forecast, generate it for six one-month periods using a different in-sample period that advances by one month progressively. By conducting multiple back tests, you can see which statistical forecasting approach will consistently perform better across various test periods. It’s also essential to determine what qualifies as a “better” forecast. Let’s say your supply chain is continually running out of raw material. Look at the forecast bias and the Weighted Mean Absolute Percentage Error (WMAPE) to determine what that better forecast looks like.
4. Choose your algorithms. Some forecast segments/profiles respond better to certain classes of statistical forecasting algorithms. This approach involves hierarchical selection by examining the accuracy of multiple algorithms for a segment/profile. Short-listing the winners, and then doing additional simulations by tweaking other parameters of those algorithms, will give better results.
5. Seek transparency and understand statistical forecast behavior. The aim of a demand planning organization is to improve the quality of the consensus forecast. Many demand planners have no visibility into why the baseline statistical forecast is the way it is. If demand planners don't trust the statistical baseline forecast, the chances of their overriding the forecast are higher.

Using simple dashboards helps demand planners visualize historical time series versus the forecasted data points. It’ll help you understand why the statistical forecast is how it is, and build confidence in the forecast. Dashboards also help demand planners understand where and where not to apply their own business insights-based enrichment.

How do you prevent your forecast quality from deteriorating over time? 

Setting up statistical forecasting models during the implementation phase and expecting them to work over the long-term is the most common mistake in traditional statistical forecasting implementations. Additionally, with product and channel proliferation across most businesses — the portfolio of products and customers for which you’re trying to forecast is ever-changing. That means your portfolio’s forecastability also changes. So, it’s necessary to your review your forecastability-based segmentation and refreshing your forecasting strategies for the segments to maintain the forecast quality on  which you’ve come to depend on.

It’s also crucial to remember that improving forecast accuracy is a journey. Demand planning must set targets for forecast accuracy and you must strive to improve your forecasting process. Create a roadmap of improvement initiatives in your forecasting process by: 

• Finding better ways to improve the quality of the time series in your forecasting process 
• Augmenting your data feeds with external indicators 
• Identifying which parts of sales enrichment to incorporate into the statistical forecasting process
• Improving current forecasting strategies and adopting new ones, such as disaggregation and outlier clean-up techniques