1. Introduction: What is demand forecasting, and how is it done?
Demand forecasting is, in essence, developing the best possible understanding of future demand. In practice, this means analyzing the impact of a range of variables that affect demand—from historical demand patterns to internal business decisions and external factors—to increase the accuracy of these predictions. Accurate demand forecasts can be leveraged throughout your supply chain to improve decision-making and outcomes in functions such as store and distribution center replenishment, capacity planning, and resource planning.
Demand forecasts can be developed on different levels of granularity—monthly, weekly, daily, or even hourly—to support various planning processes and business decisions, but highly granular forecasts are always extremely valuable. The benefits of a granular forecast are apparent when thinking of fresh food products whose short shelf lives sometimes call for intra-day forecasts at the product-location level to prevent spoilage.
For CPG companies, detailed forecasts allow planners to model the effects on demand based on sales or promotional information. Another critical benefit of granular forecasting lies in its ability to enable a manufacturer to aggregate data on products that share a particular raw material and create a forecast that will reveal the actual amount of materials needed to produce each product, helping them to reduce costs by optimizing bulk purchasing and reducing the number of shipments required. Additionally, modeling can help optimize production capacity.
Why, then, would slow-moving items that sell only a couple of units per location per day, if that, require the same level of forecast granularity? Even if the day-product-location level forecast for a slow-moving item is itself somewhat inaccurate, a planning system that can analyze data from multiple product locations simultaneously will help identify effects in scarce data situations. You need a system that combines this granular, product-location-day level sales forecasting data with the ability to automatically identify and apply sales patterns across multiple stores within a region or for a product category in a specific store or sales channel.
To effectively execute replenishment, capacity planning, and other business decisions, retailers and CPG companies need multiple forecasts with different levels of granularity that look at different time spans. This is why flexible data pooling across products or over different planning horizons is critical to a company’s ability to leverage the same demand forecast in all their supply chain planning.
Consider planograms, for example, which are not adjusted daily but are generally revised every few months as part of a more extensive assortment review process, with smaller adjustments often made between review periods. While many retailers still base planograms on past sales examined in weekly or monthly time buckets, demand-driven planograms offer a much higher degree of accuracy because they take advantage of this highly granular, day-product-location level forecast data. The same applies to markdown optimization, workforce shift optimization, and any other planning process with a more extended period than daily replenishment.
For CPG companies, granular forecasts allow them to automatically leverage retail sales and assortment data for more accurate short-term forecasts. A forecasting system that uses machine learning can calculate complex and variable demand drivers automatically, allowing manufacturers to quickly adapt their forecasts to changes in demand patterns at the right level of granularity. With automated and centralized assortment reviews, CPG companies can see marked benefits, including more than 90% weekly forecast accuracy, a 9-pp improvement in peak season forecast accuracy, and a 10% increase in forecast accuracy when using retailer data.
Despite the knock-on benefits of accurate demand forecasts throughout retail and CPG operations, many companies’ forecasting capabilities are, unfortunately, still quite limited.
1.1 What are the benefits of accurate demand forecasting?
Reliable demand forecasts are integral to the health of your supply chain because, simply put, they reduce uncertainty. With an accurate calculation of how many items they will sell at any given time, retail and CPG companies can order, allocate, and replenish those items accordingly. Beyond supply chain and inventory management, an accurate forecast in a unified supply chain and retail planning platform aligns sales and operations, supply planning, and raw material replenishment. In short, the demand forecast is the foundation from which retailers and CPG companies can drive a wide range of benefits across their business functions.
Benefits of accurate demand forecasting in the supply chain:
- Increased sales from better product availability.
- Reduced spoilage and fresher, more appealing products through more accurate stock allocation.
- Increased inventory turnover through reduced need for safety stock.
- Better margins through proactive, optimized markdowns.
- Improved capacity utilization and more reliable fulfillment through better visibility into capacity requirements and proactive bottleneck resolution.
- Reduced personnel costs through forecast-based shift optimization in stores and distribution centers.
1.2 How do retailers and CPG companies forecast demand?
From omnichannel giants to the smallest local manufacturers, all retailers and CPG companies rely on demand forecasts to order items based on their best estimate of how much they will sell. Modern demand forecasting is a sophisticated statistical analysis considering numerous variables to optimize that prediction.
While some retailers and CPG companies still rely on spreadsheets and manual calculations, such high-powered statistical analysis is best executed by specialized software designed to process enormous supply chain data sets. The best software transparently shows users what data is being used to build forecasts and how the forecasts are being calculated. Modern demand forecasting software automates difficult and time-consuming decisions, using machine learning to optimize predictions.
2. Demand forecasting methodology: How to create your forecast
Traditionally, retailers and CPG companies have created baseline forecasts using time-series modeling, looking at historical data to predict future demand. These forecasts were often adjusted based on causal modeling and manual input. Modern companies, though, have replaced these older approaches to demand forecasting with machine learning.
2.1 Machine learning improves demand forecast accuracy
Machine learning not only increases the accuracy of demand forecasts but also automates large amounts of planner work and can process enormous data sets—far more than any human planner would be capable of.
To generate an accurate demand forecast, a system must be able to process an enormous amount of data on the wide range of variables that can potentially impact demand. With advancements in large-scale data processing and in-memory computing, modern demand planning systems can make millions of forecast calculations within a minute, considering more variables than ever possible.
Consider the three broad areas of variability that continuously impact demand: recurring variations in baseline demand patterns, internal business decisions, and external factors such as weather or local events.
Demand forecasting software must be transparent about the models it’s using to calculate forecasts using all this data. Transparency, after all, is critical to retail and CPG planners’ ability to understand and trust their automated forecasts fully.
For a more in-depth discussion of different approaches to forecasting, take a look at The Complete Guide to Machine Learning in Retail Demand Forecasting.
2.2 Predicting the effects of business decisions: Promotions, price changes, product introductions, and more
Commercial choices, such as promotions, price changes, or new product introductions, significantly impact sales volumes. Because these decisions can introduce so much hard-to-predict variation, they absolutely must be accounted for in forecast calculations.
To predict the impact of business decisions, you must leverage machine learning algorithms that can process large amounts of data and integrate them into the baseline demand forecast to be accounted for.
The variables most commonly included in these calculations include:
- Promotion types, such as price reduction or multi-buy.
- Marketing activities like newspaper ads or in-store signage.
- In-store display, such as when retailers present a promoted product on an endcap or table or when CPG companies provide a product-specific display to retailers.
- Price elasticity, or how a change in price impacts a product’s demand.
- Visibility, such as when a CPG company wins or loses placement in a grocery store.
- Impact of a price change on other products within that category.
Accurate price elasticity modeling is especially important for markdown optimization, as it provides planners with a clear picture of how to price markdown stock to sell quickly while maintaining the highest possible margin.
To learn more about how to accurately take the impact of commercial choices into account, read our white paper, More Accurate Promotion Forecasting with Machine Learning.
2.2.1 Accounting for price changes
When you lower one product’s price, you usually also reduce demand for other products in that category as demand shifts to the cheaper product—a phenomenon known as “cannibalization.” To avoid over-ordering cannibalized products, retailers should adjust forecasts for these non-promoted products down and then incorporate the revised forecasts into their replenishment planning. This level of accuracy, of course, is especially relevant when replenishing products with short shelf-life.
However, for factors outside your organization’s control, effective supply chain collaboration can ensure your demand forecasting software will accurately account for price changes impacting demand. When retailers and CPG companies establish a regular data-sharing routine, it allows both companies visibility into their end-to-end supply chain, which can help mitigate a broad range of issues that arise from unexpected changes to demand signals.
2.2.2 New product introductions
New product introductions present a different kind of challenge to sales forecasts because you don’t have the luxury of historical sales data to serve as a foundation. Reference products with historical data serve as a blueprint until you’ve gathered enough data to create an actual forecast for the new product. Typically, in product introduction scenarios, planners must choose reference products themselves—a time-consuming process that is also often inaccurate when considering wide assortments and high renewal rates.
A far more efficient and accurate way to solve this problem is to utilize a planning system capable of automatically selecting the appropriate reference product based on the most relevant attributes in the product category (brand, size, use, color, flavor, etc.). Of course, this system should also be able to quickly update its SKU-store/channel forecasts as actual sales patterns for the new product emerge.
2.3 External factors that impact demand
When we say “external factors,” we’re referring broadly to anything not in your power to decide as a retailer or CPG company. Examples of external factors might include weather forecasts, local events, or even your competitors’ business decisions, all of which can cause significant changes to your demand.
We know, for example, that a heatwave almost always boosts ice cream sales and, conversely, that the first snowfall of the year sends customers flocking to buy winter coats. That seems straightforward, but identifying and accurately predicting all demand shifts for all external events across the entire product range at multiple locations is enormously complex.
To demonstrate how impossible this task would be for human calculation, let’s consider a small retailer with 100 stores and a range of 5,000 products. If their planners were to attempt to manually account for the effects of weather alone on a reasonably granular level, they would have to examine some 280 million potential relationships between variables (100 stores x 5,000 products x 20 weather variables x 7 weekdays x 4 seasons). The same logic applies to CPG companies, which tend to have fewer locations and smaller product ranges but can serve many different retail channels. The data quickly becomes impossible for any team of humans to compute.
Fortunately, machine learning automates a large portion of this work and can integrate these external factors into your forecast. These algorithms can secure your ice cream supply just before a heatwave rolls through or reduce your supply before a torrential rain settles in for the week. In our experience, incorporating weather can reduce forecast errors by 5–15% on the product level and by up to 40% on the product group and location level.
Retailer’s commercial decisions on store assortments, including product displays, promotions, and in-store events, impact demand at the store level, which in turn impacts demand from the CPG company. With efficient supply chain collaboration, CPG companies can leverage retailer data in their own demand planning to attain high forecast accuracy and meet demand in every situation.
When external data is considered, demand planning software can give planners a clear understanding of how different factors impact the forecast—for example, the effect of local weather on sales. Demand forecast accuracy depends on a system’s ability to incorporate a wide range of potential external data sources, including but by no means limited to:
- Local weather data and weather forecasts.
- Passenger number data and passenger forecasts for transportation hubs (read the case study).
- Data about past and upcoming events, such as fairs or concerts.
- Competitor pricing data.
- For CPGs, retailer data, like POS, retailer promotion and assortment, including store count.
While machine learning in demand forecasting helps automate the bulk of the work, it has limitations. After all, consumer trends are constantly changing, and the unexpected always happens when you least expect it. There will always be a risk that forecasts reflect how things happened in the past instead of how things will actually be. That’s why machine learning can never replace human expertise and experience. We will always need demand planners to observe and understand real-world changes and correct the automated forecasts accordingly.
We’ve now built a highly automated demand forecast leveraging machine learning to develop an accurate baseline forecast that identifies recurring demand patterns, incorporates promotions and other internal business decisions, and then factors in external data such as local events and competitor pricing.
Next, we’ll look at omnichannel operations, one of the most disruptive developments in the supply chain, and certainly in demand forecasting.
3. Demand forecasting in omnichannel
Retailers and CPG companies who execute an omnichannel strategy must deliver a good customer experience in every channel, whether in-store, online, or through hybrid channels like click-and-collect. From product availability to fulfillment speed, customer experience relies heavily on a retailer’s supply chain efficiency and visibility.
To successfully forecast demand across multiple channels, companies must link online sales to the correct fulfillment channel. For example, suppose your online orders get picked from your local stores. This online demand must be incorporated into the store-level demand forecast to ensure accurate replenishment that meets both online and in-store demand. But that’s just the beginning.
Online orders tend to follow a different sales pattern than brick-and-mortar sales. There are many reasons behind this difference — for example, price comparison is far faster and easier online than when shopping at a physical store. Holiday seasons are another variation example; retailers and CPG companies usually see online orders placed well ahead of time, followed by a wave of procrastinators rushing to brick-and-mortar stores for last-minute purchases. Of course, this pattern may change as consumers expect faster online fulfillment, even during rushed holiday seasons.
Because demand patterns can vary significantly by channel, demand planning systems must be able to separate the forecasts for online and in-store sales, adding even more granularity. These forecasts can be used to enable virtual ringfencing at warehouses and distribution centers, ensuring availability across channels.
Omnichannel retailers and CPG companies must be able to forecast by store, by sales channel, and by fulfillment channel to ensure the right stock is available in the right places and maintain customer satisfaction across the board.
4. Demand forecast accuracy
No matter how advanced technology grows, predicting demand with 100% accuracy will always be impossible. That’s why supply chain planning processes must always be able to accommodate a certain degree of uncertainty. But how accurate do you need your forecasts to be?
Ultimately, accuracy is always important but should be analyzed on different levels. Depending on the use case, forecast accuracy should be evaluated for different periods and levels of aggregation. For example, when replenishing quick-selling, short shelf-life perishable products, highly accurate day-product-location level forecasts are imperative.
For slow-moving products, on the other hand, it’s more important to get the total volume fulfilled by a specific DC correct and to avoid bias. No matter what your business goals or assortment characteristics, though, there’s always a point of diminishing returns on forecast accuracy.
All retailers and CPG companies reach a plateau where it becomes more cost-effective to accept and prepare for a certain degree of inaccuracy than investing more time and money on further improvements. For example, if a product has a long shelf life, a slight safety stock increase may be less than the cost of asking demand planners to refine the forecast further.
Remember, too, that demand forecasting is just one part of the larger supply chain planning operation. Even near-perfect forecasts will fail to produce the desired results if the other parts of your planning process fall short. You’ll still end up with overly large batch sizes or too much presentation stock. For a full review of demand planning processes, consult our Forecast Accuracy Guide for more strategies to increase accuracy.
So, when is your forecast accurate enough? In short, further improvements in accuracy will only marginally improve your actual business results.
5. Picking a demand forecasting software
We’re not too humble to say that we believe our demand forecasting solution is a powerful asset for most companies with a sizeable continuous assortment, fresh products, and those with a wide product range offering direct-to-store deliveries. Our solution can improve end-to-end supply chain collaboration for retailers and CPG companies.
Whatever solution you pick, ensure it leverages a complete AI toolbox to capture the complexity of demand-influencing factors from all available internal and external data sources. The only way to calculate a good forecast is by feeding your demand planning system with enough relevant data. The data required to make accurate predictions is far too heavy and complex for humans to calculate, so retailers and CPG companies must look for an AI-driven solution whose machine learning algorithms can automate most of that work.
It’s essential, though, that this complexity doesn’t result in a “black box” of a system that forces planners to simply trust automated recommendations. The best demand forecasting systems are transparent. They provide planners with a clear understanding of how they calculate forecasts to support a wide range of short and long-term planning activities—from replenishment planning to workforce optimization and capacity management.
Furthermore, the demand forecasting system must be powerful enough to process data at scale. Both retailers and CPG companies generate enormous amounts of sales data every single day. Add to that the data from internal business decisions and external factors, and the level of granularity required for accurate planning and the importance of data processing power becomes apparent. Choose a solution with a fast, modern database, or you’ll end up waiting hours every time you need to run a calculation.
If a solution doesn’t satisfy these basic requirements, move on to the next one. To read in more detail about evaluating and choosing the demand forecasting software that will best serve your business needs, read Supply Chain Transformation: The Complete Guide.
Demand Forecasting FAQ
1. What is demand forecasting?
Demand forecasting is developing the best possible understanding of future demand by analyzing the impact of various variables that affect demand. These variables include historical demand patterns, internal business decisions, and external factors.
Accurate demand forecasts can be used throughout the supply chain to bolster decision-making and outcomes in store and distribution center replenishment, capacity planning, resource planning, and more. These forecasts can be developed at different levels of granularity (monthly, weekly, daily, or even hourly) to support various planning processes and business decisions.
2. How do you improve the accuracy of your demand forecasts?
Demand forecast accuracy is crucial for retailers and CPG companies as it helps optimize inventory levels, reduce stockouts and overstocks, improve customer satisfaction, and enhance supply chain efficiency. Accurate forecasting enables better decision-making, cost reduction, and increased profitability.
To improve the accuracy of demand forecasts, companies should consider:
- Using machine learning algorithms to analyze large amounts of data and identify patterns that human planners may not see.
- Incorporating internal business decisions, such as promotions, price changes, and product introductions, into your forecasts.
- Factoring in external factors that can impact demand, such as weather, local events, and competitor pricing.
- Leveraging granular data and flexible aggregation to create forecasts that support a wide range of planning processes and planning horizons.
- Continuously evaluating and adjusting their forecasting models based on real-world changes and feedback from demand planners.
3. What are the most essential features to seek in demand forecasting software?
The best software for retailers and consumer goods companies will differ depending on their business goals. However, several features remain a “must” for any company serious about creating accurate demand forecasts, including:
- AI-driven machine learning algorithms that can process large amounts of data and automate complex calculations.
- Transparency into forecasts, allowing planners to understand and trust automated forecasts.
- Scalability and processing power to handle large data sets and provide fast calculations.
- The flexibility to create forecasts at different levels of granularity and support various planning processes and business decisions.
- Integration with other supply chain planning tools and systems, enabling end-to-end visibility and collaboration.
- The ability to incorporate both internal and external data sources, such as promotions, weather, local events, and competitor pricing, to improve forecast accuracy.