Method for verifying prediction accuracy and method for responding to prediction failure
TECH
January 15, 2025
This is some text inside of a div block.

Historical purchase forecasts were relatively simple. All we did was reflect a certain percentage of growth in the previous year's data and make slight adjustments based on market conditions, so there was no process to verify the accuracy of the forecast.

“This year, we set our budget based on data from last year and the year before. However, the market moved in a direction that was completely different from our predictions, resulting in a gap of more than 30% between the budget and the actual purchase amount.” This is a reality that many companies' purchasing teams are experiencing.

The disruption to global supply chains experienced in recent years has demonstrated the limitations of these traditional forecasting methods. The uncertainties faced by purchasing teams have never been greater, including rapid fluctuations in raw material prices, unpredictable logistics delays, and supply disruptions due to geopolitical risks.

In addition, the supply chain reorganization due to the tightening of ESG regulations, the shortened component life cycle due to the acceleration of technological innovation, and the resulting increase in inventory management complexity are creating a situation that is no longer possible to respond to with the existing empirical forecasting method.

Limitations of traditional forecasting methods

In this situation, the purchasing team is realizing the need for a more sophisticated forecasting methodology. It is necessary to go beyond simply extrapolating historical data and take a systematic approach that comprehensively considers various market variables and identifies potential risks in advance.

This includes the use of new technologies such as AI and big data analytics, but more importantly, it is the practical know-how to effectively utilize and verify these technologies.

In this article, we will look at the forecasting challenges faced by companies in this context and suggest practical measures to improve forecasting reliability.

In particular, we aim to systematically organize the measures to be taken when forecasting fails, so that stable purchasing operations can be carried out even in the current market environment with high uncertainty.

How should we judge the reliability of purchasing forecasts?

Criteria for forecasting accuracy by industry and actual cost impact

Forecast accuracy standards by industry and actual cost impact
Predictive value-added is a key concept for understanding the value that forecasting brings to an organization and whether there is room for improvement. It can play an important role in determining how to allocate budget for supply chain process improvement across different parts of the business and how to invest in demand forecasting and supply response capabilities. (Source: What's the Value of Demand Forecasting?)

In the manufacturing industry, the accuracy of purchase forecasting is directly linked to a company's profitability.

In the global automotive parts industry, companies with an average forecasting accuracy of 85% or higher had a 1.5 times higher inventory turnover rate than companies without such accuracy. In the electronics manufacturing industry, companies with a forecasting accuracy of 90% or higher had an average operating cost 12% lower than their competitors.

Improving forecast accuracy is not just a matter of numbers. Accurate forecasts create real value in many ways, including reducing inventory carrying costs, cutting emergency ordering costs, and optimizing production planning.

For example, for mid-sized manufacturing companies, every 5% improvement in forecast accuracy has been shown to result in an average 8% reduction in inventory costs.

The forecast accuracy required by each industry varies. In industries with long lead times and large fluctuations in raw material prices, such as semiconductors, achieving a prediction accuracy of 80% or higher is enough to secure a competitive edge.

On the other hand, the consumer goods industry requires a high accuracy of 95% or higher. Therefore, each company should set an appropriate prediction accuracy target in consideration of its industry characteristics and operating environment.

Key factors for verifying the reality of a prediction model

Predictive purification analysis process
Reliability Prediction Analysis Process (Source:The Reliability Prediction Analysis Process: A Best Practices Approach)

To determine the reliability of a predictive model, it is necessary to verify it from various angles. The first thing to look at is the consistency between the predicted value and the actual market trend.

No matter how sophisticated a predictive model is, if it fails to properly reflect the real market situation, its value will be limited.

In market trend analysis, various external factors such as macroeconomic indicators, industry-specific growth rates, and technological development trends must be considered comprehensively.

In particular, market changes due to the strengthening of ESG regulations, geopolitical risks, and the introduction of new technologies have been accelerating recently, so it is important to ensure that these factors are properly reflected in the forecasting model.

In addition, the volatility and trend of the forecast value should be closely examined. If there is a sudden change in the forecast value, there must be clear evidence to explain the change.

For example, for items with strong seasonality, it is necessary to check whether the past seasonal pattern and the current forecast are consistent.

Systematic response to forecast failure

Response strategy for demand underestimation scenario

The problem that occurs when demand is underestimated is more than just stock shortage, and it has a serious impact on the company's market competitiveness and customer trust.

In particular, a shortage of key parts in the manufacturing industry can lead to the shutdown of the entire production line, which has a very significant ripple effect.

Short-term responses to this situation include urgent orders and the use of expedited procurement methods such as air transportation. Even if costs increase, minimizing production disruptions should be the priority.

At the same time, a network of alternative suppliers should be quickly activated to make up for the shortage.

In the medium to long term, the safety stock level needs to be re-evaluated. At this time, it is necessary to establish a differentiated inventory policy that takes into account the importance of each item and the procurement lead time, rather than simply increasing the inventory level.

In addition, a strategy is needed to build a supply network distributed by region to spread out risks.

A systematic response plan for failure to predict

Deepflow predicts future demand by analyzing various variables such as past sales data, prices, energy costs, changes in supply and demand, and global economic trends. This allows companies to more accurately execute their ordering plans and improve inventory asset turnover.

In particular, the prediction accuracy can be improved by using advanced AI prediction models. We have independently developed more than 200 AI prediction models to provide global top-tier prediction performance.

Strategies for responding to demand overestimation scenarios

If demand is overestimated, the main problems that arise are increased storage costs due to excess inventory and worsened liquidity. In particular, for products with a shelf life or parts that undergo rapid technological changes, the value of inventory can drop rapidly, requiring a quick response.

First, a strategic approach is needed to deplete inventory. Various measures should be considered, such as promoting sales through promotions, securing alternative sales channels, and transferring inventory to other business units or overseas subsidiaries.

The key here is to consider the balance between the loss from disposing of inventory and the cost of storage.

In the long term, it is necessary to review the terms of the contracts with suppliers and switch to a flexible contract structure that allows for volume adjustments. In addition, the conservativeness of the forecasting model should be reviewed, and a system should be established to flexibly adjust the forecast according to market conditions.

A solution that can easily analyze prediction accuracy in accordance with the usability of the field

Deepflow is an AI-based demand forecasting system that can accurately predict future demand volumes to prevent overstocking. It can predict future demand by training a deep learning model based on various data such as historical sales data, market environment data, and product feature data.

If demand is overestimated, the production/order volume can be adjusted by referring to the prediction results of Deepflow to prevent overstocking.

Deepflow is a solution that goes beyond simply providing an inventory management system and collaborates with users to solve inventory management problems. Users can refer to Deepflow's AI prediction results to establish and execute inventory management plans, and Deepflow continuously improves its AI prediction model by reflecting user feedback.

Market Price Fluctuation Prediction Failure Response Strategy

Rapid fluctuations in raw material prices have a direct impact on a company's profitability. In particular, price volatility is increasing in situations where there is high uncertainty in the global supply chain, such as in recent times.

In such situations, a multi-layered response strategy is required.

In the short term, it is important to hedge price volatility through futures contracts or long-term supply contracts. For major raw materials, it is necessary to use financial instruments that can diversify price volatility risks and consider sharing risks through price linkage contracts with suppliers.

In the medium to long term, innovation is needed to improve the cost structure. A multifaceted approach is required, including the development of alternative raw materials, cost reduction through design changes, and process efficiency.

In addition, it is necessary to consider redesigning the product structure in a direction that reduces the proportion of raw materials with high price volatility.

Market Price Fluctuation Prediction Failure Response Strategy

Deepflow trains AI models using customer ERP data, external price data, external market environment data, and augmented/synthetic data. This allows for even greater prediction accuracy.

Deepflow has also built a collaborative model in which AI prediction models suggest draft order volumes and receipt dates, and users modify them. This increases the accuracy of AI prediction results and supports user decision-making.

Strategies for Responding to Quality Prediction Failures

In the process of introducing new suppliers or developing new products, more serious quality problems than expected may occur. This can lead to not only production disruptions but also a decline in the quality of finished products, which can negatively affect a company's brand value.

In particular, in products composed of complex components, such as automobiles and electronic products, a single component quality problem can affect the reliability of the entire product.

In the short term, quality inspection and sorting out defective products should be prioritized. Inspection standards for incoming goods should be strengthened, and if necessary, a full inspection should be conducted to prevent defective products from entering the production line.

At the same time, close cooperation with suppliers should be established to identify the root causes of quality issues and provide technical support for improvement.

In the long term, the entire quality management system should be reviewed. The supplier evaluation system should be improved to increase the weight of quality-related indicators, and a system should be established to proactively identify potential problems through regular quality audits.

In addition, risks should be diversified by securing alternative suppliers, and dual sourcing should be considered for core components.

Response strategy to failure to predict delivery date

As the global logistics environment becomes more uncertain, the risk of delivery date prediction failure is also increasing. In particular, in the case of international logistics, various variables such as port congestion, lack of transportation means, and weather conditions can cause delays that are larger than expected.

This can lead to disruptions in production planning and delays in customer delivery, which requires a systematic response.

Short-term responses require diversifying logistics routes and flexible switching of transportation methods. For example, if a problem occurs in maritime transportation, it is necessary to consider switching some of the cargo to air or rail.

It is also necessary to secure temporary stockpiles at intermediate transit points in preparation for emergencies.

In the long term, it is important to increase visibility across the supply chain. A real-time logistics tracking system should be established, and the systems of suppliers and logistics companies should be linked to enable early detection of potential problems.

In addition, a strategy is needed to secure regional logistics hubs and distribute delivery risks through local inventory management.

Technology Change Prediction Failure Response Strategy

Rapid technological advancements can lead to the rapid obsolescence of existing parts or raw materials. In particular, the technology life cycle of electronic components and software-related items is very short, making it even more difficult to predict.

Failure to predict such technological changes can lead to a decline in inventory value and weakening of product competitiveness.

Technology change prediction Failure response strategy

In the short term, strategies are needed to preserve the value of existing inventory. Measures should be considered, such as finding alternative uses for old parts or securing aftermarket demand.

At the same time, partnerships with suppliers should be strengthened to secure new technology parts early.

In the long term, the link between the technology roadmap and the purchasing strategy should be strengthened. Close cooperation with the R&D department should be established to regularly monitor technological development trends and reflect them in the purchasing plan.

In addition, strategic partnerships with new technology suppliers should be established to lay the foundation for preemptively responding to technological changes.

Strategies for Responding to Failure to Predict Regulatory Changes

Stronger environmental regulations and changes in trade policies have a significant impact on companies' purchasing activities. In particular, as ESG-related regulations are being strengthened worldwide, the use of certain raw materials or parts may be restricted or incur additional costs.

If these regulatory changes are not anticipated, companies may experience serious operational disruptions.

In the short term, it is necessary to quickly secure alternative raw materials or parts for regulatory compliance. A plan should be established to minimize the inventory of regulated substances and switch to certified alternatives when necessary.

In addition, the status of suppliers' regulatory compliance should be checked and, if necessary, technical support or certification should be provided.

In the long term, a system should be established to monitor regulatory trends and respond proactively. Cooperation with relevant departments, such as the legal team and the environmental safety team, should be strengthened to comprehensively analyze the impact of regulatory changes and reflect them in mid- to long-term purchasing strategies.

Investments are also needed to build a sustainable supply chain, such as developing eco-friendly raw materials and establishing a recycling system.

Continuous improvement measures to improve prediction accuracy

Data quality management and model update process

Continuous improvement plan to improve prediction accuracy
source: What is Data Quality? Why You Need It & Best Practices

Continuous data quality management is essential to maintain the reliability of the predictive model and improve the accuracy of the prediction.

In particular, standardization and integrated management of data collected from various sources are important, and a dedicated organization or system should be established for this purpose.

The most important aspects of data quality management are outlier handling and missing value correction. Clear standards are needed for how to handle outliers that occur due to rapid changes in the market or special circumstances.

In addition, rules should be established for how to correct missing values.

Regular updates of the predictive model are also important. As the market environment changes, predictive models that were effective in the past may not be suitable today.

Therefore, the performance of the model should be evaluated regularly, and improvements such as adding new variables or changing the model structure should be made if necessary.

Building and operating an early warning system

To minimize the damage caused by failure to predict, it is essential to build an early warning system. This is a system that goes beyond simply monitoring the difference between the predicted value and the actual value and manages various leading indicators in an integrated manner.

Typical leading indicators include changes in supplier production capacity, trends in raw material prices, and market activities of competitors.

An early warning system should include both quantitative and qualitative indicators. For example, in addition to quantitative indicators such as the financial health of suppliers, quality control levels, and delivery compliance rates, qualitative factors such as market reputation, technological innovation capabilities, and the stability of partnerships should also be considered in a comprehensive manner.

In addition, the alert level should be set in stages to enable differentiated responses depending on the severity of the situation. In the initial alert stage, monitoring should be strengthened and response measures should be reviewed, and as the alert level increases, specific response measures should be implemented. This system can be built in a way that allows specific response measures to be implemented as the alert level increases.

Recommendations for building a reliable prediction accuracy verification system

While it is important to improve the accuracy of predictions, it is even more important to be able to systematically respond to prediction failures. There is no such thing as a perfect prediction, and it is a more realistic approach to acknowledge the uncertainty of predictions and have a response system in place.

Organizational culture is also important for this. It is necessary to have a culture that does not treat prediction failures as mere mistakes, but as opportunities for learning and improvement. In addition, cooperation between departments is required to verify the validity of predictions from various perspectives and to establish a system that enables quick response when problems arise.

As digital transformation accelerates, AI-based predictive models are gaining attention, but these technologies can only deliver true value when they are used in an organization's systematic operating processes. Therefore, a balanced approach is required to develop the organization's processes and culture along with the introduction of technology.

연관 아티클