A recent McKinsey study found that many companies still rely on manual forecasting methods, believing AI will require high-quality data. This could be a ‘costly mistake’
Forecasting in data-light environments
Data-light environments aren’t uncommon for companies. The custom of storing silos of data for usage by AI technologies is a rather new one, and most companies today are at a stage where they have just started or are very early in the process of making this tumultuous transition one step at a time. Hence, issues such as a lack of structured data is rampant, and the belief that they can’t be used currently is widespread as well.
The argument to be made here is that the right AI models may, in fact, be powerful enough to use even in such data-light environments. McKinsey finds, for example, that not only do AI models have clear advantages over spreadsheet-based counterparts, but aspects such as AI supply chain management can, in fact, reduce errors by anything between 20-50%, reducing lost sales and product unavailability by almost 65% already. Furthermore:
“Continuing the virtuous circle, warehousing costs can fall by 5 to 10 percent, and administration costs by 25 to 40 percent. Companies in the telecommunications, electric power, natural gas, and healthcare industries have found that AI forecasting engines can automate up to 50 percent of workforce-management tasks, leading to cost reductions of 10 to 15 percent while gradually improving hiring decisions—and operational resilience.”
Strategies for the data-light
Automated AI-driven forecasting models promote the benefits of consuming real-time data, and continuously identifying new patterns. This enables fast and agile actions as models can respond to continually-changing demand parameters pre-emptively, rather than just responding to them. In contrast, traditional forecasting requires constant manual updating of data, with constant changes to forecast outputs. This is not only time-consuming but also does not allow for agile responses to immediate changes in data patterns – aspects particularly relevant in risk assessment, workforce planning, or expenditure management.
From a technical standpoint, writes McKinsey, there may be four major strategies that companies may adopt to tackle this:
- Choosing the right AI model: This is absolutely imperative to the adoption of any kind of AI technology that can assist a company. This must be the foremost step to any problem: to identify the most appropriate algorithm based on the quantity and quality of data available. In many instances, for example, machine learning models may test and validate several models to find the optimum choice, even one with minimum required supervision.
Leveraging data-smoothing and augmentation: This is a technique used primarily with time-series data when a period in the series is not representative of the rest of the data. Consider the COVID-19 pandemic as an example. The effect of the anomalous trends in data needs to be continuously modelled, to teach the algorithm how to deal with similar recurring issues.
- Preparing for uncertainties in prediction: Forecasting models may often turn out to be unsatisfactory in terms of accuracy, especially when very limited historical data may be available. In such cases, sophisticated scenario-planning tools may prove to be of extreme value, allowing users to input a wide variety of parameters to their models to help append their forecasting accuracy.
- Incorporating external APIs: Externally sourced data can cover a variety of content and sources, including “social-media activity, web-scraping content, financial transactions, weather forecasts, mobile-device location data, and satellite images. Incorporating these data sets can significantly improve forecast accuracy, especially in data-light environments. These sources provide an excellent option for the inputs required for AI-driven models and create reasonable outputs. The market for external data is expected to have a CAGR of 58 percent, reflecting the increasing popularity of these data sources and the significant expansion in the types of external data available.”