IndustryWeek : Eight Ways to Critically Evaluate a Forecast
Home : Leadership & Strategy : Best Practices : Eight Ways to Critically Evaluate a Forecast


Eight Ways to Critically Evaluate a Forecast

Don't get caught up in 'worthless' prediction measures, such as long-range, detailed forecasts.

By Adam Gordon

April 7, 2010

Forecasts are a bit like a spouse: you can't live with them; you can't live without them. Anticipating future conditions and requirements are keys to future success. In manufacturing industries in particular, forecasts are necessary to make better decisions about industry conditions and market demand.

At the same time, forecasts are at best a patchy guide to the world of tomorrow. They are often wide of the mark, sometimes absurdly so. It's never hard to get a view of the future. What's hard is knowing whether it's correct.

Given that forecasts are necessary but usually of low quality, manufacturing managers need to assert a strong independence of mind to probe whether any forecast is valid, and if so, how valid? By asking tough questions we can get to the value in a forecast and avoid being duped by bias, cowed by expert credentials or bamboozled by fancy software.

Some key questions to ask include:

1. What is the purpose of the forecast? What can be gleaned about why it exists, who put it out or what the intention of the forecaster was? Is the forecast upfront about its purpose? All forecasting is done for benefit. By recognizing the interests at work behind a forecast, assessing what benefit or benefits are sought by the forecaster or whoever commissioned it, one can make a better judgment as to potential strengths and weaknesses. We may ask: What action or concerns is the forecast trying to arouse? How is it legitimating a view that the forecaster or forecast organization holds?

2. Is the forecast telling you what will happen or trying to make something happen? Forecasts fall into two main categories: future-aligning, where forecasters anticipate change in order to adapt early and successfully to it; or future-influencing, where forecasters are trying to influence events. Future-aligning approaches aim to be objective. They may fail, but the intention is there. So on balance, this approach will be more accurate. Future-influencing forecasts aim to succeed on other terms -- alerting and shaping opinion, changing minds, and harnessing action. Forecasts that are trying to lobby or change industry conditions make themselves known by seeking publicity and often being a forecast of extreme optimistic or pessimistic outcomes (to be aspired to or negated).

3. Is there too much certainty? Certainty is a warning sign. In short-term situations or closed systems with few variables, the attempt to pinpoint outcomes is reasonable. But the forecast consumer should consider claims to medium- and long-term accuracy with acute skepticism. Long-range detailed point predictions are almost always worthless. Beyond the short term the world is complex and effectively unpredictable. Anyone who says they can predict it is announcing themselves as a charlatan.

4. Does the forecast rely on "experts?" Experts are necessary in a specialized world, and expertise and credentials are important in forecasting, but experts are wrong as much as anyone. This is because a field's experts are particularly likely to be heavily invested in the status quo and have expertise in its existing procedures, attitudes and prejudices. Change often comes from outside, and experts -- blinkered by their knowledge of today -- are often the last to see it.

5. Are blocking forces identified and fully accounted for? All drivers of change work against the frictional resistance of the status quo -- the systems and solutions that people are currently invested in and comfortable with. They also face direct 'blockers' and 'turners,' which are forces that have a vested interest in the status quo and don't want to see change, or that have an interest in another type of change. A good forecast will assess the strength of resistance to change and anticipate specifically if and how this resistance will be overcome, if indeed it will be, and account for the resources required to achieve this. Rather than running with the breathless wow-of-the-new, the forecast will display a measured pragmatism in the face of constraints, and adjust the forecast direction and/or timing accordingly.

Displaying 1 of 2
Page:<< Back ยท Next >>
View article on one page
Spotlight

Klein Steel Rewards Values in Action

By Jill Jusko
Company's employee recognition program keeps firm's core values front and center.

Read Full Story
Click here to learn more
Also on IndustryWeek.com

New White Papers

More White Papers »

Poll
In a recent article for IndustryWeek.com, Michael Newkirk asks: "Is manufacturing dead in America?" What do you think?



Comment in the IW Forums.