How a company’s judgment can be improved

THE just concluded Sabah 16th State Election has one thing to tell us – how reliable or unreliable our judgment or prediction can be. The coalition of Warisan-Plus was so confident of regaining the state office and almost all opinion polls were pointing to its muchanticipated landslide victory. But what went wrong?

How reliable is our prediction and judgment in business? What are the bases of making a sound judgment or decision?

The business community is threatened with unprecedented turbulent times ahead. Companies and individuals are notoriously inept at judging the likelihood of uncertain events. Getting judgment wrong can have serious adverse consequences.

However, improving a firm’s forecasting competence even a little can yield a marked competitive advantage.

For issues that can be predicted with great accuracy using econometric and operations research tools, there is no advantage to be gained by developing subjective judgment skills in those areas.

Data speaks louder than guesstimate. At the other end of the spectrum, it is also futile to predict issues that are too complex, poorly understood or tough to quantify, such as guessing the patterns of the clouds on a given day.

However, companies can improve its forecasting competence by focusing on issues where data, logic, and analysis can be used to forecast a more accurate outcome.

To improve predictions, we can focus on improving our forecasting ability through training; using teams to boost accuracy and tracking prediction performance and providing rapid feedback.

Most predictions made in companies are influenced by the forecaster’s understanding of the basic statistical arguments, susceptibility to cognitive biases, desire to influence others’ thinking and concerns about reputation. Predictions are often intentionally vague to give room for explanation should they be proved wrong.

Training in reasoning and debiasing can reliably strengthen a firm’s forecasting competence. Basic reasoning errors such as believing that a coin that has landed heads up three times in a row is likely to land tails up on the next flip. T

here is still a 50-50 chance that the coin is to be landed either heads up or tail up. Understanding the basics of statistics can remove inherent statistical misconception.

Cognitive biases are widely known to skew judgment. They lead people to follow the crowd, to look for information that confirms their views, and to strive to prove just how right they are.

A trained forecaster watches out for confirmation bias that can create false confidence and give due weight to evidence that challenges their conclusions.

It is safer to play the devil’s advocate than to deliver rosy recommendations that give false hope. We should not look at problems or proposals in isolation. For example, in predicting how long a project will take to sell, say 80%, we should first find out how long it typically takes to sell similar projects, preferably in similar economic conditions.

Training can also help us to avoid reliance on flawed intuition instead of careful analysis. It exposes cognitive traps, emotional influences or just wishful thinking. We do not want to be trapped in our own silo and put something in our 90% confidence ranges that do not contain the correct answer.

Forecasters produce more accurate judgment when they work collaboratively in teams than when they work alone in isolation. This is because individual biases tend to cancel out with more members in the team. To assemble such a team, companies should look for natural forecasters who show an alertness to bias, a knack for sound reasoning, and a respect for data. It is also important that forecasting teams be intellectually diverse.

It should have at least one domain expert on the subject to be forecasted. Non-experts who will not shy away from challenging the presumed experts are essential too.

A forecasting team needs to manage three stages in making a prediction. The first stage involves gathering all the assumptions and approaches to find an answer from multiple angles.

The second stage involves intense brain-storming and managing productive disagreement. The final stage is to sit down together, analyse all the data and settle on a prediction. The team must avoid tunnel vision or focusing too narrowly and locking quickly into a wrong answer. It must focus on gathering new information and testing assumptions relevant to the forecast.

Lastly, tracking prediction outcomes and providing timely feedback is essential to improving forecasting performance. To ensure success, companies should systematically collect real-time progress data of how their forecasters make judgment; and keep records of assumptions made, data used, experts consulted, economic situation and so on. This will help the company to realign its strategic direction as necessary if the progress appears to be going off tangent.