In-Depth Insights

1. Know The Limitations of Forecasting

Our forecasts reflect our expectations for our future but life in our complex world means that small events can lead to large unforeseen consequences making regular forecasting rather limited.

Take the Arab Spring for example, the uprisings started because 1 Tunisian street vendor set himself on fire after he was humiliated by deceitful police officers. Forecasters cannot predict events such as this due to the chaos theory aka ‘the butterfly effect’. These limitations don’t mean that forecasting should be scrapped though.

2. Measure and Update Your Forecasts

Weather forecasts are quite reliable when made a few days in advance because forecasters analyze the accuracy of their forecasts with the actual weather that has occurred, this allows them to improve their forecasts alongside their understanding of how the weather works. Unfortunately, forecasters in other fields do not usually measure the accuracy of their forecasts so in order to improve forecasting we have to get serious not only about comparing but about measuring and updating our findings.

You can also use the Brier score system to see how accurate past forecasts were. The Brier score system states that the lower the score, the more accurate the forecast so if the forecast was perfect, it gets 0. Forecasts which are randomly guessed are given a score of 0.5 whilst completely wrong forecasts are given a maximum score of 2.0. Interpreting the score depends on the question being asked as you might have a Brier score of 0.2 which seems great yet the forecast could be terrible.

3. Be Precise with Your Measurements

You might think that measuring forecasts is as simple as gathering the data and doing the calculations to judge their accuracy but it’s not quite that simple as you have to understand the meaning behind the original forecast.

For increased precision, forecasters should avoid using vague language like “might”, “could” and “likely” because different people attach different meanings to these words, it’s far better to use numbers, particularly percentages. 

The false claim of Saddam Hussein having weapons of mass destruction was made by the NSA and CIA intelligence agencies, here is an example where percentages rather than words should have been used - If the forecast said that the chances of Iraq having weapons of mass destruction was 60% likely, this would have meant that there was a 40% chance they didn’t resulting in insufficient knowledge to justify the U.S invading Iraq.

4. Break Down Problems

How do you eat an elephant? One bite at a time! Seemingly impossible forecast problems can be tackled the same way, not by eating them but by applying Fermi-style thinking which means breaking problems down into smaller bite-size units to analyze. Effective superforecasters separate the known from the unknown, getting to grips understanding the basics first before looking for assumptions.

Let’s look at the forecast of the cause of death of Yasser Arafat, leader of the Palestine Liberation Organization. Initially, his cause of death was unknown but it was later discovered that he had been in contact with lethally high levels of polonium-210 which were found on his belongings. Conspiracy theories had always mentioned poison but now this seemed a very likely cause of death. His body was exhumed and examined in France and Switzerland with volunteer forecasters on the Good Judgment Project asked to forecast whether scientists would find high levels of polonium in Yasser Arafat’s remains. One volunteer concluded that there was a 60% chance thanks to the Fermi-style approach which he used to break down the information he had. People said that Polonium decays rapidly so it was thought that the chance of it still existing in Arafat’s remains, 8 years after he had died, were slim however, the forecaster did his own research and discovered that it could still be detectable. Also taking into consideration that the leader had Palestinian enemies and that there could have been foul play during the postmortem due to Palestine wanting to blame Israel for Arafat’s death and the forecaster came to his 60% conclusion.

5. Don’t Jump To Conclusions

Don’t judge a case too quickly as every situation is unique. Approach it from the outside with a concept called anchoring which has forecasters finding the initial figure, the base rate.

Let’s say you want to predict the chances of a particular Italian family owning a pet. A regular forecaster would look at the specific facts about the family first, learning that they live in the U.S in a modest house and that the father is a bookkeeper and the mother a part-time childcare provider. They have 1 child and the grandmother lives in the same house. 

Superforecasters don’t use the inside facts first, the details, they go to the outside view first discovering what percentage (base rate) of American households own a pet. Thanks to Google they quickly learn the base rate is 62%. It’s only now that the superforecaster looks at the inside view taking into consideration the family dynamics as this allows them to get more specific with their data and adjust the base rate accordingly. Knowing that the family is Italian, their next step might be to check the percentage of Italian families owning a pet in the U.S.

6. Stay Up To Date

Once your initial forecast is made, it’s no good sitting back to wait and see if you were right, you’ve got to adjust your forecast as new information comes to light.

This is exactly what the researcher who forecasted that Yasser Arafat’s remains had a 60% chance of being detected with polonium did. By keeping an eye on the news surrounding this case, you can also set a Google Alert, he would update his forecast as new data arose. The official Swiss research team announced that they needed to do additional testing long after Bill Flack made his original forecast. Because Bill had done a lot of research on polonium he knew that the delay meant the Swiss team had found polonium and that they needed to do further tests to confirm its source. He adjusted his original forecast from 60% to 65% certainty that Arafat had died of polonium. The Swiss team did find polonium so Bill’s final Brier score came out at an impressive 0.36.

This case-study doesn’t mean that new information is always helpful though - it can also misguide you. To update a forecast you must use your skill to tease out subtle details from extraneous information. On the other hand, you shouldn’t be afraid to change your mind and update your forecast.

7. Working In Teams Can Be Beneficial

The term groupthink was coined by psychologist Irving Janis who hypothesized that people working in small groups build team spirit by unconsciously creating shared illusions that disrupt critical thinking because of people agreeing with each other.

The research team on the Good Judgment Project decided to see if teamwork would influence accuracy and concluded, after 1 year, that on average, the people who worked in teams had forecasts that were 23% more accurate than forecasts made by individuals. In the 2nd year, the team replaced regular forecasters with superforecasters and found they outperformed the regular group of forecasters tremendously but that the group dynamics were impacted with everyone overly polite with very little critical discussion and no one willing to present an alternative view. 

To combat this problem teams can work more effectively by using precision questioning, a method that encourages people to rethink their argument by defining the finer details of their argument i.e by asking them to define a particular term.

Remember, by not conforming, your ideas can be a tremendous asset to the group.

The above is inspired from the bestselling book Superforecasting' by Philip Tetlock

Exercises | Workbook


Apply What You Read

* With Elevate Uni’s All-Access Pass you can get unlimited access to a growing collection of study guides, courses and actionable resources from the best minds in leadership, business and personal growth. Each resource includes not only in-depth insights but also self exploration exercises, reflective questions and action points so you integrate what you learn.