Superforecasters’ Toolbox: Fermi-ization in Forecasting
Superforecasters’ Toolbox: Fermi-ization in Forecasting
Fermi-ization is a key technique Superforecasters use to tackle complex questions when little prior information is available
Although usually a very private person, Superforecaster Peter Stamp agreed to be interviewed by a major Polish daily, Rzeczpospolita, on Good Judgment’s request. The reporter started the interview with a pop quiz. He asked Peter to estimate the number of tram cars that serve the city of Warsaw, Poland’s capital. Without using the internet, or having ever been to Warsaw, in under three minutes Peter came up with a remarkably accurate answer (only 10% away from the actual number, according to the reporter, Marek Wierciszewski). All he needed to know for his calculations was the typical size of a Warsaw tram and the relative importance of this means of transportation.
The method Peter used was Fermi-ization, and it is one of the key techniques Superforecasters employ to tackle complex questions even with minimal prior information.
What Is Fermi-ization?
In his day, physicist Enrico Fermi (1901-1954) was known not only for his groundbreaking contributions to nuclear physics. He was also able to come up with surprisingly accurate estimates using scarce information. The technique he used was elegant in its simplicity: He would break down grand, seemingly intractable questions into smaller sub-questions or components that could be analyzed or researched. He would then make educated guesses about each component until he arrived at his final estimate.
Many science and engineering faculties today teach this method, including through assignments like “estimate the number of square inches of pizza the students will eat during one semester.” Instead of blurting out a random number, students are expected to break the question down into smaller bits and engage with each one to produce a thoughtful answer (in this example, the estimate would depend on such factors as the number of students, the number of pizzas a student would eat per week, and the size of an average pizza).
Fermi-ization is a valuable tool in a Superforecaster’s toolbox. Since the days of the original Good Judgment Project and continuing in Good Judgment Inc’s work today, Superforecasters have proved the usefulness of this technique in producing accurate forecasts on seemingly impossible questions—from the scale of bird-flu epidemics, oil prices, and interest rates to election outcomes, regional conflict, and vaccinations during the Covid-19 pandemic.
Uses of Fermi-ization in Forecasting
In their seminal book Superforecasting, Philip Tetlock and Dan Gardner list Fermi-ization as the second of the Ten Commandments for Aspiring Superforecasters. This placement is not a coincidence. In the world of Superforecasters—experts known for their consistently accurate forecasts—Fermi-ization is a fundamental tool, enabling them to arrive at accurate predictions even in response to questions that initially seem impossible to quantify.
“Channel the playful but disciplined spirit of Enrico Fermi,” Tetlock and Gardner write. “Decompose the problem into its knowable and unknowable parts. Flush ignorance into the open. Expose and examine your assumptions. Dare to be wrong by making your best guesses. Better to discover errors quickly than to hide them behind vague verbiage.”
Depending on the question, this process can take just a few minutes, as it did when Peter worked out an estimated number of Warsaw’s tram cars, or it could be methodical, slow, and painstaking. But it is an invaluable road map whether accuracy is the goal.
Fermi-ization in forecasting has multiple uses:
It helps the forecaster to avoid the classic cognitive trap of relying on quick-and-easy and often incorrect answers where more thought is called for.
It forces the forecaster to sort the relevant components from the irrelevant ones.
It enables the forecaster to separate the elements of the question that are knowable from those that are unknowable.
It makes the forecasters examine their assumptions more carefully and pushes them toward making educated—rather than blind—guesses.
It informs both the outside and the inside view in approaching the question.
Three Steps in Fermi-ization
Fermi-ization becomes easier and increasingly effective with practice. Keep these three steps in mind as you give it a try.
Unpack the question by asking, “What would it take for the answer to be yes? What would it take for it to be no?” or “What information would allow me to answer the question?”
Give each scenario your best estimate.
Dare to be wrong.
Not the Only Tool
Of course, Fermi-ization is not the only tool in a Superforecaster’s toolbox. Mitigation of cognitive biases, ability to recognize and minimize noise, being actively open-minded, and keeping scores are all crucial components of the Superforecasting process. You can learn these techniques during one of our Superforecasting Workshops, or you can pose your own questions for Superforecasters to engage with through a subscription to FutureFirstTM.