How Superforecasters and Prediction Markets Are Changing the Science of Decision-Making

Every day, we are bombarded with predictions about the future. Will inflation drop? Will a new technology succeed? Will a geopolitical conflict escalate? Traditional media and experts often leave the public more confused than informed, hedging their bets with vague words like "might" or "could." Because these predictions lack quantifiable probabilities, they are nearly impossible to verify, allowing pundits to claim victory regardless of the actual outcome.[1][6]

For decades, the accuracy of expert predictions was rarely checked or held to any rigorous standard. In his landmark research, psychologist Philip E. Tetlock famously demonstrated that the average expert forecaster was roughly as accurate as a "dart-throwing chimpanzee." When experts speculated on major global events, their deep domain knowledge often made them overconfident rather than accurate, blinding them to alternative outcomes and unexpected variables.[1][2]

But forecasting is not a lost cause; it is a measurable, improvable skill. The paradigm shift began in 2011 when the U.S. Intelligence Advanced Research Projects Activity (IARPA) launched a massive tournament to improve the intelligence community's ability to foresee global events. The goal was to see if structured, crowdsourced forecasting could outperform traditional, siloed intelligence gathering.[2][5]

Tetlock entered the tournament with a team of ordinary citizens, dubbed the Good Judgment Project. Over four years and hundreds of questions, this group of volunteers consistently outperformed trained intelligence analysts who had access to classified information. They also beat the accuracy of random guessing and early prediction markets, proving that a specific methodology could yield remarkable foresight.[2]

These elite predictors became known as "superforecasters." Their success was not due to insider knowledge, genius-level IQs, or psychic intuition, but rather a specific, disciplined approach to thinking. They broke large, complex questions into smaller, manageable parts, actively sought out opposing viewpoints, and constantly updated their probabilities as new information arrived.[2][6]

To truly improve forecasting, one must be able to measure it objectively. The gold standard for this measurement is the Brier score, a mathematical formula that evaluates the accuracy of probabilistic predictions. Developed in 1950, the Brier score calculates the squared deviation between the probabilities assigned to forecasts and the actual binary outcomes.[5]

A Brier score ranges from 0 to 1. A perfect score is 0, indicating absolute accuracy, while a forecaster who simply guesses 50/50 on every question receives a baseline score of 0.25. Crucially, the Brier score heavily penalizes overconfidence—being 100% certain about an event that ultimately does not happen destroys a forecaster's rating, structurally encouraging intellectual humility.[5]

While tournaments proved that humans could be trained to predict better, a parallel movement sought to crowdsource accuracy using financial incentives: prediction markets. Platforms like Polymarket and Kalshi have turned probability into a tradable asset, allowing users to buy and sell shares based on the likelihood of future events.[1][4]

In a prediction market, if a share for a "Yes" outcome trades at 60 cents, the market implies a 60% probability of that event occurring. Because traders have their own money on the line, the market naturally filters out low-quality, performative punditry. People who are consistently wrong lose money, while accurate forecasters profit, creating a powerful financial incentive for truth-seeking.[1]

By 2026, these platforms are processing billions of dollars in volume each month. They track everything from the success of new cancer drugs to macroeconomic stability, with risk-monitoring markets alone accounting for roughly $3.8 billion in trading volume. The financial stakes force participants to calculate thoughtfully and weigh evidence, creating a real-time, living assessment of global probabilities.[1][4]

The combination of superforecasting techniques and prediction markets has created a powerful engine for collective intelligence. But a new challenger has recently entered the arena: artificial intelligence. Large Language Models (LLMs) are now being deployed as a new form of collective intelligence, aggregating human knowledge at an unprecedented scale.[3]

Recent studies have tested these AI models against human aggregates in forecasting tournaments like ForecastBench. AI models excel at processing vast amounts of historical data and identifying patterns, making them highly competitive on questions with established datasets and short feedback loops.[3]

However, human superforecasters still hold a commanding lead on complex, novel market questions where historical data is sparse and human intuition is required to weigh unprecedented variables. The current frontier of forecasting relies on the synergy between human reasoning and AI processing power, rather than one replacing the other.[3][6]

For the average person, the lessons of superforecasting offer a concrete toolkit for better decision-making in everyday life. Research shows that simply learning the basic principles of forecasting—such as embracing doubt, avoiding absolute certainty, and updating beliefs incrementally—can improve an individual's predictive accuracy by 10%.[2]

Ultimately, the science of forecasting teaches us that "knowing what we don't know is better than thinking we know what we don't." By demanding verifiable accuracy, utilizing the wisdom of crowds, and thinking in probabilities rather than absolutes, society can navigate an uncertain world with greater clarity, confidence, and capability.[1][2]