Forecaster

Source Document for Event 11 (29-08-2026)

Below is the fully updated and integrated research document, now incorporating the most recent findings—particularly those from late August 2026—while retaining, contextualizing, and weaving in the previously established content. The new material is clearly integrated for coherence and enhanced forecasting utility.

Event Title
AI superforecasters outperform Metaculus Pro Forecasters before January 1, 2028

Description
AI-driven forecasting agents—bots utilizing advanced language models, automated world‑model pipelines, question generation tools, and real‐time adaptation—are increasingly on track to outperform Metaculus Pro Forecasters. The forecasted event posits that an AI system, whether private or public, will regularly achieve higher forecasting accuracy than Metaculus Pro Forecasters across diverse benchmarks and live tournament contexts before January 1, 2028.

Resolution Date
January 1, 2028

1. Baseline Recap (through mid‑ to late‑August 2026)

  • FutureSearch demonstrated parity or superiority in pastcasting benchmarks like ForecastBench and BTF‑3, with competitive Brier scores around ~0.122—positioning it as a frontier AI forecaster.
  • In Metaculus’s FutureEval model leaderboard, human Pro Forecasters continued to dominate with log-scores around ~35–36, significantly outpacing bots such as Claude Fable 5 High (~12–13) and GPT‑5.5 Instant (~12–13).
  • In live tournaments, FutureSearch reigned as #1 among bots in the Summer 2026 FutureEval Bot Tournament and outperformed top humans in peer-score rankings in both the Metaculus Cup and Market Pulse Challenge.
  • Slight variability appeared in rolling MiniBench series: top rankings in June/July 2026 but slipping to #10 by early August.
  • Methodologically, FutureSearch’s use of automated question generation (with ~96% resolution accuracy) and world-model embedding infrastructure provided structural advantages.
  • Critiques highlighted that AI supremacy claims rely heavily on backtests and may not yet fully hold in live-market frameworks like ForecastBench.
  • Community sentiment—from Reddit threads in early and late August 2026—signalled growing recognition that bots may be outperforming top human superforecasters, though skepticism remained in some quarters.
  • Integration of these elements shaped strong confidence (~97–98%) that AI would surpass human forecasters by early 2028, with near certainty (~100%) in real-time tournament contexts.

2. New Findings (Late August 2026, since August 28, 2026)

2.1 Live and Benchmark Performance Updates

  • As of today, FutureSearch holds the #1 position among 253 bots in the ongoing Summer 2026 FutureEval live bot tournament—confirming its sustained live-tournament dominance (futuresearch.ai).
  • It also maintains a Brier score of 0.122 on the BTF‑3 pastcasting benchmark—a top-tier score, with the next best frontier model scoring around 0.130 as of July 2026 (futuresearch.ai).
  • The July 2026 blog post “Iterating on a Forecaster…” confirms that FutureSearch remains first among 163 bots in the Summer 2026 FutureEval tournament and exceeds the median human superforecaster percentile on ForecastBench (futuresearch.ai).

2.2 FutureEval Trend Continuity

  • Metaculus’s FutureEval site reiterates the projection that bots were expected to surpass the broader community by April 2026 and Pro Forecasters by mid‑2027—verifying the earlier timeline projections remain unchanged (metaculus.com).
  • The current FutureEval interface shows that Pro Forecasters still outpace even the best bots—e.g., Pro log-score ~35.05 vs bots hovering around ~12–13—confirming that human performance still leads overall (metaculus.com).

2.3 Independent Analysis & Perspective

  • A recent substack post (“Tetlock Among the Machines,” Aug 22, 2026) reflects growing independent belief that AI forecasters are approaching parity with elite human forecasters on benchmarks such as ForecastBench, noting statistical indistinguishability in performance on ForecastBench (tellingthefuture.substack.com).
  • This matches metagaming trends in real-time tournament performance and benchmark dominance.

2.4 Community Sentiment Reinforced

  • A Reddit thread from early August 2026 continues to emphasize FutureSearch as "approximately superhuman," citing its dominance over top individual human forecasters and its robust world-modeling capabilities (reddit.com).

3. Integrated Synthesis & Forecast Adjustment

Pastcasting and Controlled Benchmarks
FutureSearch remains the top model on BTF‑3 (Brier 0.122 vs ~0.130 peers)—its position reinforced by independent sources. Substack and similar analyses now suggest AI and elite humans may be statistically indistinguishable on benchmarks like ForecastBench. This strengthens confidence in AI's dominance in these contexts.

FutureEval / Metaculus Long-Term Forecasting Benchmark
Pro Forecasters still retain a clear lead in aggregated log-scores (~35 vs ~12 for bots). However, the consistent trajectory and stable update delays support the mid‑2027 crossover prediction. There is no evidence of delay or slippage in that timeline as of late August 2026.

Live Tournaments & Market-Based Challenges
AI has clearly taken the lead in tournaments: FutureSearch remains #1 among bots in Summer 2026 FutureEval live tournament and top in Metaculus Cup peer score rankings. These real-time outcomes underscore an operational edge even before formal parity.

Methodological & Infrastructure Edge
FutureSearch continues to leverage automated question generation and world-model embedding infrastructure, reinforcing operational superiority in both depth and scale.

Community Sentiment
External commentary increasingly supports AI’s emerging parity or superiority. Reddit threads and independent analysis enhance the credibility of this shift in sentiment, adding grounding to forecasts.

4. Updated Probability Table (as of August 29 2026)

Context / BenchmarkUpdated Probability AI Outperforms by Jan 1 2028Rationale
ForecastBench / Pastcasting Benchmarks99%FutureSearch leads BTF‑3, and independent analysis suggests near parity or better than humans.
Controlled Algorithmic Benchmarks98%Methodological advantage remains; external confirmation adds weight.
FutureEval (Metaculus Leaderboard)85%Pro Forecasters still lead, but consistent trend supports mid‑2027 crossover; slight increase due to sustained performance.
Live Tournaments & Market-based Challenges100% (practically certain)AI already outperforms humans in peer-score rankings in live competitions.
Decision-making and Strategic Foresight Use96%Infrastructure and live performance continue to enhance reliability.

Overall Integrated Estimate: ~98–99%
The probability that AI superforecasters will regularly outperform Metaculus Pro Forecasters by January 1, 2028 remains extremely high.

5. Monitoring & Follow‑Up Strategy

  1. Continued monitoring of the FutureEval model leaderboard and live bot tournament standings—particularly for convergence between bots and Pro Forecasters.
  2. Watch for BTF‑4 or new Benchmarks (e.g., ForecastBench updates) for further evidence of AI vs. human accuracy trends.
  3. Follow independent analyses (e.g., substack, forecasting research institutes) for third-party validation or critique.
  4. Track AI methodology advances, especially in question generation, embedded world models, and agent orchestration through MCP-like infrastructure.
  5. Monitor community sentiment dynamics on platforms such as Reddit, Substack, and forecasting forums.
  6. Scrutinize methodological critiques of retrospective benchmark comparisons to ensure robustness of superiority claims.

6. Conclusion (as of August 29, 2026)

  • Benchmarks: AI, and especially FutureSearch, continues to dominate pastcasting benchmarks (e.g., BTF‑3). Independent analysis suggests AI may be statistically reaching parity or exceeding human superforecasters.
  • Live Competitions: AI’s superiority is already manifest in live tournament peer-score performance.
  • FutureEval Trends: Pro Forecasters still lead overall forecasts, but the trend remains firmly in AI’s favor, with overtaking still projected by mid‑2027.
  • Methodology & Infrastructure: AI systems continue to innovate with practices like automated question generation and world-model embedding, reinforcing the forecast advantage.
  • Perception: Community opinion increasingly recognizes AI’s strength relative to human forecasters.
  • Probability of Outcome: Slight upward revision; now ~98–99% confidence that AI superforecasters will outperform Metaculus Pro Forecasters before January 1, 2028.

Let me know if you’d like a more granular model-level breakdown, domain-specific performance comparisons, or an analysis of methodological innovations across different AI forecasting systems.