Weight the dominant trend sub-question (sq1) most heavily, blending in bot participation (sq2), human field strength (sq3), and general AI capability gains (sq4) via weighted average to reflect their correlated dependence on the same underlying bot-improvement momentum.
## Cross-Market Signals ### No signal found
1. [sq1 | web_search | STRONG cred 85 | UP | DATED] In Spring 2026 Cup, Chestnut scored 1059.8 vs top human tejvn 1156.6, a ~97 point gap, after placing 3rd and beating Community Prediction. 2. [sq1 | web_search | STRONG cred 85 | UP | DATED] Bots improved across editions: ManticAI placed 8th (Summer 2025) then 4th (Fall 2025), then Chestnut 3rd (Spring 2026). 3. [sq1 | code_execution | WEAK cred 35 | NEUTRAL | VERY_RECENT] Trajectory-based model estimates P(bot beats all humans) at ~0.37 given the ~97 point Spring 2026 gap. 4. [sq1 | web_search | MODERATE cred 70 | NEUTRAL | VERY_RECENT] No Summer 2026 leaderboard rankings or scores were publicly available as of June 24, 2026; only two questions had resolved. 5. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] Summer 2026 Cup is ongoing; competitors including bots can join at any time and improve over the tournament. 6. [sq2 | web_search | MODERATE cred 75 | UP | DATED] Mantic raised $4M pre-seed funding with backing from DRW and DeepMind/Anthropic researchers, indicating sustained bot development resources. 7. [sq2 | web_search | WEAK cred 50 | NEUTRAL | VERY_RECENT] No direct confirmation that Preseen-Chestnut or ManticAI is entered in the Summer 2026 Cup was found. 8. [sq3 | web_search | MODERATE cred 80 | NEUTRAL | RECENT] Summer 2026 features journalist Dylan Matthews as a guest competitor with $2,500 bonus pool if Community beats him. 9. [sq3 | web_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] No evidence found that the Dylan Matthews competition is drawing exceptionally strong elite human forecasters into the Cup. 10. [sq4 | web_search | MODERATE cred 70 | UP | RECENT] Metaculus was preparing a $50,000 Summer Bot Tournament and runs Bi-Weekly MiniBench, providing fast feedback loops for bot improvement. 11. [sq4 | web_search | MODERATE cred 75 | UP | DATED] Bot performance rose from ~80% of top human average (Fall 2025) toward parity, showing continued capability gains. ## Cross-Market Signals ### No signal found Information gaps: - No confirmation of which strong bots are entered in Summer 2026 - No Summer 2026 leaderboard scores yet (only 2 questions resolved) - No base rate on edition-to-edition gap closure magnitude - Unknown strength of Summer 2026 top human field Key uncertainties: - Whether bot improvement trend continues or plateaus - Variance from small number of resolved questions in a 4-month tournament - Whether an exceptional human forecaster dominates the field - Whether top bots are active and tuned this edition
You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates — your job is to give the most accurate per-component probabilities.
## Question
Will a bot beat all humans in the Summer 2026 Metaculus Cup?
## Description / Resolution Criteria
## Description
The Metaculus Cup is a 4-month forecasting tournament with a variety of topics and 3-4 questions launching every week. Bots have been steadily improving through recent iterations of the Cup. ManticAI placed 8th in the Summer 2025 edition and 4th in the Fall 2025 edition, even beating the Community Prediction in the latter. In the Spring 2026 edition, ChestnutForty (now known as Preseen-Chestnut) came 3rd, handily beating the Community Prediction and coming within 100 points of beating all humans. Chestnut scored 1059.8, while tejvn, the top human, scored 1156.6.
The Summer 2026 edition features a competition against prominent forecasting-adjacent journalist Dylan Matthews, with an extra \$2500 added to the prize pool if the Community Prediction beats him.
As of June 9, 2026, only two questions have resolved in the Summer 2026 Cup.
`{"format": "metac_reveal_and_close_in_period", "info": {"post_id": 43900, "question_id": 43926}}`
## Resolution Criteria
This question will resolve as Yes if after, all questions except this one and [this question](https://www.metaculus.com/questions/43367/community-beats-dylan-matthews-in-summer-2026-cup/) (which resolves last) have resolved, a bot has a Total Score greater than any human forecaster on the Summer 2026 Metaculus Cup leaderboard.
## Fine Print
This question's information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/43900) which opened on 2026-06-22 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question's resolution could have been known before this question opened, then this question will be annulled.
## Sub-question decomposition
- (w=0.40) Will the best-performing bot's relative improvement over the top human continue its upward trend (i.e., closing the ~100 point gap from Spring 2026)? — The core trend: bots went from 8th (Summer 2025) to 4th (Fall 2025) to 3rd within 100 points (Spring 2026). Continued tr
- (w=0.25) Will a strong bot (e.g., Preseen-Chestnut, ManticAI, or similar) be entered and active in the Summer 2026 Metaculus Cup? — A bot must actually participate and perform competitively for YES; absence of top bots would lower the probability subst
- (w=0.20) Will the top human field in Summer 2026 be no stronger than prior editions (no exceptional human outperformance, e.g., from Dylan Matthews competition drawing elite forecasters)? — The Dylan Matthews competition and added prize pool may attract stronger humans, raising the bar a bot must clear.
- (w=0.15) Will LLM/forecasting-bot capabilities have improved generally between Spring 2026 and Summer 2026? — Background rate of AI forecasting improvement supports the gap closing; rapid model releases favor bots.
Combination rule: **weighted_average**
## Synthesized evidence
1. [sq1 | web_search | STRONG cred 85 | UP | DATED] In Spring 2026 Cup, Chestnut scored 1059.8 vs top human tejvn 1156.6, a ~97 point gap, after placing 3rd and beating Community Prediction.
2. [sq1 | web_search | STRONG cred 85 | UP | DATED] Bots improved across editions: ManticAI placed 8th (Summer 2025) then 4th (Fall 2025), then Chestnut 3rd (Spring 2026).
3. [sq1 | code_execution | WEAK cred 35 | NEUTRAL | VERY_RECENT] Trajectory-based model estimates P(bot beats all humans) at ~0.37 given the ~97 point Spring 2026 gap.
4. [sq1 | web_search | MODERATE cred 70 | NEUTRAL | VERY_RECENT] No Summer 2026 leaderboard rankings or scores were publicly available as of June 24, 2026; only two questions had resolved.
5. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] Summer 2026 Cup is ongoing; competitors including bots can join at any time and improve over the tournament.
6. [sq2 | web_search | MODERATE cred 75 | UP | DATED] Mantic raised $4M pre-seed funding with backing from DRW and DeepMind/Anthropic researchers, indicating sustained bot development resources.
7. [sq2 | web_search | WEAK cred 50 | NEUTRAL | VERY_RECENT] No direct confirmation that Preseen-Chestnut or ManticAI is entered in the Summer 2026 Cup was found.
8. [sq3 | web_search | MODERATE cred 80 | NEUTRAL | RECENT] Summer 2026 features journalist Dylan Matthews as a guest competitor with $2,500 bonus pool if Community beats him.
9. [sq3 | web_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] No evidence found that the Dylan Matthews competition is drawing exceptionally strong elite human forecasters into the Cup.
10. [sq4 | web_search | MODERATE cred 70 | UP | RECENT] Metaculus was preparing a $50,000 Summer Bot Tournament and runs Bi-Weekly MiniBench, providing fast feedback loops for bot improvement.
11. [sq4 | web_search | MODERATE cred 75 | UP | DATED] Bot performance rose from ~80% of top human average (Fall 2025) toward parity, showing continued capability gains.
## Cross-Market Signals
### No signal found
Information gaps:
- No confirmation of which strong bots are entered in Summer 2026
- No Summer 2026 leaderboard scores yet (only 2 questions resolved)
- No base rate on edition-to-edition gap closure magnitude
- Unknown strength of Summer 2026 top human field
Key uncertainties:
- Whether bot improvement trend continues or plateaus
- Variance from small number of resolved questions in a 4-month tournament
- Whether an exceptional human forecaster dominates the field
- Whether top bots are active and tuned this edition
## Required pre-forecast walkthrough
Before giving probabilities, walk through these explicitly:
(a) The time left until the question resolves.
(b) The status quo outcome — what happens if nothing changes from today.
(c) A brief scenario that results in NO.
(d) A brief scenario that results in YES.
## Calibration guidance
- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition — independence forces more careful reasoning per factor.
- **Anchor on the status quo.** Require strong specific evidence to move far from it.
- 0.05 / 0.95 require strong, multi-source evidence. Avoid >0.97 or <0.03 unless the outcome is already settled or near-tautological — overconfident misses are punished heavily by the log score.
- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question — move further only with specific evidence the market lacks.
- **Treat research as fallible, not ground truth.** A single-source or "very recent" claim — especially one the evidence flags as unverified, possibly AI-generated, or low-credibility — must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.
- **Also provide a holistic estimate** — your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.
## Output
Return ONLY valid JSON, no markdown fences:
{
"rationale": "<address (a) (b) (c) (d) above — 5-8 sentences total>",
"sub_question_estimates": {
"sq1": <float in [0.01, 0.99]>,
"sq2": <float in [0.01, 0.99]>,
"sq3": <float in [0.01, 0.99]>,
"sq4": <float in [0.01, 0.99]>
},
"holistic_p_yes": <float in [0.01, 0.99] — your overall estimate ignoring the decomposition>,
"what_would_change_my_mind": "<1-2 sentences: what new info would push you above 70% or below 30%>"
}
{
"forecast": {
"combination_logic": "weighted_average",
"evidence_brief": "1. [sq1 | web_search | STRONG cred 85 | UP | DATED] In Spring 2026 Cup, Chestnut scored 1059.8 vs top human tejvn 1156.6, a ~97 point gap, after placing 3rd and beating Community Prediction.\n2. [sq1 | web_search | STRONG cred 85 | UP | DATED] Bots improved across editions: ManticAI placed 8th (Summer 2025) then 4th (Fall 2025), then Chestnut 3rd (Spring 2026).\n3. [sq1 | code_execution | WEAK cred 35 | NEUTRAL | VERY_RECENT] Trajectory-based model estimates P(bot beats all humans) at ~0.37 given the ~97 point Spring 2026 gap.\n4. [sq1 | web_search | MODERATE cred 70 | NEUTRAL | VERY_RECENT] No Summer 2026 leaderboard rankings or scores were publicly available as of June 24, 2026; only two questions had resolved.\n5. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] Summer 2026 Cup is ongoing; competitors including bots can join at any time and improve over the tournament.\n6. [sq2 | web_search | MODERATE cred 75 | UP | DATED] Mantic raised $4M pre-seed funding with backing from DRW and DeepMind/Anthropic researchers, indicating sustained bot development resources.\n7. [sq2 | web_search | WEAK cred 50 | NEUTRAL | VERY_RECENT] No direct confirmation that Preseen-Chestnut or ManticAI is entered in the Summer 2026 Cup was found.\n8. [sq3 | web_search | MODERATE cred 80 | NEUTRAL | RECENT] Summer 2026 features journalist Dylan Matthews as a guest competitor with $2,500 bonus pool if Community beats him.\n9. [sq3 | web_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] No evidence found that the Dylan Matthews competition is drawing exceptionally strong elite human forecasters into the Cup.\n10. [sq4 | web_search | MODERATE cred 70 | UP | RECENT] Metaculus was preparing a $50,000 Summer Bot Tournament and runs Bi-Weekly MiniBench, providing fast feedback loops for bot improvement.\n11. [sq4 | web_search | MODERATE cred 75 | UP | DATED] Bot performance rose from ~80% of top human average (Fall 2025) toward parity, showing continued capability gains.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - No confirmation of which strong bots are entered in Summer 2026\n - No Summer 2026 leaderboard scores yet (only 2 questions resolved)\n - No base rate on edition-to-edition gap closure magnitude\n - Unknown strength of Summer 2026 top human field\n\nKey uncertainties:\n - Whether bot improvement trend continues or plateaus\n - Variance from small number of resolved questions in a 4-month tournament\n - Whether an exceptional human forecaster dominates the field\n - Whether top bots are active and tuned this edition",
"forecast_prompt": "You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates \u2014 your job is to give the most accurate per-component probabilities.\n\n## Question\nWill a bot beat all humans in the Summer 2026 Metaculus Cup?\n\n## Description / Resolution Criteria\n## Description\nThe Metaculus Cup is a 4-month forecasting tournament with a variety of topics and 3-4 questions launching every week. Bots have been steadily improving through recent iterations of the Cup. ManticAI placed 8th in the Summer 2025 edition and 4th in the Fall 2025 edition, even beating the Community Prediction in the latter. In the Spring 2026 edition, ChestnutForty (now known as Preseen-Chestnut) came 3rd, handily beating the Community Prediction and coming within 100 points of beating all humans. Chestnut scored 1059.8, while tejvn, the top human, scored 1156.6.\n\nThe Summer 2026 edition features a competition against prominent forecasting-adjacent journalist Dylan Matthews, with an extra \\$2500 added to the prize pool if the Community Prediction beats him.\n\nAs of June 9, 2026, only two questions have resolved in the Summer 2026 Cup.\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 43900, \"question_id\": 43926}}`\n\n## Resolution Criteria\nThis question will resolve as Yes if after, all questions except this one and [this question](https://www.metaculus.com/questions/43367/community-beats-dylan-matthews-in-summer-2026-cup/) (which resolves last) have resolved, a bot has a Total Score greater than any human forecaster on the Summer 2026 Metaculus Cup leaderboard.\n\n## Fine Print\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/43900) which opened on 2026-06-22 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.\n\n## Sub-question decomposition\n- (w=0.40) Will the best-performing bot\u0027s relative improvement over the top human continue its upward trend (i.e., closing the ~100 point gap from Spring 2026)? \u2014 The core trend: bots went from 8th (Summer 2025) to 4th (Fall 2025) to 3rd within 100 points (Spring 2026). Continued tr\n- (w=0.25) Will a strong bot (e.g., Preseen-Chestnut, ManticAI, or similar) be entered and active in the Summer 2026 Metaculus Cup? \u2014 A bot must actually participate and perform competitively for YES; absence of top bots would lower the probability subst\n- (w=0.20) Will the top human field in Summer 2026 be no stronger than prior editions (no exceptional human outperformance, e.g., from Dylan Matthews competition drawing elite forecasters)? \u2014 The Dylan Matthews competition and added prize pool may attract stronger humans, raising the bar a bot must clear.\n- (w=0.15) Will LLM/forecasting-bot capabilities have improved generally between Spring 2026 and Summer 2026? \u2014 Background rate of AI forecasting improvement supports the gap closing; rapid model releases favor bots.\n\nCombination rule: **weighted_average**\n\n## Synthesized evidence\n1. [sq1 | web_search | STRONG cred 85 | UP | DATED] In Spring 2026 Cup, Chestnut scored 1059.8 vs top human tejvn 1156.6, a ~97 point gap, after placing 3rd and beating Community Prediction.\n2. [sq1 | web_search | STRONG cred 85 | UP | DATED] Bots improved across editions: ManticAI placed 8th (Summer 2025) then 4th (Fall 2025), then Chestnut 3rd (Spring 2026).\n3. [sq1 | code_execution | WEAK cred 35 | NEUTRAL | VERY_RECENT] Trajectory-based model estimates P(bot beats all humans) at ~0.37 given the ~97 point Spring 2026 gap.\n4. [sq1 | web_search | MODERATE cred 70 | NEUTRAL | VERY_RECENT] No Summer 2026 leaderboard rankings or scores were publicly available as of June 24, 2026; only two questions had resolved.\n5. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] Summer 2026 Cup is ongoing; competitors including bots can join at any time and improve over the tournament.\n6. [sq2 | web_search | MODERATE cred 75 | UP | DATED] Mantic raised $4M pre-seed funding with backing from DRW and DeepMind/Anthropic researchers, indicating sustained bot development resources.\n7. [sq2 | web_search | WEAK cred 50 | NEUTRAL | VERY_RECENT] No direct confirmation that Preseen-Chestnut or ManticAI is entered in the Summer 2026 Cup was found.\n8. [sq3 | web_search | MODERATE cred 80 | NEUTRAL | RECENT] Summer 2026 features journalist Dylan Matthews as a guest competitor with $2,500 bonus pool if Community beats him.\n9. [sq3 | web_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] No evidence found that the Dylan Matthews competition is drawing exceptionally strong elite human forecasters into the Cup.\n10. [sq4 | web_search | MODERATE cred 70 | UP | RECENT] Metaculus was preparing a $50,000 Summer Bot Tournament and runs Bi-Weekly MiniBench, providing fast feedback loops for bot improvement.\n11. [sq4 | web_search | MODERATE cred 75 | UP | DATED] Bot performance rose from ~80% of top human average (Fall 2025) toward parity, showing continued capability gains.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - No confirmation of which strong bots are entered in Summer 2026\n - No Summer 2026 leaderboard scores yet (only 2 questions resolved)\n - No base rate on edition-to-edition gap closure magnitude\n - Unknown strength of Summer 2026 top human field\n\nKey uncertainties:\n - Whether bot improvement trend continues or plateaus\n - Variance from small number of resolved questions in a 4-month tournament\n - Whether an exceptional human forecaster dominates the field\n - Whether top bots are active and tuned this edition\n\n## Required pre-forecast walkthrough\n\nBefore giving probabilities, walk through these explicitly:\n (a) The time left until the question resolves.\n (b) The status quo outcome \u2014 what happens if nothing changes from today.\n (c) A brief scenario that results in NO.\n (d) A brief scenario that results in YES.\n\n## Calibration guidance\n\n- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition \u2014 independence forces more careful reasoning per factor.\n- **Anchor on the status quo.** Require strong specific evidence to move far from it.\n- 0.05 / 0.95 require strong, multi-source evidence. Avoid \u003e0.97 or \u003c0.03 unless the outcome is already settled or near-tautological \u2014 overconfident misses are punished heavily by the log score.\n- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question \u2014 move further only with specific evidence the market lacks.\n- **Treat research as fallible, not ground truth.** A single-source or \"very recent\" claim \u2014 especially one the evidence flags as unverified, possibly AI-generated, or low-credibility \u2014 must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.\n- **Also provide a holistic estimate** \u2014 your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.\n\n## Output\n\nReturn ONLY valid JSON, no markdown fences:\n\n{\n \"rationale\": \"\u003caddress (a) (b) (c) (d) above \u2014 5-8 sentences total\u003e\",\n \"sub_question_estimates\": {\n \"sq1\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq2\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq3\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq4\": \u003cfloat in [0.01, 0.99]\u003e\n },\n \"holistic_p_yes\": \u003cfloat in [0.01, 0.99] \u2014 your overall estimate ignoring the decomposition\u003e,\n \"what_would_change_my_mind\": \"\u003c1-2 sentences: what new info would push you above 70% or below 30%\u003e\"\n}\n",
"holistic_p_yes": 0.37,
"models": [
"opus"
],
"p_yes": 0.5217999999999999,
"rationale": "(a) Resolution comes after nearly all Summer 2026 Cup questions resolve, roughly 4 months out from the June 2026 start, so most outcomes are still undetermined with only 2 questions resolved. (b) Status quo trend: bots improved steadily (8th\u21924th\u21923rd, closing to within ~97 points), so a continuation would put a top bot near or at the top, but beating ALL humans remains a step beyond the best prior result. (c) NO scenario: the small-sample variance of a 4-month tournament, an exceptional human forecaster (possibly drawn by the Dylan Matthews competition and bonus pool), or a plateau in bot tuning leaves the top bot again 50-150 points short. (d) YES scenario: a well-funded, well-tuned bot like Preseen-Chestnut or ManticAI enters, benefits from continued LLM gains, and edges past a normal-strength human field, closing the modest remaining gap. The trajectory model\u0027s ~0.37 and the unbeaten-but-closing record anchor my holistic estimate slightly below even odds.",
"sub_question_estimates": {
"sq1": 0.4,
"sq2": 0.85,
"sq3": 0.6,
"sq4": 0.87
},
"what_would_change_my_mind": "Confirmation that a top-tier tuned bot is actively entered with early leaderboard dominance would push me above 70%; evidence of an elite human field assembling (or top bots absent/untuned) would push me below 30%."
},
"plan": {
"combination_logic": "weighted_average",
"domain": "tech",
"n_sub_qs": 4,
"n_tools": 3,
"reasoning_approach": "Weight the dominant trend sub-question (sq1) most heavily, blending in bot participation (sq2), human field strength (sq3), and general AI capability gains (sq4) via weighted average to reflect their correlated dependence on the same underlying bot-improvement momentum.",
"sub_questions": [
{
"id": "sq1",
"question": "Will the best-performing bot\u0027s relative improvement over the top human continue its upward trend (i.e., closing the ~100 point gap from Spring 2026)?",
"rationale": "The core trend: bots went from 8th (Summer 2025) to 4th (Fall 2025) to 3rd within 100 points (Spring 2026). Continued trajectory directly implies a bot could overtake all humans.",
"weight": 0.4
},
{
"id": "sq2",
"question": "Will a strong bot (e.g., Preseen-Chestnut, ManticAI, or similar) be entered and active in the Summer 2026 Metaculus Cup?",
"rationale": "A bot must actually participate and perform competitively for YES; absence of top bots would lower the probability substantially.",
"weight": 0.25
},
{
"id": "sq3",
"question": "Will the top human field in Summer 2026 be no stronger than prior editions (no exceptional human outperformance, e.g., from Dylan Matthews competition drawing elite forecasters)?",
"rationale": "The Dylan Matthews competition and added prize pool may attract stronger humans, raising the bar a bot must clear.",
"weight": 0.2
},
{
"id": "sq4",
"question": "Will LLM/forecasting-bot capabilities have improved generally between Spring 2026 and Summer 2026?",
"rationale": "Background rate of AI forecasting improvement supports the gap closing; rapid model releases favor bots.",
"weight": 0.15
}
],
"tool_requests": [
{
"parameters": {
"brief": "Find the latest standings, leaderboard, and bot performance in the Summer 2026 Metaculus Cup, including how bots like Preseen-Chestnut/ChestnutForty and ManticAI are performing relative to top human forecasters, and whether top bots are entered. Also look for the Dylan Matthews competition and any analysis of bots beating humans in Metaculus tournaments.",
"max_searches": 4,
"question_title": "Will a bot beat all humans in the Summer 2026 Metaculus Cup?"
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3",
"sq4"
],
"tool_name": "claude_news"
},
{
"parameters": {
"queries": [
"Metaculus Cup Summer 2026 leaderboard bot standings",
"Preseen-Chestnut ManticAI Metaculus bot ranking 2026",
"Metaculus AI bot beats humans tournament 2026"
]
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3"
],
"tool_name": "web_search"
},
{
"parameters": {
"code": "import numpy as np\n# Bot top-rank trajectory across editions\n# Summer 2025: 8th; Fall 2025: 4th (beat CP); Spring 2026: 3rd, within 100 pts (1059.8 vs 1156.6)\n# Estimate probability bot overtakes top human in Summer 2026\ngap_spring = 1156.6 - 1059.8 # ~96.8 points\nprint(\u0027Spring 2026 gap:\u0027, gap_spring)\n# Trend in placement: 8 -\u003e 4 -\u003e 3 (diminishing improvement near top)\n# Model gap as shrinking; simulate Summer 2026 outcome\nnp.random.seed(0)\n# Assume gap distribution centered near 0 with wide spread given improvement trend\n# Mean gap drift: improving ~50-100 pts per edition early, slowing near top\nmean_gap = 96.8 * 0.4 # assume ~60% closure of gap (~39 pts remaining mean)\nsd = 120 # high variance in tournament scoring\nsims = np.random.normal(mean_gap, sd, 100000)\np_bot_wins = np.mean(sims \u003c 0)\nprint(\u0027Estimated P(bot beats all humans):\u0027, round(p_bot_wins,3))",
"description": "Estimate probability a bot overtakes top human based on placement/score trajectory across recent Metaculus Cup editions."
},
"target_sub_questions": [
"sq1",
"sq4"
],
"tool_name": "code_execution"
}
]
},
"question": {
"close_time": "2026-06-25T00:41:00Z",
"description": "## Description\nThe Metaculus Cup is a 4-month forecasting tournament with a variety of topics and 3-4 questions launching every week. Bots have been steadily improving through recent iterations of the Cup. ManticAI placed 8th in the Summer 2025 edition and 4th in the Fall 2025 edition, even beating the Community Prediction in the latter. In the Spring 2026 edition, ChestnutForty (now known as Preseen-Chestnut) came 3rd, handily beating the Community Prediction and coming within 100 points of beating all humans. Chestnut scored 1059.8, while tejvn, the top human, scored 1156.6.\n\nThe Summer 2026 edition features a competition against prominent forecasting-adjacent journalist Dylan Matthews, with an extra \\$2500 added to the prize pool if the Community Prediction beats him.\n\nAs of June 9, 2026, only two questions have resolved in the Summer 2026 Cup.\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 43900, \"question_id\": 43926}}`\n\n## Resolution Criteria\nThis question will resolve as Yes if after, all questions except this one and [this question](https://www.metaculus.com/questions/43367/community-beats-dylan-matthews-in-summer-2026-cup/) (which resolves last) have resolved, a bot has a Total Score greater than any human forecaster on the Summer 2026 Metaculus Cup leaderboard.\n\n## Fine Print\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/43900) which opened on 2026-06-22 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.",
"question_type": "binary",
"title": "Will a bot beat all humans in the Summer 2026 Metaculus Cup?"
},
"research": {
"cross_market_brief": "## Cross-Market Signals\n\n### No signal found",
"errors": [],
"has_cross_market": true,
"n_errors": 0,
"n_tools": 3,
"tools": [
{
"elapsed_s": 32.91,
"error": null,
"success": true,
"summary": "10 citations",
"tool_name": "claude_news"
},
{
"elapsed_s": 49.66,
"error": null,
"success": true,
"summary": "0 results",
"tool_name": "web_search"
},
{
"elapsed_s": 0.08,
"error": null,
"success": true,
"summary": "Estimate probability a bot overtakes top human based on plac",
"tool_name": "code_execution"
}
]
},
"synthesis": {
"evidence": [
{
"claim": "In Spring 2026 Cup, Chestnut scored 1059.8 vs top human tejvn 1156.6, a ~97 point gap, after placing 3rd and beating Community Prediction.",
"credibility": 85,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq1"
},
{
"claim": "Bots improved across editions: ManticAI placed 8th (Summer 2025) then 4th (Fall 2025), then Chestnut 3rd (Spring 2026).",
"credibility": 85,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq1"
},
{
"claim": "Trajectory-based model estimates P(bot beats all humans) at ~0.37 given the ~97 point Spring 2026 gap.",
"credibility": 35,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "code_execution",
"strength": "WEAK",
"sub_question_id": "sq1"
},
{
"claim": "No Summer 2026 leaderboard rankings or scores were publicly available as of June 24, 2026; only two questions had resolved.",
"credibility": 70,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq1"
},
{
"claim": "Summer 2026 Cup is ongoing; competitors including bots can join at any time and improve over the tournament.",
"credibility": 70,
"direction": "UP",
"priced_in": true,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "Mantic raised $4M pre-seed funding with backing from DRW and DeepMind/Anthropic researchers, indicating sustained bot development resources.",
"credibility": 75,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "No direct confirmation that Preseen-Chestnut or ManticAI is entered in the Summer 2026 Cup was found.",
"credibility": 50,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq2"
},
{
"claim": "Summer 2026 features journalist Dylan Matthews as a guest competitor with $2,500 bonus pool if Community beats him.",
"credibility": 80,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq3"
},
{
"claim": "No evidence found that the Dylan Matthews competition is drawing exceptionally strong elite human forecasters into the Cup.",
"credibility": 45,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq3"
},
{
"claim": "Metaculus was preparing a $50,000 Summer Bot Tournament and runs Bi-Weekly MiniBench, providing fast feedback loops for bot improvement.",
"credibility": 70,
"direction": "UP",
"priced_in": true,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq4"
},
{
"claim": "Bot performance rose from ~80% of top human average (Fall 2025) toward parity, showing continued capability gains.",
"credibility": 75,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq4"
}
],
"information_gaps": [
"No confirmation of which strong bots are entered in Summer 2026",
"No Summer 2026 leaderboard scores yet (only 2 questions resolved)",
"No base rate on edition-to-edition gap closure magnitude",
"Unknown strength of Summer 2026 top human field"
],
"key_uncertainties": [
"Whether bot improvement trend continues or plateaus",
"Variance from small number of resolved questions in a 4-month tournament",
"Whether an exceptional human forecaster dominates the field",
"Whether top bots are active and tuned this edition"
],
"n_evidence": 11
},
"timings": {
"forecast": 18.5,
"plan": 20.82,
"research": 49.66,
"synthesis": 19.01
}
}