Anchor on the latest observed leaderboard position of the leading bot (sq1) and adjust with the rank-persistence base rate from simulation/history (sq4) plus the small extra path from other bots (sq3), blending these into the central estimate of laertes finishing top 5 (sq2) via weighted average.
## Cross-Market Signals ### No signal found
1. [sq1 | question_description | STRONG cred 90 | UP | VERY_RECENT] Per the question's own background, bot 🤖 laertes held the fifth-highest score on the Summer 2026 Metaculus Cup leaderboard as of July 23, 2026. 2. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] Web searches returned no accessible public leaderboard standings or bot rankings for the Summer 2026 Metaculus Cup beyond the screenshot in the question. 3. [sq4 | web_search | STRONG cred 75 | UP | DATED] The Spring 2026 Metaculus Cup 'Top Bot Rank' question resolved at 3rd place, indicating a bot finished third overall in the immediately preceding Cup season. 4. [sq3 | web_search | MODERATE cred 70 | NEUTRAL | DATED] Metaculus's bot tournament rules let new bots join mid-window starting at the leaderboard middle with 0 points, and bot makers iteratively improve bots during the season. 5. [sq2 | web_search | STRONG cred 85 | NEUTRAL | DATED] Summer 2026 Metaculus Cup has 25 questions, opened May 4 and closes Sep 1, 2026, with a $5,000 prize pool. 6. [sq2 | code_execution | WEAK cred 30 | DOWN | VERY_RECENT] A Monte Carlo simulation of a mid-tournament 5th-place forecaster retaining top-5 gave only 11.5–14.8% depending on fraction resolved, under assumed field size and noise. 7. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] With the Cup running May 4–Sep 1 and the observation dated July 23, roughly two-thirds to three-quarters of the tournament window had elapsed, limiting remaining rank churn. 8. [sq2 | web_search | MODERATE cred 80 | DOWN | DATED] Metaculus states Pro Forecasters have beaten the best custom bots in every FutureEval season so far, and bots significantly underperform Pros head-to-head. 9. [sq4 | web_search | WEAK cred 65 | NEUTRAL | DATED] The Spring 2026 'Top Bot Rank' question drew 106 forecasters and 106 comments, indicating an active community norm of forecasting bot leaderboard placement each season. 10. [sq3 | web_search | MODERATE cred 75 | UP | DATED] A separate $50,000 Summer 2026 FutureEval Bot Tournament runs concurrently with the Cup, incentivizing many bots to forecast on Cup-adjacent questions. 11. [sq1 | article_search | WEAK cred 20 | NEUTRAL | VERY_RECENT] General news article searches surfaced only unrelated content (World Cup soccer, AI commentary) with no coverage of Metaculus leaderboard standings. ## Cross-Market Signals ### No signal found Information gaps: - No current (post-July-23) leaderboard snapshot or laertes score margin over 6th place - No historical base rate of mid-tournament rank persistence in actual Metaculus Cups - Unknown how many Cup questions remain unresolved and their scoring weight - Unknown whether other bots are near the top 5 in Summer 2026 Key uncertainties: - Size of laertes' point cushion vs. 6th-10th place - Whether laertes keeps forecasting through Sep 1 close - Volatility of remaining question resolutions (peer scoring swings) - Whether Metaculus Cup peer scores penalize late-joining or partial coverage
You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates — your job is to give the most accurate per-component probabilities.
## Question
Will a bot finish in the top 5 of the Summer 2026 Metaculus Cup?
## Description / Resolution Criteria
## Description
Though AI bots significantly underperform human Metaculus Pros in head-to-head tournaments such as [FutureEval](https://www.metaculus.com/futureeval/), one bot, 🤖 laertes, as of July 23, 2026, had the fifth highest score on the leaderboard of the [Metaculus Cup Summer 2026](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/): 
`{"format": "metac_reveal_and_close_in_period", "info": {"post_id": 44799, "question_id": 44946}}`
## Resolution Criteria
This question resolves as **Yes** if a bot user earns the fifth highest score or better on the Summer 2026 [Metaculus Cup ](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/)leaderboard.
## Fine Print
This question's information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44799) which opened on 2026-07-24 18:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question's resolution could have been known before this question opened, then this question will be annulled.
## Sub-question decomposition
- (w=0.30) Is a bot (e.g. 🤖 laertes) still ranked in the top 5 of the Summer 2026 Metaculus Cup leaderboard as of the most recent available data (late July / early August 2026)? — Current standing is the strongest single predictor; the question was created because laertes was 5th on July 23, 2026, b
- (w=0.40) Will 🤖 laertes specifically hold a top-5 score at the final close of the Summer 2026 Metaculus Cup (after all questions resolve)? — The main path to YES; requires persistence through remaining question resolutions, which historically erodes bot ranking
- (w=0.15) Will some bot other than laertes reach the top 5 by tournament end? — Provides an alternate path to YES; several bots participate in the Metaculus Cup and one could surge.
- (w=0.15) Is the top-5 composition of Metaculus Cup leaderboards historically stable from mid-tournament to final resolution (i.e., does a mid-tournament 5th place typically hold)? — Base-rate on rank persistence determines how much weight to give the July 23 snapshot; peer-scored tournaments with many
Combination rule: **weighted_average**
## Synthesized evidence
1. [sq1 | question_description | STRONG cred 90 | UP | VERY_RECENT] Per the question's own background, bot 🤖 laertes held the fifth-highest score on the Summer 2026 Metaculus Cup leaderboard as of July 23, 2026.
2. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] Web searches returned no accessible public leaderboard standings or bot rankings for the Summer 2026 Metaculus Cup beyond the screenshot in the question.
3. [sq4 | web_search | STRONG cred 75 | UP | DATED] The Spring 2026 Metaculus Cup 'Top Bot Rank' question resolved at 3rd place, indicating a bot finished third overall in the immediately preceding Cup season.
4. [sq3 | web_search | MODERATE cred 70 | NEUTRAL | DATED] Metaculus's bot tournament rules let new bots join mid-window starting at the leaderboard middle with 0 points, and bot makers iteratively improve bots during the season.
5. [sq2 | web_search | STRONG cred 85 | NEUTRAL | DATED] Summer 2026 Metaculus Cup has 25 questions, opened May 4 and closes Sep 1, 2026, with a $5,000 prize pool.
6. [sq2 | code_execution | WEAK cred 30 | DOWN | VERY_RECENT] A Monte Carlo simulation of a mid-tournament 5th-place forecaster retaining top-5 gave only 11.5–14.8% depending on fraction resolved, under assumed field size and noise.
7. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] With the Cup running May 4–Sep 1 and the observation dated July 23, roughly two-thirds to three-quarters of the tournament window had elapsed, limiting remaining rank churn.
8. [sq2 | web_search | MODERATE cred 80 | DOWN | DATED] Metaculus states Pro Forecasters have beaten the best custom bots in every FutureEval season so far, and bots significantly underperform Pros head-to-head.
9. [sq4 | web_search | WEAK cred 65 | NEUTRAL | DATED] The Spring 2026 'Top Bot Rank' question drew 106 forecasters and 106 comments, indicating an active community norm of forecasting bot leaderboard placement each season.
10. [sq3 | web_search | MODERATE cred 75 | UP | DATED] A separate $50,000 Summer 2026 FutureEval Bot Tournament runs concurrently with the Cup, incentivizing many bots to forecast on Cup-adjacent questions.
11. [sq1 | article_search | WEAK cred 20 | NEUTRAL | VERY_RECENT] General news article searches surfaced only unrelated content (World Cup soccer, AI commentary) with no coverage of Metaculus leaderboard standings.
## Cross-Market Signals
### No signal found
Information gaps:
- No current (post-July-23) leaderboard snapshot or laertes score margin over 6th place
- No historical base rate of mid-tournament rank persistence in actual Metaculus Cups
- Unknown how many Cup questions remain unresolved and their scoring weight
- Unknown whether other bots are near the top 5 in Summer 2026
Key uncertainties:
- Size of laertes' point cushion vs. 6th-10th place
- Whether laertes keeps forecasting through Sep 1 close
- Volatility of remaining question resolutions (peer scoring swings)
- Whether Metaculus Cup peer scores penalize late-joining or partial coverage
## Required pre-forecast walkthrough
Before giving probabilities, walk through these explicitly:
(a) The time left until the question resolves.
(b) The status quo outcome — what happens if nothing changes from today.
(c) A brief scenario that results in NO.
(d) A brief scenario that results in YES.
## Calibration guidance
- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition — independence forces more careful reasoning per factor.
- **Anchor on the status quo.** Require strong specific evidence to move far from it.
- 0.05 / 0.95 require strong, multi-source evidence. Avoid >0.97 or <0.03 unless the outcome is already settled or near-tautological — overconfident misses are punished heavily by the log score.
- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question — move further only with specific evidence the market lacks.
- **Treat research as fallible, not ground truth.** A single-source or "very recent" claim — especially one the evidence flags as unverified, possibly AI-generated, or low-credibility — must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.
- **Also provide a holistic estimate** — your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.
## Output
Return ONLY valid JSON, no markdown fences:
{
"rationale": "<address (a) (b) (c) (d) above — 5-8 sentences total>",
"sub_question_estimates": {
"sq1": <float in [0.01, 0.99]>,
"sq2": <float in [0.01, 0.99]>,
"sq3": <float in [0.01, 0.99]>,
"sq4": <float in [0.01, 0.99]>
},
"holistic_p_yes": <float in [0.01, 0.99] — your overall estimate ignoring the decomposition>,
"what_would_change_my_mind": "<1-2 sentences: what new info would push you above 70% or below 30%>"
}
{
"forecast": {
"combination_logic": "weighted_average",
"evidence_brief": "1. [sq1 | question_description | STRONG cred 90 | UP | VERY_RECENT] Per the question\u0027s own background, bot \ud83e\udd16 laertes held the fifth-highest score on the Summer 2026 Metaculus Cup leaderboard as of July 23, 2026.\n2. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] Web searches returned no accessible public leaderboard standings or bot rankings for the Summer 2026 Metaculus Cup beyond the screenshot in the question.\n3. [sq4 | web_search | STRONG cred 75 | UP | DATED] The Spring 2026 Metaculus Cup \u0027Top Bot Rank\u0027 question resolved at 3rd place, indicating a bot finished third overall in the immediately preceding Cup season.\n4. [sq3 | web_search | MODERATE cred 70 | NEUTRAL | DATED] Metaculus\u0027s bot tournament rules let new bots join mid-window starting at the leaderboard middle with 0 points, and bot makers iteratively improve bots during the season.\n5. [sq2 | web_search | STRONG cred 85 | NEUTRAL | DATED] Summer 2026 Metaculus Cup has 25 questions, opened May 4 and closes Sep 1, 2026, with a $5,000 prize pool.\n6. [sq2 | code_execution | WEAK cred 30 | DOWN | VERY_RECENT] A Monte Carlo simulation of a mid-tournament 5th-place forecaster retaining top-5 gave only 11.5\u201314.8% depending on fraction resolved, under assumed field size and noise.\n7. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] With the Cup running May 4\u2013Sep 1 and the observation dated July 23, roughly two-thirds to three-quarters of the tournament window had elapsed, limiting remaining rank churn.\n8. [sq2 | web_search | MODERATE cred 80 | DOWN | DATED] Metaculus states Pro Forecasters have beaten the best custom bots in every FutureEval season so far, and bots significantly underperform Pros head-to-head.\n9. [sq4 | web_search | WEAK cred 65 | NEUTRAL | DATED] The Spring 2026 \u0027Top Bot Rank\u0027 question drew 106 forecasters and 106 comments, indicating an active community norm of forecasting bot leaderboard placement each season.\n10. [sq3 | web_search | MODERATE cred 75 | UP | DATED] A separate $50,000 Summer 2026 FutureEval Bot Tournament runs concurrently with the Cup, incentivizing many bots to forecast on Cup-adjacent questions.\n11. [sq1 | article_search | WEAK cred 20 | NEUTRAL | VERY_RECENT] General news article searches surfaced only unrelated content (World Cup soccer, AI commentary) with no coverage of Metaculus leaderboard standings.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - No current (post-July-23) leaderboard snapshot or laertes score margin over 6th place\n - No historical base rate of mid-tournament rank persistence in actual Metaculus Cups\n - Unknown how many Cup questions remain unresolved and their scoring weight\n - Unknown whether other bots are near the top 5 in Summer 2026\n\nKey uncertainties:\n - Size of laertes\u0027 point cushion vs. 6th-10th place\n - Whether laertes keeps forecasting through Sep 1 close\n - Volatility of remaining question resolutions (peer scoring swings)\n - Whether Metaculus Cup peer scores penalize late-joining or partial coverage",
"forecast_prompt": "You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates \u2014 your job is to give the most accurate per-component probabilities.\n\n## Question\nWill a bot finish in the top 5 of the Summer 2026 Metaculus Cup?\n\n## Description / Resolution Criteria\n## Description\nThough AI bots significantly underperform human Metaculus Pros in head-to-head tournaments such as [FutureEval](https://www.metaculus.com/futureeval/), one bot, \ud83e\udd16 laertes, as of July 23, 2026, had the fifth highest score on the leaderboard of the [Metaculus Cup Summer 2026](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/):\u0026#x20;\n\n\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 44799, \"question_id\": 44946}}`\n\n## Resolution Criteria\nThis question resolves as **Yes** if a bot user earns the fifth highest score or better on the Summer 2026 [Metaculus Cup ](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/)leaderboard.\n\n## Fine Print\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44799) which opened on 2026-07-24 18:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.\n\n## Sub-question decomposition\n- (w=0.30) Is a bot (e.g. \ud83e\udd16 laertes) still ranked in the top 5 of the Summer 2026 Metaculus Cup leaderboard as of the most recent available data (late July / early August 2026)? \u2014 Current standing is the strongest single predictor; the question was created because laertes was 5th on July 23, 2026, b\n- (w=0.40) Will \ud83e\udd16 laertes specifically hold a top-5 score at the final close of the Summer 2026 Metaculus Cup (after all questions resolve)? \u2014 The main path to YES; requires persistence through remaining question resolutions, which historically erodes bot ranking\n- (w=0.15) Will some bot other than laertes reach the top 5 by tournament end? \u2014 Provides an alternate path to YES; several bots participate in the Metaculus Cup and one could surge.\n- (w=0.15) Is the top-5 composition of Metaculus Cup leaderboards historically stable from mid-tournament to final resolution (i.e., does a mid-tournament 5th place typically hold)? \u2014 Base-rate on rank persistence determines how much weight to give the July 23 snapshot; peer-scored tournaments with many\n\nCombination rule: **weighted_average**\n\n## Synthesized evidence\n1. [sq1 | question_description | STRONG cred 90 | UP | VERY_RECENT] Per the question\u0027s own background, bot \ud83e\udd16 laertes held the fifth-highest score on the Summer 2026 Metaculus Cup leaderboard as of July 23, 2026.\n2. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] Web searches returned no accessible public leaderboard standings or bot rankings for the Summer 2026 Metaculus Cup beyond the screenshot in the question.\n3. [sq4 | web_search | STRONG cred 75 | UP | DATED] The Spring 2026 Metaculus Cup \u0027Top Bot Rank\u0027 question resolved at 3rd place, indicating a bot finished third overall in the immediately preceding Cup season.\n4. [sq3 | web_search | MODERATE cred 70 | NEUTRAL | DATED] Metaculus\u0027s bot tournament rules let new bots join mid-window starting at the leaderboard middle with 0 points, and bot makers iteratively improve bots during the season.\n5. [sq2 | web_search | STRONG cred 85 | NEUTRAL | DATED] Summer 2026 Metaculus Cup has 25 questions, opened May 4 and closes Sep 1, 2026, with a $5,000 prize pool.\n6. [sq2 | code_execution | WEAK cred 30 | DOWN | VERY_RECENT] A Monte Carlo simulation of a mid-tournament 5th-place forecaster retaining top-5 gave only 11.5\u201314.8% depending on fraction resolved, under assumed field size and noise.\n7. [sq2 | web_search | MODERATE cred 70 | UP | VERY_RECENT] With the Cup running May 4\u2013Sep 1 and the observation dated July 23, roughly two-thirds to three-quarters of the tournament window had elapsed, limiting remaining rank churn.\n8. [sq2 | web_search | MODERATE cred 80 | DOWN | DATED] Metaculus states Pro Forecasters have beaten the best custom bots in every FutureEval season so far, and bots significantly underperform Pros head-to-head.\n9. [sq4 | web_search | WEAK cred 65 | NEUTRAL | DATED] The Spring 2026 \u0027Top Bot Rank\u0027 question drew 106 forecasters and 106 comments, indicating an active community norm of forecasting bot leaderboard placement each season.\n10. [sq3 | web_search | MODERATE cred 75 | UP | DATED] A separate $50,000 Summer 2026 FutureEval Bot Tournament runs concurrently with the Cup, incentivizing many bots to forecast on Cup-adjacent questions.\n11. [sq1 | article_search | WEAK cred 20 | NEUTRAL | VERY_RECENT] General news article searches surfaced only unrelated content (World Cup soccer, AI commentary) with no coverage of Metaculus leaderboard standings.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - No current (post-July-23) leaderboard snapshot or laertes score margin over 6th place\n - No historical base rate of mid-tournament rank persistence in actual Metaculus Cups\n - Unknown how many Cup questions remain unresolved and their scoring weight\n - Unknown whether other bots are near the top 5 in Summer 2026\n\nKey uncertainties:\n - Size of laertes\u0027 point cushion vs. 6th-10th place\n - Whether laertes keeps forecasting through Sep 1 close\n - Volatility of remaining question resolutions (peer scoring swings)\n - Whether Metaculus Cup peer scores penalize late-joining or partial coverage\n\n## Required pre-forecast walkthrough\n\nBefore giving probabilities, walk through these explicitly:\n (a) The time left until the question resolves.\n (b) The status quo outcome \u2014 what happens if nothing changes from today.\n (c) A brief scenario that results in NO.\n (d) A brief scenario that results in YES.\n\n## Calibration guidance\n\n- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition \u2014 independence forces more careful reasoning per factor.\n- **Anchor on the status quo.** Require strong specific evidence to move far from it.\n- 0.05 / 0.95 require strong, multi-source evidence. Avoid \u003e0.97 or \u003c0.03 unless the outcome is already settled or near-tautological \u2014 overconfident misses are punished heavily by the log score.\n- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question \u2014 move further only with specific evidence the market lacks.\n- **Treat research as fallible, not ground truth.** A single-source or \"very recent\" claim \u2014 especially one the evidence flags as unverified, possibly AI-generated, or low-credibility \u2014 must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.\n- **Also provide a holistic estimate** \u2014 your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.\n\n## Output\n\nReturn ONLY valid JSON, no markdown fences:\n\n{\n \"rationale\": \"\u003caddress (a) (b) (c) (d) above \u2014 5-8 sentences total\u003e\",\n \"sub_question_estimates\": {\n \"sq1\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq2\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq3\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq4\": \u003cfloat in [0.01, 0.99]\u003e\n },\n \"holistic_p_yes\": \u003cfloat in [0.01, 0.99] \u2014 your overall estimate ignoring the decomposition\u003e,\n \"what_would_change_my_mind\": \"\u003c1-2 sentences: what new info would push you above 70% or below 30%\u003e\"\n}\n",
"holistic_p_yes": 0.58,
"models": [
"opus",
"secondary"
],
"p_yes": 0.50945,
"rationale": "(a) The Cup closes Sep 1, 2026, with resolution of remaining questions and final scoring shortly after \u2014 roughly 5-6 weeks of scoring churn from the July 23 snapshot. (b) Status quo: \ud83e\udd16 laertes sits exactly 5th, so if nothing changes the question resolves YES, but 5th is the marginal boundary and small peer-score swings on remaining resolutions flip it either way. (c) NO scenario: a handful of late-resolving questions go against laertes (bots tend to be poorly calibrated on the idiosyncratic, judgment-heavy questions that dominate Cup tails), one or two humans with strong late accuracy pass it, and laertes drops to 6th-8th; the FutureEval evidence that Pros consistently beat bots supports this drift. (d) YES scenario: laertes\u0027 broad question coverage and consistent participation (bots forecast every question and update frequently, an edge in peer-scored tournaments where coverage matters) hold its cushion, or another bot from the concurrent $50k FutureEval field surges into the top 5 \u2014 notably, the Spring 2026 Cup\u0027s \u0027Top Bot Rank\u0027 question resolved at 3rd, showing a bot beat all but two humans just one season earlier. That prior-season precedent is the single most important piece of evidence: bots in top 5 of a Metaculus Cup is not a fluke but a recurring outcome. Balancing the marginal current position against a demonstrated recent base rate of bots finishing even higher, I land modestly above even odds.",
"sub_question_estimates": {
"sq1": 0.85,
"sq2": 0.47,
"sq3": 0.2,
"sq4": 0.57
},
"what_would_change_my_mind": "A fresh leaderboard snapshot showing laertes\u0027 point margin over 6th place (a large cushion or a clear drop out of the top 5) would move me sharply; likewise evidence that most Cup questions have already resolved (locking in standings) would push me above 70%, while news that laertes stopped forecasting or that several late-resolving questions cut against it would push me below 30%."
},
"plan": {
"combination_logic": "weighted_average",
"domain": "tech",
"n_sub_qs": 4,
"n_tools": 4,
"reasoning_approach": "Anchor on the latest observed leaderboard position of the leading bot (sq1) and adjust with the rank-persistence base rate from simulation/history (sq4) plus the small extra path from other bots (sq3), blending these into the central estimate of laertes finishing top 5 (sq2) via weighted average.",
"sub_questions": [
{
"id": "sq1",
"question": "Is a bot (e.g. \ud83e\udd16 laertes) still ranked in the top 5 of the Summer 2026 Metaculus Cup leaderboard as of the most recent available data (late July / early August 2026)?",
"rationale": "Current standing is the strongest single predictor; the question was created because laertes was 5th on July 23, 2026, but weekly reshuffling is common.",
"weight": 0.3
},
{
"id": "sq2",
"question": "Will \ud83e\udd16 laertes specifically hold a top-5 score at the final close of the Summer 2026 Metaculus Cup (after all questions resolve)?",
"rationale": "The main path to YES; requires persistence through remaining question resolutions, which historically erodes bot rankings relative to top humans.",
"weight": 0.4
},
{
"id": "sq3",
"question": "Will some bot other than laertes reach the top 5 by tournament end?",
"rationale": "Provides an alternate path to YES; several bots participate in the Metaculus Cup and one could surge.",
"weight": 0.15
},
{
"id": "sq4",
"question": "Is the top-5 composition of Metaculus Cup leaderboards historically stable from mid-tournament to final resolution (i.e., does a mid-tournament 5th place typically hold)?",
"rationale": "Base-rate on rank persistence determines how much weight to give the July 23 snapshot; peer-scored tournaments with many unresolved questions can shuffle substantially.",
"weight": 0.15
}
],
"tool_requests": [
{
"parameters": {
"queries": [
"Metaculus Cup Summer 2026 leaderboard laertes bot",
"Metaculus Cup Summer 2026 tournament standings top 5",
"Metaculus bot leaderboard Metaculus Cup 2026"
]
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3"
],
"tool_name": "web_search"
},
{
"parameters": {
"brief": "Find the current standings of the Metaculus Cup Summer 2026 tournament leaderboard (metaculus.com/tournament/metaculus-cup-summer-2026/), specifically whether the bot user \u0027\ud83e\udd16 laertes\u0027 or any other bot is in the top 5 by score, its score margin vs nearby humans, the tournament\u0027s end date and how many questions remain unresolved. Also find historical precedent: have bots ever finished top 5 in prior Metaculus Cup seasons (Spring 2026, Winter 2025-26, Fall 2025), and how bot rankings evolved from mid-tournament to final resolution.",
"max_searches": 5,
"question_title": "Will a bot finish in the top 5 of the Summer 2026 Metaculus Cup?"
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3",
"sq4"
],
"tool_name": "claude_news"
},
{
"parameters": {
"lookback_days": 90,
"queries": [
"Metaculus Cup bot forecaster leaderboard",
"AI forecasting bot beats human Metaculus tournament",
"Metaculus AI benchmark bot performance 2026"
]
},
"target_sub_questions": [
"sq3",
"sq4"
],
"tool_name": "article_search"
},
{
"parameters": {
"code": "import numpy as np\nrng=np.random.default_rng(0)\n# Monte Carlo: probability a mid-tournament 5th-place finisher stays in top 5 at close\n# Model: N competitors with latent skill; current score = skill*frac_resolved + noise\nN=250\nsims=40000\nfor frac in [0.4,0.5,0.6,0.7]:\n hold=0\n for s in range(sims):\n skill=rng.normal(0,1,N)\n # current observed score after frac of questions resolved\n cur=skill*frac+rng.normal(0,np.sqrt(frac)*0.8,N)\n order=np.argsort(-cur)\n target=order[4] # currently 5th\n final=skill*1.0+rng.normal(0,0.8,N)\n rank=(final\u003efinal[target]).sum()+1\n hold+= (rank\u003c=5)\n print(frac, round(hold/sims,3))\n",
"description": "Monte Carlo base rate for a mid-tournament 5th-place forecaster retaining a top-5 finish, under varying fractions of the tournament already resolved and noise levels."
},
"target_sub_questions": [
"sq4",
"sq2"
],
"tool_name": "code_execution"
}
]
},
"question": {
"close_time": "2026-08-01T18:00:00Z",
"description": "## Description\nThough AI bots significantly underperform human Metaculus Pros in head-to-head tournaments such as [FutureEval](https://www.metaculus.com/futureeval/), one bot, \ud83e\udd16 laertes, as of July 23, 2026, had the fifth highest score on the leaderboard of the [Metaculus Cup Summer 2026](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/):\u0026#x20;\n\n\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 44799, \"question_id\": 44946}}`\n\n## Resolution Criteria\nThis question resolves as **Yes** if a bot user earns the fifth highest score or better on the Summer 2026 [Metaculus Cup ](https://www.metaculus.com/tournament/metaculus-cup-summer-2026/)leaderboard.\n\n## Fine Print\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44799) which opened on 2026-07-24 18:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.",
"question_type": "binary",
"title": "Will a bot finish in the top 5 of the Summer 2026 Metaculus Cup?"
},
"research": {
"cross_market_brief": "## Cross-Market Signals\n\n### No signal found",
"errors": [],
"has_cross_market": true,
"n_errors": 0,
"n_tools": 4,
"tools": [
{
"elapsed_s": 19.75,
"error": null,
"success": true,
"summary": "0 results",
"tool_name": "web_search"
},
{
"elapsed_s": 32.41,
"error": null,
"success": true,
"summary": "9 citations",
"tool_name": "claude_news"
},
{
"elapsed_s": 0.85,
"error": null,
"success": true,
"summary": "27 articles",
"tool_name": "article_search"
},
{
"elapsed_s": 10.58,
"error": null,
"success": true,
"summary": "Monte Carlo base rate for a mid-tournament 5th-place forecas",
"tool_name": "code_execution"
}
]
},
"synthesis": {
"evidence": [
{
"claim": "Per the question\u0027s own background, bot \ud83e\udd16 laertes held the fifth-highest score on the Summer 2026 Metaculus Cup leaderboard as of July 23, 2026.",
"credibility": 90,
"direction": "UP",
"priced_in": true,
"recency": "VERY_RECENT",
"source": "question_description",
"strength": "STRONG",
"sub_question_id": "sq1"
},
{
"claim": "Web searches returned no accessible public leaderboard standings or bot rankings for the Summer 2026 Metaculus Cup beyond the screenshot in the question.",
"credibility": 40,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq1"
},
{
"claim": "The Spring 2026 Metaculus Cup \u0027Top Bot Rank\u0027 question resolved at 3rd place, indicating a bot finished third overall in the immediately preceding Cup season.",
"credibility": 75,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq4"
},
{
"claim": "Metaculus\u0027s bot tournament rules let new bots join mid-window starting at the leaderboard middle with 0 points, and bot makers iteratively improve bots during the season.",
"credibility": 70,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq3"
},
{
"claim": "Summer 2026 Metaculus Cup has 25 questions, opened May 4 and closes Sep 1, 2026, with a $5,000 prize pool.",
"credibility": 85,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq2"
},
{
"claim": "A Monte Carlo simulation of a mid-tournament 5th-place forecaster retaining top-5 gave only 11.5\u201314.8% depending on fraction resolved, under assumed field size and noise.",
"credibility": 30,
"direction": "DOWN",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "code_execution",
"strength": "WEAK",
"sub_question_id": "sq2"
},
{
"claim": "With the Cup running May 4\u2013Sep 1 and the observation dated July 23, roughly two-thirds to three-quarters of the tournament window had elapsed, limiting remaining rank churn.",
"credibility": 70,
"direction": "UP",
"priced_in": true,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "Metaculus states Pro Forecasters have beaten the best custom bots in every FutureEval season so far, and bots significantly underperform Pros head-to-head.",
"credibility": 80,
"direction": "DOWN",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "The Spring 2026 \u0027Top Bot Rank\u0027 question drew 106 forecasters and 106 comments, indicating an active community norm of forecasting bot leaderboard placement each season.",
"credibility": 65,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq4"
},
{
"claim": "A separate $50,000 Summer 2026 FutureEval Bot Tournament runs concurrently with the Cup, incentivizing many bots to forecast on Cup-adjacent questions.",
"credibility": 75,
"direction": "UP",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq3"
},
{
"claim": "General news article searches surfaced only unrelated content (World Cup soccer, AI commentary) with no coverage of Metaculus leaderboard standings.",
"credibility": 20,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "VERY_RECENT",
"source": "article_search",
"strength": "WEAK",
"sub_question_id": "sq1"
}
],
"information_gaps": [
"No current (post-July-23) leaderboard snapshot or laertes score margin over 6th place",
"No historical base rate of mid-tournament rank persistence in actual Metaculus Cups",
"Unknown how many Cup questions remain unresolved and their scoring weight",
"Unknown whether other bots are near the top 5 in Summer 2026"
],
"key_uncertainties": [
"Size of laertes\u0027 point cushion vs. 6th-10th place",
"Whether laertes keeps forecasting through Sep 1 close",
"Volatility of remaining question resolutions (peer scoring swings)",
"Whether Metaculus Cup peer scores penalize late-joining or partial coverage"
],
"n_evidence": 11
},
"timings": {
"forecast": 47.02,
"plan": 31.03,
"research": 32.42,
"synthesis": 25.97
}
}