← back to scans
Will the next Claude Opus model debut with a Humanity’s Last Exam score of 60% or higher?
12%
Agent
10%
Market Price
+2.3%
Edge
54%
Confidence
38,029
Volume
1.1c
Spread
95
Days to resolution
4
Markets in event
final rationale
Mean of 5 draws (0.12, 0.12, 0.10, 0.12, 0.14). (0) No source shows an Opus 5.5 card on the agi.safe.ai chart yet: the chart did not render in the fetch, the latest news is HLE-Diamond on 09/22, and sibling markets are still uncertain (55%+ at 51.5%). Resolution requires the site to add Opus 5.5 and display at least 60.0% HLE Accuracy at noon ET the next day. (a) About three months remain until 2026-12-31. (b) Status quo: if no Opus is added the market resolves No; realistically Opus 5.5 is added soon and its displayed score decides it. The Polymarket ladder (60%+ at 9.65%, 55%+ at 51.5%, 50%+ at 88.7%) implies traders expect an official score around 55-59%, and the fall from a 70% high suggests they learned the site's numbers run below vendor figures. (c) NO: the site adds Opus 5.5 at 56-59%, near Fable 5.1's level on its lower CAIS/Scale-style grading (46.5% on the Scale Labs board, 59.1% on other trackers, with Scale Labs running 15-20 points below vendor claims). (d) YES: the site mirrors a no-tools figure like Artificial Analysis's 61.4% or Anthropic's 64.4%. The prior Opus 5 debut clearing 50%+ fits an Artificial Analysis-like 54.9%, which justifies a slight upward shade from the market. Confidence is moderate because the official chart's methodology and score were not observed.
price vs. prediction
market price
agent forecast at t+0
22 follow-ups
1. Planner Opus 5.5 · picks the research tools
Tools chosen:
polymarket_direct
resolution_source
polymarket_related
kalshi_related
wikipedia
Planner reasoning
The Polymarket price is the anchor, and the agi.safe.ai page is the source that decides resolution: it shows current HLE scores, including the latest Opus score and what frontier models reach. Related markets on both venues (HLE thresholds, Claude release timing) give cross-market signals, and Wikipedia covers the background on Claude models.
2. Research parallel tool calls + 4 web research loops (Sonnet 5) · read raw by the forecasters
at a glance
RESEARCH DIGEST (built in code: tool summaries and each report's bottom line; the forecasters read the full tool outputs and reports)
Tools
- polymarket_direct: ## This Market's Polymarket Data **Will the next Claude Opus model debut with a Humanity’s Last Exam score of 60% or higher?** - Current price (probability): 9.65% (live; bid 9% / ask 10%) - 7-day price change: +4.40% - 30-day price change: -23.35% - Total volume: $38,029 (USD notional) - Price range: 1.25% - 70.50% - Data points: 42 days
- resolution_source: already_resolved=unclear; matches_criteria=True; last_updated=09/22/2026. The fetched page shows only general HLE site navigation, news updates (HLE-Diamond release, Nature publication, HLE-Rolling fork), and a long list of dataset contributors. No model leaderboard entries, no Claude Opus model, and no HLE Accuracy percentages are visible in the fetched text.
- polymarket_related: Polymarket search over 3 keyword(s), kept 14 matches. keyword "Humanity's Last Exam": 5 markets | keyword 'Claude Opus': 4 markets | keyword 'Anthropic': 5 markets
- kalshi_related: 2 related markets / summaries. keyword "Humanity's Last Exam": ok | keyword 'Claude': no matches | keyword 'Anthropic model': ok
- wikipedia: Fetched 2 Wikipedia entries (0 missing pages).
Research loops, bottom lines
- current state: As of 2026-09-27, Claude Opus 5.5 (released 2026-09-22) is the strongest candidate to be the qualifying "next Opus" model, and every independent or vendor-reported HLE figure found (61.4% no-tools per Artificial Analysis; 67.7% with tools per Anthropic's own system card; ~63–65% in various trackers) sits at or above the 60% threshold, with related Polymarket sub-markets pricing very high confidence (~96%) that even the lower 50%+ threshold clears — suggesting current evidence leans toward a "Yes
- base rate: The outside-view/base-rate reference class — successive Claude Opus debuts on HLE and analogous "Next Opus debut" Polymarket contracts — shows a strong, consistent upward trend (roughly +10–15 points HLE per release cycle through 2025–2026) and a track record where prior editions of this exact market type resolved YES even at demanding thresholds (e.g., 50%+ in the immediately preceding cycle), so the base rate favors continuation of the trend toward, and plausibly above, 60% for the next debut,
- yes case: The single most likely path to YES is straightforward and largely mechanical at this point: Claude Opus 5.5 (released 2026-09-22) has already posted HLE scores of 61.4% (independent, no-tools, Artificial Analysis) and 64.4% (Anthropic's own no-tools system-card figure), both above 60%, with two independent sources converging on 61.4% and adjacent Polymarket sub-markets (55%+ at 88%, 45%+ at 96%) pricing high confidence in strong performance — so YES resolution mainly requires only that agi.safe.
- no case: The market's resolution source (agi.safe.ai) is affiliated with a documented, independently-run HLE grading pipeline (via Scale AI/CAIS) that historically scores models far below vendor self-reports and third-party "adaptive/max-effort" trackers, has not yet listed any Opus 5/5.5 entry in the version accessible for this research, and shows no "latest news" confirmation of a new Opus addition — so there is a real, evidence-backed risk that either no qualifying Opus model is added to the official
tools
| tool | status | time | items | summary |
|---|---|---|---|---|
| polymarket_direct | OK | 0.4s | 1 | ## This Market's Polymarket Data **Will the next Claude Opus model debut with a Humanity’s Last Exam score of 60% or higher?** - Current price (probability): 9.65% (live; bid 9% / ask 10%) - 7-day price change: +4.40% - 30-day price change: -23.35% - Total volume: $38,029 (USD notional) - Price ran |
| resolution_source | OK | 4.4s | 1 | already_resolved=unclear; matches_criteria=True; last_updated=09/22/2026. The fetched page shows only general HLE site navigation, news updates (HLE-Diamond release, Nature publication, HLE-Rolling fork), and a long list of dataset contributors. No model leaderboard entries, no Claude Opus model, an |
| polymarket_related | OK | 0.5s | 14 | Polymarket search over 3 keyword(s), kept 14 matches. keyword "Humanity's Last Exam": 5 markets | keyword 'Claude Opus': 4 markets | keyword 'Anthropic': 5 markets |
| kalshi_related | OK | 3.4s | 2 | 2 related markets / summaries. keyword "Humanity's Last Exam": ok | keyword 'Claude': no matches | keyword 'Anthropic model': ok |
| wikipedia | OK | 0.2s | 2 | Fetched 2 Wikipedia entries (0 missing pages). |
research loops
·
4/4 ok ·
18 searches ·
8 page reads
| dimension | status | time | searches | page reads | report |
|---|---|---|---|---|---|
| current state | OK | 60s | 7 | 2 | 5599 chars |
| base rate | OK | 62s | 5 | 2 | 5222 chars |
| yes case | OK | 36s | 2 | 2 | 5044 chars |
| no case | OK | 61s | 4 | 2 | 5439 chars |
report: current state
- [2026-09-22] Anthropic released **Claude Opus 5.5**, the first model in its new 5.5 family, positioned as the likely "next Claude Opus" that would qualify for this market (opened 2026-08-17, after Claude Opus 5's July 24 debut).
Anthropic introduced Claude Opus 5.5, the first release in a new family of Claude 5.5 models, that performs at the level of Claude Fable 5.1 on most work, and costs around 40% less to run than Opus 5.
Source: https://sdtimes.com/ai-models/claude-opus-5-5-keeping-safety-ahead-of-capabilities/
- [2026-09-22] Anthropic's own launch materials (system-card benchmark table) report Opus 5.5's HLE score **"with tools" at 67.7%**, ahead of Fable 5.1 (65.6%) and Opus 5 (63.6%).
67.7% with tools, ahead of Fable 5.1 (65.6%) and Opus 5 (63.6%), and 10.5 points ahead of Astra's 57.2%.
Source: https://codingfleet.com/blog/claude-opus-5-5-review/
- [2026-09-24] Independent evaluator Artificial Analysis, running its own standardized (no-tools, max-effort) HLE protocol, put Opus 5.5 at **61.4%**, above Fable 5.1's prior best of 59.1%.
Opus 5.5 leads six of the ten evaluations that make up the index, including Humanity's Last Exam (61.4 percent, up from the previous best of 59.1 percent by Claude Fable 5.1)
. Source: https://www.trendingtopics.eu/claude-opus-5-5-anthropic-launches-new-top-model-despite-calling-for-ai-slowdown/
- [2026-09-24] Third-party aggregator BenchLM.ai's snapshot (which draws on Artificial Analysis' standardized runs) put "Claude Opus 5" (not yet updated for 5.5 at time of snapshot) at 64.7%, with Fable 5.1 leading at 65%.
As of September 24, 2026, Claude Fable 5.1 leads the HLE leaderboard with 65%, followed by Claude Opus 5 (64.7%) and Claude Mythos 5 (64.5%)
. Source: https://benchlm.ai/benchmarks/hle
- [2026-09-27] The official resolution source, agi.safe.ai (lastexam.ai, run by Center for AI Safety/Scale AI), currently shows its most recent "Latest News" update as the release of a new **HLE-Diamond** sub-benchmark on 09/22/2026, not an explicit confirmation that Claude Opus 5.5 has been added as a scored entry on the official model leaderboard/chart. Source: https://agi.safe.ai/
- [2026-09-27] The official site's homepage content (fetched directly) does not display the live per-model "AI Progress" chart/results table needed to confirm Opus 5.5's official HLE Accuracy number — that data appears to be rendered dynamically and was not retrievable via static fetch. NOT FOUND (see below).
- [~2026-08-09] A related, earlier-vintage Polymarket market on the prior "next Opus" (Claude Opus 5, released July 24, 2026) settled with all thresholds from 35%+ up through 50%+ resolving Yes.
The final snapshot shows 35%+ at 100%, 40%+ at 100%, 45%+ at 100%, and 50%+ at 100%.
Source: https://cryptoslate.com/predictions/market/next-claude-opus-humanitys-last-exam-debut/
- [2026-09-27 snapshot] A currently active companion Polymarket market, "Next Claude Opus Model: Humanity's Last Exam Debut?", shows the 50%+ threshold trading at roughly 96%, implying traders see it as very likely Opus 5.5 (or whichever model is deemed qualifying) cleared 50% on the official leaderboard, though this does not directly confirm the 60%+ threshold. Source: https://polymarket.com/predictions/optimus
- [2026-09-08] Analyst commentary noted Anthropic's Opus release cadence (Opus 4.6 → 4.7 → 4.8 → 5) has been roughly 6–10 weeks apart, and that Opus 5.5 (Sept 22) followed Opus 5 (July 24) by about 8 weeks, consistent with rapid-iteration behavior that continues to push HLE scores upward release over release.
Anthropic released Opus 4.6 on February 5, Opus 4.7 on April 16, Opus 4.8 on May 28 and Opus 5 on July 24. Those gaps were 70, 42 and 57 days.
Source: https://predictfact.net/next-claude-opus-model-text-arena-debut-polymarket-analysis/
- [2026-09-22, related caveat] Third-party benchmarking notes emphasize that Anthropic's own tool-enabled HLE number (67.7%) and various no-tools numbers (61.4%–64.7%, depending on source) diverge meaningfully, and that resolution will hinge on exactly which configuration/number the official agi.safe.ai leaderboard displays for Opus 5.5.
Anthropic's own evaluations place Opus on opposite sides of the decisive threshold depending on tool access. The market's sharp break therefore depends on which testing configuration reaches the official leaderboard.
Source: https://cryptoslate.com/predictions/market/next-claude-opus-humanitys-last-exam-debut/
BOTTOM LINE: As of 2026-09-27, Claude Opus 5.5 (released 2026-09-22) is the strongest candidate to be the qualifying "next Opus" model, and every independent or vendor-reported HLE figure found (61.4% no-tools per Artificial Analysis; 67.7% with tools per Anthropic's own system card; ~63–65% in various trackers) sits at or above the 60% threshold, with related Polymarket sub-markets pricing very high confidence (~96%) that even the lower 50%+ threshold clears — suggesting current evidence leans toward a "Yes" outcome, contingent on confirmation of the specific number the official agi.safe.ai leaderboard ultimately displays for this model.
NOT FOUND: Direct confirmation from the live agi.safe.ai "AI Progress on Humanity's Last Exam" chart/dashboard showing (a) that Claude Opus 5.5 (or another Opus-labeled model) has actually been added as a scored entry, (b) the exact calendar date it first appeared on that specific site, and (c) the precise displayed "HLE Accuracy" percentage used for official resolution — the site's model-results chart is rendered dynamically and was not retrievable via static page fetch.
report: base rate
- [2025-11-24] Claude Opus 4.5 debuted with an HLE score of
30.8% without search and 43.2% with search on Humanity's Last Exam
, source: LessWrong summary of Anthropic's release data (https://www.lesswrong.com/posts/HtdrtF5kcpLtWe5dW/claude-opus-4-5-is-the-best-model-available).
- [2026-08-09] A prior, structurally identical Polymarket market ("Next Claude Opus: Humanity's Last Exam Debut") settled with
the final snapshot shows 35%+ at 100%, 40%+ at 100%, 45%+ at 100%, and 50%+ at 100%
, meaning the immediately preceding Opus release cleared even the 50%+ threshold — evidence that recent Opus HLE debuts have consistently beaten market-set thresholds (https://cryptoslate.com/predictions/market/next-claude-opus-humanitys-last-exam-debut/).
- [2026-01-31] Earlier in the cycle,
recent releases of Claude Opus 4.7 variants have lifted Anthropic's top models to the mid-to-high 30s on Humanity's Last Exam
, showing scores were far below 60% just seven months before the current market's window (https://polymarket.com/event/anthropic-claude-score-on-humanitys-last-exam-by-june-30).
- [2026-09-24] Third-party tracker Artificial Analysis reports
Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback) currently has the highest Humanity's Last Exam score, with a score of 61.4% among models with published Humanity's Last Exam results
(https://artificialanalysis.ai/evaluations/humanitys-last-exam).
- [2026-09-24] By contrast, another aggregator using different (possibly the CAIS official-leaderboard-style) methodology reports the top Anthropic model still under 60%:
the top-scoring model on Humanity's Last Exam is Claude Fable 5.1 at 59.1%, followed by Claude Fable 5 at 55.5% and Claude Opus 5 at 54.9%
(https://pricepertoken.com/leaderboards/benchmark/hle), illustrating substantial cross-source disagreement on exact scores and configurations near the 60% line.
- [2026-09-22] Anthropic confirmed the launch of the model that is likely the "next Opus" relevant to this market:
Anthropic's official launch of Claude Opus 5.5 on September 22 drives the near-certain market consensus, with the model now live across the Claude platform, AWS, Google Cloud, and Azure. The release delivers Fable 5.1-level performance on coding and knowledge work benchmarks at roughly 40% lower cost than Opus 5
(https://polymarket.com/event/next-claude-opus-released-onptptpt-20260915).
- [2026-07-20] Base-rate context on Anthropic's release cadence:
Prediction market traders see a 91% chance that Anthropic releases its next Claude Opus model by the end of July, with odds surging 36 points in recent days
, indicating a historical pattern of rapid, roughly every-4-to-8-week Opus iteration through 2026 (https://ca.finance.yahoo.com/news/traders-bet-anthropic-ship-claude-100100549.html).
- [2026-01-25] For general reference class calibration, HLE scores rose from near-zero at launch to the 40-60% range within about 18 months:
When the benchmark was first released in early 2025, leading AI models scored in the single digits: GPT-4o managed just 2.7%, Claude 3.5 Sonnet reached 4.1%, and OpenAI's o1 model achieved only 8.0%
, and even by
the latest frontier models – GPT-5 Pro, Gemini 3 Pro, and Claude Opus 4.6 – have yet to break 40% on HLE
as of that (October 2025) snapshot — showing an extremely steep capability trajectory afterward.
- [2026-09-08] A related, currently-live Polymarket market on the "next Opus" debut (Text Arena, not HLE) shows traders pricing high confidence in strong performance:
Polymarket prices 1500+ at 65% and 1520+ at 31%
, reflecting a general market prior that new Opus releases usually clear the "current-generation-level" bar comfortably (https://predictfact.net/next-claude-opus-model-text-arena-debut-polymarket-analysis/).
- [2026-09-08] Historically, similar nested "Next Claude Opus" threshold markets on other benchmarks have almost always resolved with the lower/middle thresholds hit at ~100% and only the highest, "stretch" thresholds staying uncertain, consistent with the pattern that Anthropic model debuts tend to modestly-to-clearly beat market-implied central expectations rather than undershoot them.
BOTTOM LINE: The outside-view/base-rate reference class — successive Claude Opus debuts on HLE and analogous "Next Opus debut" Polymarket contracts — shows a strong, consistent upward trend (roughly +10–15 points HLE per release cycle through 2025–2026) and a track record where prior editions of this exact market type resolved YES even at demanding thresholds (e.g., 50%+ in the immediately preceding cycle), so the base rate favors continuation of the trend toward, and plausibly above, 60% for the next debut, though exact resolution is sensitive to which configuration/source the official agi.safe.ai site displays.
NOT FOUND: The actual current HLE Accuracy figure displayed for the newest Opus model directly on the official https://agi.safe.ai/ leaderboard (the site's leaderboard/dashboard content was not retrievable via fetch, only the paper/contributors page); could not confirm whether Claude Opus 5.5 (launched 2026-09-22) has actually been added to that specific official leaderboard yet, nor its official displayed score.
report: yes case
- [2026-09-22] Anthropic's own Opus 5.5 system card reports a **64.4% HLE score without tool use** (distinct from the previously-cited 67.7% with-tools figure), meaning even the tool-free vendor number clears the 60% bar:
Humanity's Last Exam, with tools — 67.7%. Vendor-reported. Fable 5.1 65.6%, Opus 5 63.6%, GPT-6 Astra 57.2%. Anthropic's system card separately reports 64.4% without tools, which is the figure to reach for if you are comparing against a source that does not give its models tool access.
- [2026-09-22] Artificial Analysis's independent, standardized run (max-effort, no tools) puts Opus 5.5 not just above 60% but as the new outright HLE leader across all measured models:
Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback) currently has the highest Humanity's Last Exam score, with a score of 61.4% among models with published Humanity's Last Exam results, followed by Claude Fable 5.1 with a score of 59.1%.
- [2026-09-22] This same evaluator frames the result as part of a broad sweep, not an outlier:
Consistent strong performance, with leading scores on six of the ten Intelligence Index evaluations: Humanity's Last Exam 61.4% (previous best 59.1%, Claude Fable 5.1), SciCode 66.9% (63.1%, Fable 5.1), GDPval-AA v2.1, AA-Briefcase v1.1, AA-Omniscience and AutomationBench-AA.
This cross-benchmark consistency reduces the odds the 61.4% HLE figure is a fluke/measurement artifact.
- [2026-09-22] Independent replication detail:
Humanity's Last Exam — 61.4%. Independent, and the previous best on the same board was 59.1%, held by Claude Fable 5.1.
Two independent write-ups (Artificial Analysis and orcarouter's summary of it) converge on the identical 61.4% figure, adding cross-source corroboration beyond a single press release.
- [2026-09-27 snapshot] A separate, directly relevant companion Polymarket market — "Highest Claude score on Humanity's Last Exam in 2026?" — currently prices the 55%+ threshold at 88%, i.e., traders assign high probability that some Claude model (very plausibly Opus 5.5, the newest and highest-scoring entrant) clears 55% on the official board this year, one data point below but adjacent to the 60% line in this question:
Highest Claude score on Humanity's Last Exam in 2026? Ends in 3 months · 88% 55%+.
- [2026-09-27 snapshot] The nested "Next Claude Opus Model: Humanity's Last Exam Debut?" market itself lists a 45%+ threshold priced at 96% in one live snapshot, consistent with (not contradicting) the colleague-reported 50%+ at ~96%, reinforcing that traders see essentially no chance the debut score falls in the 40s:
Next Claude Opus Model: Humanity's Last Exam Debut? Ends in 3 months · 96% 45%+.
- [2026-09-22] Pricing/positioning context strengthens the "genuine frontier upgrade" narrative rather than a cost-cutting downgrade:
Anthropic has cut Opus pricing to $4/$20 per 1M input/output tokens (Opus 5: $5/$25) and cache reads from $0.50 to $0.20.
A cheaper model that still leads the field on HLE argues against the score being an artifact of a heavier, unrepresentative configuration.
- [2026-09-22] Artificial Analysis's overall composite score for Opus 5.5 is described as a step-change versus prior bests:
At max effort it scores 58 on the Artificial Analysis Intelligence Index, the highest score we have measured by several points.
A broad-based capability jump of this magnitude is mechanistically consistent with the model also being the first to durably clear 60% on HLE specifically.
- [2026-09-27] Some third-party aggregators (modelgrep.com, pricepertoken.com) still show pre-Opus-5.5 leaderboard snapshots topping out at 59.1% (Fable 5.1), which could create ambiguity if the official agi.safe.ai site lags in ingesting the new Opus 5.5 score by the market's 60% resolution numbers — but this is a data-lag/timing risk to the mechanism, not evidence against the underlying capability level.
BOTTOM LINE: The single most likely path to YES is straightforward and largely mechanical at this point: Claude Opus 5.5 (released 2026-09-22) has already posted HLE scores of 61.4% (independent, no-tools, Artificial Analysis) and 64.4% (Anthropic's own no-tools system-card figure), both above 60%, with two independent sources converging on 61.4% and adjacent Polymarket sub-markets (55%+ at 88%, 45%+ at 96%) pricing high confidence in strong performance — so YES resolution mainly requires only that agi.safe.ai add Opus 5.5 (or a similarly-scoring successor) to its official chart before the market's deadline and display a number consistent with these already-published figures.
NOT FOUND: Direct visual/DOM confirmation that Opus 5.5 has been added as a live, dated entry on the official agi.safe.ai "AI Progress on Humanity's Last Exam" chart with an explicit displayed percentage (the site's chart renders dynamically and was not retrievable via fetch); also could not find a Polymarket order-book snapshot with an explicit 60%+ threshold price (only 45%+/50%+/55%+ thresholds were visible in fetched snapshots).
report: no case
- [2026-09-27] The official resolution source's own "Latest News" feed at agi.safe.ai lists only four items — HLE-Diamond (09/22/2026), Nature publication (01/28/2026), the HLE-Rolling fork (10/08/2025), and the 2,500-question finalization (04/03/2025) — with no announcement of any new Anthropic model (Opus 5, Opus 5.5, or otherwise) being added to the results chart, indicating the site's own team has not flagged a fresh Opus entry as of the fetch. Source: https://agi.safe.ai/
- [2026-09-27] The companion Scale AI/CAIS-partnered "Scale Labs" HLE leaderboard — built by the same organizing team credited on agi.safe.ai (Center for AI Safety and Scale AI) — currently tops out with GPT 6 Astra at only 54.80% and Fable 5.1 (xhigh) at 46.50%, and lists no Claude Opus 5 or Opus 5.5 entry at all; the highest-ranked Claude model shown is claude-opus-4-7 at just 36.20%. Source: https://labs.scale.com/leaderboard/humanitys_last_exam
- [2026-09-27] This independently-run methodology (temperature-0, automatic extractor/judge, no tool access, confidence-interval-based ranking) produces scores roughly 15-20 points lower than vendor self-reported or no-tools "adaptive/max-effort" figures for the same benchmark generation, meaning that if the agi.safe.ai chart uses this same conservative independent-evaluation pipeline rather than Anthropic's self-reported or Artificial Analysis's standardized-but-vendor-API numbers, a fresh Opus entry clearing 60% is far from guaranteed. Source: https://labs.scale.com/leaderboard/humanitys_last_exam
- [2026-09-24] Even among third-party trackers that do incorporate newer models, one major aggregator (pricepertoken, sourcing from Artificial Analysis and LayerLens) still had Claude Fable 5.1 at only 59.1% as the top scorer as of Sept 24 — two days after Opus 5.5's launch — with no Opus 5.5 entry yet visible, underscoring that even fast-moving third-party leaderboards can lag a launch by several days or more before a new model's official-style score appears. Source: https://pricepertoken.com/leaderboards/benchmark/hle
- [2026-09-24] A separate tracker (modelgrep, "updated September 2026") likewise still shows Claude Fable 5.1 leading at 59.1%, with Claude Opus 5 (not 5.5) as the top Opus entry at only 54.9%, reinforcing that multiple independent trackers had not yet incorporated Opus 5.5 data days after release. Source: https://modelgrep.com/benchmarks/humanitys-last-exam
- [2026-09-23] A benchmark breakdown piece flags that Anthropic's own system card reports Opus 5.5's HLE score without tools at 64.4%, distinctly lower than the heavily-publicized 67.7% with-tools figure, illustrating that even Anthropic's self-reported "no-tools" number sits closer to the 60% line and is not an overwhelming margin above threshold once the "with tools" boost is excluded. Source: https://llm-stats.com/blog/research/claude-opus-5-5-launch
- [2026-09-23] The same source notes a "harness caveat": Anthropic's self-reported benchmark table figures used no fallback models, meaning safeguard interventions were scored as failures on some metrics, while the vendor claims other numbers "understate" real performance — a reminder that Anthropic's self-reported figures are not straightforwardly comparable to an independent grader's methodology, which is what agi.safe.ai/CAIS uses for its own headline chart. Source: https://llm-stats.com/blog/research/claude-opus-5-5-launch
- [2026-09-24] Independent benchmarker BenchLM explicitly warns that its own HLE ledger "mixed rows reported with tools and without tools" at its September 22 editorial check, showing that configuration/protocol ambiguity around Opus 5.5's HLE score is an acknowledged, unresolved data-quality issue industry-wide, not just a hypothetical risk for this market. Source: https://benchlm.ai/benchmarks/hle
- [2026-09-27] No search or fetch turned up any direct statement, screenshot, or press mention confirming that Claude Opus 5.5 (or any Opus model) has actually appeared as a scored card on the specific "AI Progress on Humanity's Last Exam" chart at agi.safe.ai — all "60%+" claims currently trace to Artificial Analysis, Anthropic's own materials, or secondary aggregators, none of which is the named resolution source.
BOTTOM LINE: The market's resolution source (agi.safe.ai) is affiliated with a documented, independently-run HLE grading pipeline (via Scale AI/CAIS) that historically scores models far below vendor self-reports and third-party "adaptive/max-effort" trackers, has not yet listed any Opus 5/5.5 entry in the version accessible for this research, and shows no "latest news" confirmation of a new Opus addition — so there is a real, evidence-backed risk that either no qualifying Opus model is added to the official chart in time, or that once added under the official/independent grading protocol its displayed score misses the aggressive vendor-reported 61-68% figures and lands nearer to (or below) the 60% line.
NOT FOUND: A live, dynamically-rendered view of the actual agi.safe.ai "AI Progress on Humanity's Last Exam" chart/model cards showing whether Claude Opus 5.5 (or another Opus-labeled model) has been added and what specific "HLE Accuracy" percentage is displayed for it; confirmation of whether agi.safe.ai's displayed chart uses the same conservative Scale Labs/CAIS independent-grading methodology found at labs.scale.com or a different (e.g., vendor-reported) data feed.
3. Forecasters 5 draws · Opus 5.5 + GPT-5.6 Sol · plain mean
claude-opus-5-5
50% conf
| Yes | 12% |
| No | 88% |
(0) No source shows an Opus 5.5 card on the agi.safe.ai chart yet: the chart did not render in the fetch, the latest news is HLE-Diamond on 09/22, and sibling markets are still uncertain (55%+ at 51.5%). Resolution requires the site to add Opus 5.5 and display at least 60.0% HLE Accuracy at noon ET the next day. (a) About three months remain until 2026-12-31. (b) Status quo: if no Opus is added the market resolves No; realistically Opus 5.5 is added soon and its displayed score decides it. The Polymarket ladder (60%+ at 9.65%, 55%+ at 51.5%, 50%+ at 88.7%) implies traders expect an official score around 55-59%, and the fall from a 70% high suggests they learned the site's numbers run below vendor figures. (c) NO: the site adds Opus 5.5 at 56-59%, near Fable 5.1's level on its lower CAIS/Scale-style grading (46.5% on the Scale Labs board, 59.1% on other trackers, with Scale Labs running 15-20 points below vendor claims). (d) YES: the site mirrors a no-tools figure like Artificial Analysis's 61.4% or Anthropic's 64.4%. The prior Opus 5 debut clearing 50%+ fits an Artificial Analysis-like 54.9%, which justifies a slight upward shade from the market. Confidence is moderate because the official chart's methodology and score were not observed.
claude-opus-5-5
45% conf
| Yes | 12% |
| No | 88% |
(0) Resolution check: nothing shows that a new Opus has already appeared on the agi.safe.ai chart. The static fetch could not render the model cards, and the site's news feed mentions no new Opus entry, so the outcome is not yet documented. (a) About three months remain until the Dec 31, 2026 deadline. Claude Opus 5.5, released Sept 22, is almost certainly the model that will qualify once CAIS adds it. (b) Status quo: if nothing is added, the market resolves No. What matters is the number agi.safe.ai displays, not vendor or third-party figures. (c) NO scenario: CAIS runs its own grading pipeline, which has historically scored lower than vendor reports (Scale Labs shows much lower numbers), and posts Opus 5.5 somewhere in the 50s. The sibling markets price this as likely: 50%+ at 89% and 55%+ at 52%, which implies traders expect a debut roughly between 50% and 60%. (d) YES scenario: the site's figures track Artificial Analysis-style numbers. AA puts Opus 5.5 at 61.4% no-tools, and Anthropic's system card reports 64.4% without tools and 67.7% with tools, so the displayed accuracy would clear 60%. I anchor on the 9.65% Polymarket price. The internally consistent threshold ladder suggests traders know the official methodology runs well below vendor figures. I nudge slightly upward because the independent AA figure sits above 60% and it is still uncertain which pipeline the chart uses.
claude-opus-5-5
50% conf
| Yes | 10% |
| No | 90% |
(0) Not resolved. The fetched agi.safe.ai page did not show the dynamically rendered chart, and no source confirms that an Opus model has been added or what score it displays. The market is still trading at around 10% rather than near 0 or 1, which suggests no displayed score is known yet. (a) About 3 months remain until the Dec 31, 2026 deadline; Claude Opus 5.5, released Sept 22, is the obvious candidate. (b) Status quo: Opus 5.5 gets added eventually. The question is whether CAIS's own evaluation shows at least 60%, or whether no Opus is added in time, which resolves No. (c) NO: CAIS grades Opus 5.5 with its own no-tools pipeline and displays something like 52-58%. This is consistent with the sibling ladder (50%+ at ~89%, 55%+ at ~52%), which centres the expected displayed score near 55%. It is also consistent with the Scale Labs board showing far lower scores than vendor figures. (d) YES: the official chart mirrors Artificial Analysis's 61.4% or Anthropic's 64.4% no-tools figure, or a stronger Opus variant is added. The published 61-68% figures come from different configurations than the resolution source. The market fell from ~33% to ~10% over 30 days despite those publicly known numbers, implying traders expect the official grading to land lower. I see no unpriced information strong enough to move far from the ~10% anchor.
gpt-5.6-sol
58% conf
| Yes | 12% |
| No | 88% |
(0) The condition is not documented as already met: the September 27 fetch of agi.safe.ai did not expose any qualifying Opus result card or official HLE Accuracy, so YES still requires a newly added Opus model to display at least 60% on the named chart. (a) Roughly 95 days remain until the December 31, 2026, 11:59 PM ET deadline, leaving substantial time for Opus 5.5 to be added, although its absence five days after release indicates meaningful ingestion or methodology uncertainty. (b) If nothing changes, the market resolves No because no qualifying model is confirmed on the resolution source. (c) No results if Opus 5.5 is never added by the deadline or if CAIS's displayed score follows the more conservative evaluation pattern and lands below 60%; the affiliated Scale Labs leaderboard has materially lower scores and no Opus 5/5.5 entry. (d) Yes results if Opus 5.5 is added and agi.safe.ai displays a score consistent with Anthropic's 64.4% no-tools result or Artificial Analysis's independent 61.4% result. I anchor near Polymarket's 9.65% YES price and move modestly upward because multiple published Opus 5.5 evaluations clear 60%, but the inaccessible official chart and coherent companion pricing—about 51.5% at 55% versus only 9.65% at 60%—suggest the market expects the official score to cluster below this threshold.
gpt-5.6-sol
67% conf
| Yes | 14% |
| No | 86% |
(0) The condition is not confirmed as already met: the September 27 fetch of agi.safe.ai did not expose the dynamic model cards, so there is no resolution-source evidence that Opus 5.5 has appeared there with a displayed HLE Accuracy of at least 60%. (a) Roughly 95 days remain until December 31, 2026, although any qualifying model's decisive reading occurs at noon ET on the calendar day after it first appears. (b) If nothing changes and no qualifying Opus is added by the deadline, the rules resolve the market No. (c) No also results if Opus 5.5 is added but the official CAIS-style evaluation displays under 60%; that is credible because related conservative leaderboards score models materially below Anthropic and Artificial Analysis, while companion Polymarket thresholds imply substantial mass around 55–60%. (d) Yes results if the official chart adds Opus 5.5 and reproduces its published 61.4% independent score or Anthropic's 64.4% no-tools score, or if a later qualifying Opus clears 60%. I anchor on Polymarket's 9.65% Yes price and move modestly upward because several reported Opus 5.5 measurements exceed 60%, but not dramatically because none comes from the named resolution display and the market's coherent threshold ladder likely already incorporates those public figures.
4. Final plain mean of 5 draws
12%
(54% confidence)
Yes: 12%
No: 88%
Mean of 5 draws (0.12, 0.12, 0.10, 0.12, 0.14). (0) No source shows an Opus 5.5 card on the agi.safe.ai chart yet: the chart did not render in the fetch, the latest news is HLE-Diamond on 09/22, and sibling markets are still uncertain (55%+ at 51.5%). Resolution requires the site to add Opus 5.5 and display at least 60.0% HLE Accuracy at noon ET the next day. (a) About three months remain until 2026-12-31. (b) Status quo: if no Opus is added the market resolves No; realistically Opus 5.5 is added soon and its displayed score decides it. The Polymarket ladder (60%+ at 9.65%, 55%+ at 51.5%, 50%+ at 88.7%) implies traders expect an official score around 55-59%, and the fall from a 70% high suggests they learned the site's numbers run below vendor figures. (c) NO: the site adds Opus 5.5 at 56-59%, near Fable 5.1's level on its lower CAIS/Scale-style grading (46.5% on the Scale Labs board, 59.1% on other trackers, with Scale Labs running 15-20 points below vendor claims). (d) YES: the site mirrors a no-tools figure like Artificial Analysis's 61.4% or Anthropic's 64.4%. The prior Opus 5 debut clearing 50%+ fits an Artificial Analysis-like 54.9%, which justifies a slight upward shade from the market. Confidence is moderate because the official chart's methodology and score were not observed.
Pipeline Timing
Total pipeline time: 146.7s
Per-tool research timings shown in the Research section above.