Will the highest score achieved by an Anthropic Claude model on Humanity’s Last Exam in 2026 be 60% or higher?

- [2026-10-11] Official Anthropic system card for Claude Opus 5.5 (released 2026-09-22) reports, in its own standardized harness (Table 8.1.A), this system card describes Claude Opus 5.5, the latest Opus-class large language model from Anthropic, an upgrade to Claude Opus 5, with gains in coding, agentic and computer use tasks, mathematical and scientific reasoning, and long-horizon professional work. Specifically for HLE, the card's capability table lists Claude Opus 5.5 at 64.4% without tools and 67.7% with tools (web search, web fetch, programmatic tool calling, code execution), averaged over five trials at max reasoning effort — comfortably above the 60% strike, but this is Anthropic's self-run, non-standard harness, not the official CAIS/Scale leaderboard named in the resolution criteria. Source: Anthropic Claude Opus 5.5 System Card PDF (anthropic.com, hosted at www-cdn.anthropic.com). - [2026-09-22] The same system card states the HLE evaluation used "thinking set to auto," a 1M total-token cap, no context compaction, and Claude Opus 4.6 as model grader, with a blocklist of 100+ domains (including agi.safe.ai itself) to prevent contamination from sites discussing HLE — confirming Anthropic treats agi.safe.ai/lastexam.ai as a separate, "official" entity distinct from its own harness. - [2026-10-11] The agi.safe.ai homepage (the market's named resolution source) shows only project news/links (HLE-Diamond released 2025-09-22, Nature publication 2026-01-28, HLE-Rolling fork from 2025-10-08) and a "Submission Dashboard" link; it does not publicly display a live leaderboard table on the page itself. - [2026-10-11] The agi.safe.ai/dashboard page (likely location of the actual leaderboard) returned a Clerk-powered "Sign in to Humanity's Last Exam" authentication wall when fetched, meaning the live official leaderboard is not currently viewable without an account — a potential resolution risk/ambiguity for the market. - [~2026-09-30 to 10-09] Third-party aggregator snapshots citing the "Scale Labs" / official CAIS-Scale protocol report Claude Opus 5.5 much lower than Anthropic's self-reported figure: one snapshot (dated 2026-09-30) lists GPT-6 Astra leading the public Scale snapshot at 60.6%, with Claude Opus 5.5 at only 55% and Claude Fable 5.1 at 51.3% — all below the 60% strike on that specific protocol. - [2026-10-06] Another tracker citing "Scale Labs, full set" (accessed 2026-10-06) lists even older Claude models (Opus 4.7, Opus 4.6) in the 34–36% range on the official protocol, underscoring that the strict CAIS/Scale grading historically produces materially lower scores than vendor self-reported harnesses. - [2026-10-07 to 10-09] Independent trackers using other protocols (Artificial Analysis text-only subset, BenchLM) show mixed results: Artificial Analysis (2026-10-07) has Claude Opus 5.5 leading at 61.4%, just above the strike; BenchLM's "HLE" page (2026-10-08) reports a different ranking with "Claude Fable 5.1" leading at 65% and Opus 5.5 not in the top 3 shown; pricepertoken.com (updated 2026-10-09) reports "Claude Fable 5.1" leading the overall leaderboard at 59.1%, below 60%. - [2026-09-22] Per Anthropic's own card, Claude Fable 5.1 (a sibling/concurrent Anthropic model) scored 65.6% with tools and 60.9% without tools in the same internal harness — i.e., multiple Anthropic models already self-report above 60% on HLE as of the most recent release, if Anthropic's own harness were used as the resolution source. - [2025-01-28] Background: HLE was published in Nature (Nature 649, 1139–1146), confirming it is an actively maintained, continuously tracked benchmark with periodic dataset revisions (e.g., HLE-Diamond released 2026-09-22, HLE-Rolling fork from 2025-10-08), meaning the exact scoring protocol/question set that "official" leaderboard uses could still change before the 2026-12-31 deadline. NOT FOUND: The actual current numeric value and date-stamp shown on the live, authoritative agi.safe.ai/dashboard leaderboard (the market's literal resolution source) for Claude Opus 5.5 or any other current Claude model — this page requires sign-in and could not be viewed directly. Could not find an archived/cached public mirror of that specific dashboard's current leaderboard table. BOTTOM LINE: Anthropic's own September 2026 system card self-reports Claude Opus 5.5 (and even Claude Fable 5.1) scoring above 60% on HLE using its internal harness with tools, but the market's resolution source — the official agi.safe.ai leaderboard — is currently behind a login wall, and independent trackers citing the stricter "official" CAIS/Scale protocol put the best Claude model closer to 51-61%, straddling the 60% line with meaningful downside risk if the official site's grading proves stricter than Anthropic's self-reported numbers.

- [2025-01-24] At launch (Jan 2025) the official HLE leaderboard showed GPT-4o scored 2.7% and o3-mini (high) scored 13.4% , illustrating how low the benchmark started; https://klu.ai/benchmarks/humanitys-last-exam - [2026-02-03] By early February 2026 the top official-leaderboard score was still well under 40%: Gemini 3 Pro Preview led at 37.52% with Claude Opus 4.5 (Thinking) at 25.20% ; https://letsdatascience.com/blog/humanitys-last-exam-the-test-thats-humbling-the-worlds-smartest-ai - [2026-08-16] As late as mid-August 2026 the official public-question leaderboard had not even crossed the "50% by end of 2025" milestone that HLE's own organizers had predicted: the HLE organizers had predicted models "could exceed 50% accuracy on HLE by the end of 2025" if advances continued... as of August 16, 2026, no model listed on the official public leaderboard has reached 50% ; https://intuitionlabs.ai/articles/humanitys-last-exam-ai-benchmark - [2026-09-22] This base rate shifted abruptly with new model releases and a new question set: CAIS released HLE-Diamond, and the Scale Labs leaderboard (no tools) showed gpt-6-astra 60.6%, claude-opus-5-5 55.0%, claude-fable-5-1 51.3%, claude-opus-5 38.6% , with Claude still short of 60% on this no-tools official metric; https://postcutoff.com/e/2026-09-22-hle-diamond/ - [2026-09-22] On the same CAIS/Scale HLE-Diamond release, when tools were allowed, Anthropic's own best model reportedly crossed the 60% line on an official CAIS/Scale-run table: Claude Opus 5.5 — Anthropic — 73.9% — HLE-Diamond, 1,000 questions, web and code tools — Sep 22, 2026 — CAIS and Scale blog, +19.3 over no tools ; https://klu.ai/benchmarks/humanitys-last-exam - [2026-09-22/30] Independent of CAIS, Anthropic's own self-reported harness (not the official leaderboard) claims a much higher figure: Humanity's Last Exam, run with tools, returned 67.7% for Claude Opus 5.5; https://siliconangle.com/2026/09/22/anthropic-releases-claude-opus-5-5-and-openai-counters-with-two-cheaper-gpt-6-models/ - [2026-10-07/09] Third-party aggregator snapshots disagree sharply by protocol: Artificial Analysis's text-only board had Claude Opus 5.5 leading the AA-HLE leaderboard with 61.4%, followed by Claude Fable 5.1 (59.1%) and Gemini 4 Argon (57.1%) as of Oct 7, 2026, while another tracker put Claude Fable 5.1 at 59.1% as of Oct 9, 2026; https://benchlm.ai/benchmarks/aahle and https://pricepertoken.com/leaderboards/benchmark/hle - [2026-10-11] A dedicated blog analysis found the spread across "official" and third-party trackers was enormous for the same model: the gap between Artificial Analysis's 59.1% and BenchLM's 65% ceiling — as of mid-September 2026, both now led by the newly released Claude Fable 5.1 — is still nearly six points , while Scale AI's own run of the same model sat at only ~47%; https://digitortoise.com/humanitys-last-exam/ - [2026-09-29] A separate market-tracking note confirms the resolution ambiguity matters: commentary on a sibling "75%+" HLE market explicitly flagged that the market will now focus on whether competing AI labs, such as Anthropic or Google, release models that can surpass GPT-6 Astra's 82.9% score... the contract's settlement source, agi.safe.ai, is the official home for the benchmark, and traders will be watching for updates to the main leaderboard ; https://www.octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds/ - [Base rate pattern] Across 2025-2026, HLE scores for the frontier leader rose from ~2-13% (launch) → ~25-38% (early-mid 2026) → 55-82.9% (official CAIS HLE-Diamond, with-tools, late Sept 2026) in under 21 months — a roughly 4-6x acceleration once tool-use and a cleaned question set were introduced, showing the benchmark's "ceiling" is highly protocol-dependent rather than a smooth, predictable trend. - [Reference class caveat] No comparable binary market/forecast track record specific to HLE thresholds was found; the closest analog is a sibling Polymarket-style "HLE 75%+" contract referenced in the Octagon AI article, which itself moved on the same Sept 22, 2026 HLE-Diamond release, indicating these benchmark-threshold markets are highly sensitive to single leaderboard updates rather than gradual trends. BOTTOM LINE: Historically, the official HLE leaderboard has been a slow, high bar (no Claude model topped 50% for most of 2026), but the September 22, 2026 introduction of HLE-Diamond and tool-assisted scoring produced a sudden jump, with CAIS/Scale's own published tables already showing an Anthropic model (Opus 5.5) at 73.9% with tools — meaning the "60%+ by Dec 31, 2026" bar for Claude may already be functionally cleared on at least one recognized official metric, though whether that specific Diamond/tools figure counts as "the" HLE Accuracy on agi.safe.ai's main leaderboard is the key unresolved interpretive question. NOT FOUND: a live, publicly rendered snapshot of the current agi.safe.ai/dashboard leaderboard table itself (the page requires sign-in via Clerk authentication), so the exact figure currently displayed as "HLE Accuracy" for Claude on the canonical resolution source could not be directly confirmed; no dedicated prior prediction-market or survey forecast history specific to this exact threshold/model combination was found.

YES means: an Anthropic Claude model's official/recognized "HLE Accuracy" reaches ≥60% on the agi.safe.ai-recognized Humanity's Last Exam benchmark (or an accepted official-alternative source) at some point before 2026-12-31 23:59 ET. - [2026-09-22] The official CAIS/Scale AI blog post at lastexam.ai/blog/hle-diamond — the same organization that runs agi.safe.ai, contactable at agibenchmark@safe.ai — directly publishes a "HLE-Diamond with tools" chart showing Claude Opus 5.5 at 73.9% with tools (web+code), versus 54.6% without tools , confirmed by fetching the primary page itself. This is not a vendor self-report; it is CAIS's and Scale AI's own published table on the canonical lastexam.ai domain that the agi.safe.ai ecosystem treats as its authoritative dataset/results source. - [2026-09-22] The same official page also shows a second Anthropic model clearing 60% with tools: Claude Fable 5.1 at 50.7% without tools and 72.3% with tools (web+code) — meaning two separate current Claude models exceed the strike on this official with-tools table, not just one, reducing reliance on a single model's performance. - [2026-09-22] Scale Labs' own mirrored leaderboard page (labs.scale.com/leaderboard/hle-diamond), built "in partnership with the Center for AI Safety," independently republishes the no-tools Diamond numbers and explicitly flags that "Evaluation with tools" is a recommended, documented CAIS/Scale methodology (linking to centerforaisafety/hle's own evaluation-with-tools.md on GitHub) — showing tools-based scoring is an endorsed, not informal, official evaluation mode, not an Anthropic-only invention. - [2026-10-06] A third-party synthesis (iqmetrics.org) independently cross-checked and reproduced the same figures, stating plainly that on the HLE-Diamond page "HLE-Diamond with web and code tools: GPT-6 Astra 82.9%, Claude Opus 5.5 73.9%, Claude Fable 5.1 72.3%" , corroborating the primary source reading from an independent analyst rather than Anthropic's own marketing. - [2026-10-06] The same analysis explains the mechanism behind the gap: "the Diamond page reports the same models without tools and with web search and code, and the with-tools column is 17 to 23 points higher for the three models listed" — tool use is a large, structurally reproducible lift (not noise), so once a frontier Claude model is evaluated with tools on any CAIS-affiliated table, clearing 60% is the expected outcome rather than an outlier. - [2026-09-22] CAIS/Scale themselves frame HLE-Diamond as the dataset "where that work lands now" and explicitly link it from lastexam.ai (the domain used for the dataset referenced by agi.safe.ai), making it plausible that if/when the agi.safe.ai dashboard surfaces a current table, it will reflect this same Diamond/with-tools data rather than a stricter, lower "no-tools full-set" number. - [2026-10-07] Independent of the Diamond release, the Artificial Analysis text-only tracker (benchlm.ai mirror) already shows "Claude Opus 5.5 leads the AA-HLE leaderboard with 61.4%" as of October 7, 2026 — a second, independently-run, non-Anthropic protocol where a Claude model clears 60% on the full question set without any tools-based inflation, giving a secondary path to YES even if the stricter Scale no-tools board is used. - [2026-10-08] A further independent aggregator shows a Claude model exceeding 60% on the base (non-Diamond) HLE set as well: "Claude Fable 5.1 leads the HLE leaderboard with 65%, followed by Claude Opus 5 (64.7%) and Claude Mythos 5 (64.5%)" , indicating multiple scoring frameworks beyond Diamond-with-tools already place Claude models above the strike. - Mechanism for YES: the resolution only requires the official source (or an accepted official alternative if agi.safe.ai is "otherwise confirmed to be permanently unavailable" or stays down) to display an "HLE Accuracy" figure ≥60% for any Claude model at any point through year-end; given CAIS/Scale's own published Diamond-with-tools table already shows 73.9% (Opus 5.5) and 72.3% (Fable 5.1), and given Anthropic is likely to release further Opus/Fable updates before December 31, the most likely YES path is simply that the with-tools HLE-Diamond figures (or a comparable future CAIS/Scale table) are treated as "the" leaderboard value, or that CAIS updates the public-facing dashboard to show these already-public, officially-computed numbers. BOTTOM LINE: The strongest case for YES rests on a primary-source fact already confirmed on CAIS/Scale's own lastexam.ai page: Claude Opus 5.5 scores 73.9% and Claude Fable 5.1 scores 72.3% with tools on HLE-Diamond, an officially CAIS/Scale-computed (not Anthropic self-reported) result, plus two independent third-party trackers (Artificial Analysis, BenchLM) that already show Claude models above 60% on non-Diamond protocols — so if any of these officially-run, publicly documented figures are accepted as "HLE Accuracy" for resolution, the 60% bar for an Anthropic Claude model is already cleared well before the December 31, 2026 deadline. NOT FOUND: a rendered view of the live agi.safe.ai/dashboard table itself (still behind a Clerk sign-in wall as of the most recent check) confirming which of these competing figures (Diamond-with-tools, Diamond-no-tools, AA-HLE, or full-set Scale) it currently displays as the canonical "HLE Accuracy" value.

YES means: an Anthropic Claude model's score on the official Humanity's Last Exam leaderboard (agi.safe.ai, or an official alternative if that source is confirmed permanently unavailable) reaches at least 60% accuracy by December 31, 2026, 11:59 PM ET. - [2026-10-11] The market's named resolution source, agi.safe.ai/dashboard, remains behind a Clerk-powered sign-in wall ("Sign in to Humanity's Last Exam... Secured by Clerk") with no public leaderboard table rendered — confirmed again today by direct render, meaning as of this date there is no viewable "HLE Accuracy" figure on the literal resolution page at all. https://agi.safe.ai/dashboard - [2026-10-11] The agi.safe.ai homepage itself contains no live leaderboard table; its "Quantitative Results" section is static prose stating "All frontier models achieve low accuracy on Humanity's Last Exam," and it points visitors elsewhere ("Also available at CAIS AI Dashboard and SEAL LLM Leaderboards") rather than hosting an updating table in-page. https://agi.safe.ai/ - [2026-09-22] On the Scale Labs leaderboard co-built with CAIS and explicitly linked from the HLE dataset page (labs.scale.com/leaderboard/hle-diamond) — the closest fetchable primary "official" source — the frozen, stable-ID HLE-Diamond "Main Results" headline table shows Claude Opus 5.5 at only 55.0% overall (Reasoning 63.2%, Knowledge 46.8%), with no tools, below the 60% strike. https://labs.scale.com/leaderboard/hle-diamond - [2026-09-22] The same Scale Labs page states explicitly that the question set is frozen by design ("A frozen question set... so a score from today is comparable to a score from next quarter") and that "the strongest model we tested lands at 60.6%, well below the noise ceiling," meaning the organizers intentionally built HLE-Diamond so even the top model (GPT-6 Astra, OpenAI) barely clears 60% while Claude's best entry sits 5.6 points under it. https://labs.scale.com/leaderboard/hle-diamond - [2026-09-22] On that same official table, the next-best Anthropic entry, Claude Fable 5.1, scores even lower at 51.3% overall — so no Claude model clears 60% on this no-tools "official" protocol. https://labs.scale.com/leaderboard/hle-diamond - [2026-10-06] Klu's benchmark tracker, synthesizing the fragmented 2026 landscape, states plainly "There is no single HLE leaderboard in 2026," cataloguing at least four divergent protocols (Anthropic's own harness, Artificial Analysis text-only subset, Scale's full-set board, and CAIS/Scale's HLE-Diamond) with materially different Claude scores (from 38.6%–67.7%), creating genuine resolution ambiguity about which number counts. https://klu.ai/benchmarks/humanitys-last-exam - [2026-09-22] CAIS's own HLE-Diamond announcement, mirrored by Postcutoff, reports a separate "lastexam.ai no-tools chart (reasoning high)" giving Claude Opus 5.5 54.6% and Claude Fable 5.1 50.7% — a second independent CAIS-affiliated citation corroborating the sub-60% Claude result on the no-tools official protocol. https://postcutoff.com/e/2026-09-22-hle-diamond/ - [2026-10-06] A GitHub pull request tracking automated HLE-Diamond scraping notes that as of its update "llm-stats, Epoch and Artificial Analysis have no HLE-Diamond board" at all, underscoring that most third-party trackers still cannot even surface the CAIS/Scale Diamond protocol, let alone the agi.safe.ai dashboard itself, reinforcing how incomplete public access to the literal resolution source remains. https://github.com/dgrieser/ai-bench/pull/261 - [2026-10-09 to 10-11] Multiple non-CAIS aggregator snapshots (pricepertoken.com at 59.1% for Claude Fable 5.1, Artificial Analysis/BenchLM's AA-HLE at 61.4% for Claude Opus 5.5) straddle the 60% line but are explicitly "independent" re-runs, not the CAIS/Scale-run or agi.safe.ai-hosted figures the resolution criteria specify. https://pricepertoken.com/leaderboards/benchmark/hle, https://benchlm.ai/benchmarks/aahle - [Mechanism] Per the resolution text, if agi.safe.ai stays unavailable (i.e., gated) through year-end, the market must fall back to "official Humanity's Last Exam results published elsewhere," and if none is available it resolves "No" by default — a structural tiebreaker that favors NO if the sign-in wall persists and no other CAIS-attributed, no-tools, Claude-at-60%+ figure materializes before Dec 31, 2026. BOTTOM LINE: The most CAIS/Scale-proximate figure actually readable today — the frozen HLE-Diamond "Main Results" table on labs.scale.com — puts Anthropic's best model at 55.0%, below the 60% strike, while the literal named resolution source (agi.safe.ai/dashboard) remains fully login-gated with no public table, giving the NO case both a substantive scoring argument and a resolution-mechanics fallback argument. NOT FOUND: Any authenticated view or leaked/cached screenshot of the actual agi.safe.ai/dashboard table with current Claude numbers; any CAIS statement on when (or whether) the dashboard will be made public before December 31, 2026.