Will any LLM score at least 60% on Humanity's Last Exam before Dec 31, 2026?

- [2026-09-22] The Center for AI Safety/lastexam.ai published "HLE-Diamond" results; a blog post from the Center for AI Safety on September 22, 2026, detailed the performance of several frontier models, with OpenAI's GPT-6 Astra scoring 82.9% when evaluated with tools like web search and code execution . This is well above the 60% threshold. Source: octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds. - [2026-09-22] Independent evaluator Artificial Analysis reports Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback) currently has the highest Humanity's Last Exam score, with a score of 61.4% among models with published Humanity's Last Exam results — the first independently-run (non-tool) score crossing 60%. Source: artificialanalysis.ai/evaluations/humanitys-last-exam. - [2026-09-22] Anthropic's own system card reports Claude Opus 5.5 scoring higher with tools: Humanity's Last Exam, with tools — 67.7%, vendor-reported, with Fable 5.1 at 65.6%, Opus 5 at 63.6%, GPT-6 Astra at 57.2%; Anthropic's system card separately reports 64.4% without tools . Source: orcarouter.ai/blog/claude-opus-5-5-benchmark. - [2026-09-24] Kalshi's own sibling "highest score" market repriced sharply: the contract pricing the odds of any AI achieving "At least 75%" on Humanity's Last Exam (HLE) before the end of 2026 surged to 87% from just 14% , and per the reported distribution table the "At least 60%" contract stood at 60% before the move (already implying meaningful probability mass above 60% even before the Astra news). Source: octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds. - [2026-09-17] Prior to the Astra release, the highest recorded score on the main HLE leaderboard was 65.0%, achieved by Anthropic's Claude Fable 5.1, according to LLMboard data updated September 17, 2026 . Source: octagonai.co (citing llmboard.ai). - [2026-09-24] Third-party aggregator llm-stats.com (self-reported, unverified submissions) lists Claude Fable 5.1 by Anthropic leading with a score of 0.650 , but notes this leaderboard has 0 verified results and 104 self-reported results . Source: llm-stats.com. - [2026-09-24] Another aggregator (pricepertoken.com) shows a lower figure: the top-scoring model on Humanity's Last Exam is Claude Fable 5.1 at 59.1%, followed by Claude Fable 5 at 55.5% and Claude Opus 5 at 54.9% , illustrating that different trackers/protocols (tool use, effort settings, question subsets) produce materially different "top score" figures. Source: pricepertoken.com/leaderboards/benchmark/hle. - [2026-08-16] Earlier in the quarter, the Scale AI/CAIS public leaderboard (a stricter, protocol-controlled source) was still well under 60%: Gemini 3.1 Pro Preview (thinking high) at 46.44%, GPT-5.4 Pro at 44.32%, and Muse Spark at 40.56% . This shows the "official" benchmark administrators' board lagged independent/vendor figures for most of 2026 before the September acceleration. Source: intuitionlabs.ai/articles/humanitys-last-exam-ai-benchmark. - [2026-09-28] BenchLM.ai (tracking self-reported/mixed-protocol scores) shows continued clustering near saturation at the top: Claude Fable 5.1 leads the HLE leaderboard with 65%, followed by Claude Opus 5 (64.7%) and Claude Mythos 5 (64.5%) . Source: benchlm.ai/benchmarks/hle. - [2026-09-22] Anthropic launched Claude Opus 5.5, and Vellum's analysis of its benchmark table notes: With tools enabled, Opus 5.5 tops the board at 67.7%, advancing past Fable 5.1 (65.6%), Opus 5 (63.6%), and GPT-6 Astra (57.2%) . Source: vellum.ai/blog/claude-opus-5-5-benchmarks-explained. - [2026-09-30] No source found (as of today) showing the "official" Scale AI/CAIS agi.safe.ai leaderboard itself has been updated to display a 60%+ figure on the original, unmodified HLE test; the highest confirmed 60%+ figures come from independent evaluator Artificial Analysis (61.4%, no tools), Anthropic's own reporting (64.4%–67.7%, with/without tools), and the new HLE-Diamond variant (82.9%, GPT-6 Astra, with tools). BOTTOM LINE: Multiple credible sources (Artificial Analysis's independent run, Anthropic's own system card, and the CAIS-published HLE-Diamond results) already show LLM scores well above 60% on Humanity's Last Exam as of September 22-24, 2026, though results vary substantially depending on whether tool use is permitted and which HLE variant/protocol is used. Given this, and roughly three months remaining before the December 31, 2026 deadline with a rapidly improving frontier, the "at least 60%" bar appears to have already been cleared by multiple models on credible/independent measurements. NOT FOUND: A definitive statement from the exact official Kalshi/agi.safe.ai "canonical" leaderboard page showing an unambiguous 60%+ entry that removes all interpretation/protocol ambiguity (i.e., whether the market strictly requires the original no-tool HLE-2500 text-only score versus HLE-Diamond or tool-assisted variants); could not directly fetch agi.safe.ai's live leaderboard table (site content was mostly static/marketing text in the fetch).

- [2025-01-23] HLE launched at single-digit accuracy: "When the benchmark was first released in early 2025, leading AI models scored in the single digits: GPT-4o managed just 2.7%, Claude 3.5 Sonnet reached 4.1%, and OpenAI's o1 model achieved only 8.0%" — establishing a baseline near 0% just 20 months before the market's close. - [2026-09-17] On the official Scale AI/CAIS "Humanity's Last Exam (Text Only)" leaderboard — the primary source for this benchmark — the top-ranked model is GPT‑6 Astra at 54.17±2.09%, followed by Gemini 3.1 Pro Preview (thinking high) at 47.31%, Fable 5.1 (xhigh) at 46.80%, and gpt-5.4-pro at 45.32%, per the leaderboard update dated September 17, 2026 (source: labs.scale.com/leaderboard/humanitys_last_exam_text_only). - [2026-09-17] CAIS itself notes a shift in methodology: "AI capability is evaluated based on benchmarks, yet as their progress accelerates, benchmarks become quickly saturated, losing their utility as a measurement tool." and it has begun migrating to "HLE-Rolling" with harder replacement questions specifically because frontier models are nearing the noise ceiling of the original set. - [2026-08-16] A secondary tracker (IntuitionLabs, summarizing the Scale AI public leaderboard) reported "the Scale AI public leaderboard lists Gemini 3.1 Pro Preview (thinking high) at 46.44%, GPT-5.4 Pro at 44.32%, and Muse Spark at 40.56%" , roughly consistent with the official leaderboard fetched above. - [2026-09-24] Some third-party aggregators (not primary/official) show much higher, possibly inflated or differently-protocoled scores: Artificial Analysis lists "Claude Opus 5.5 (Adaptive Reasoning, Max Effort, Default Fallback) currently has the highest Humanity's Last Exam score, with a score of 61.4% among models with published Humanity's Last Exam results" , and llm-stats.com and pricepertoken.com separately list "Claude Fable 5.1" at 59.1%-65%. These figures are not corroborated by the official Scale/CAIS leaderboard and llm-stats itself flags its data as "104 models have been evaluated on the Humanity's Last Exam benchmark, with 0 verified results and 104 self-reported results" , indicating these third-party 60%+ claims are unverified/self-reported rather than independently confirmed. - [2025-04] The original CAIS/Scale framing anticipated rapid saturation: "recent history shows benchmarks are quickly saturated -- with models dramatically progressing from near-zero to near-perfect performance in a short timeframe... Given the rapid pace of AI development, it is plausible that models could exceed 50% accuracy on HLE by the end of 2025." That specific 2025 prediction proved too aggressive — 50% was not reached until well into 2026 per the official leaderboard. - [2026-04-13] An independent summary reported the top score around that time at "Claude Mythos Preview recently reached 64.7%, the first model to break past the mid-30s" — but this figure, like the Artificial Analysis/llm-stats numbers, diverges sharply from the official Scale leaderboard's contemporaneous ~37-54% range, suggesting inconsistent evaluation protocols (e.g., tool use, different question subsets) across trackers. - [2026-09-30] Reference-class comparison: similarly-designed "hard" benchmarks have saturated quickly once models cross the halfway mark — e.g., GPQA Diamond went from PhD-level baseline (~65%) to being led by a model at 92% within about a year of being described as near the frontier, and MATH went "from <10% to >90% in just three years" — suggesting that once HLE scores cross ~50%, further gains to 60% can arrive within a few quarters, matching the roughly linear pace of ~5-10 percentage points per new-generation model seen in the last two Scale leaderboard updates. - [2026-05-21] A related market snapshot (highest-HLE-score contract, same Dec 31 2026 deadline) showed traders pricing high confidence that some score of at least 50% would be reached, with the market having jumped from 79% to 87% "This significant initial movement established a strong bullish sentiment from the outset," though prices for the "≥60%" strike specifically were not reported in that source. BOTTOM LINE: On the only clearly primary/official tracker (Scale AI/CAIS), HLE has climbed from near-0% (Jan 2025) to a top score of ~54% (Sept 2026), a rapid trajectory consistent with other quickly-saturated hard benchmarks (GPQA, MATH); reaching the ~60% threshold in the remaining ~3 months is plausible but not yet confirmed by primary data, while several third-party leaderboards already claim scores above 60% using unverified or non-standard (self-reported/tool-assisted) methodologies that should be discounted relative to the official leaderboard. NOT FOUND: No confirmed, independently-verified (non-self-reported) result of ≥60% on the standard/official HLE text-only or full multimodal leaderboard as of the last update (2026-09-17); could not find an explicit statement from CAIS/Scale AI declaring the 60% threshold crossed, nor a reconciled explanation for the large discrepancy between the official Scale leaderboard (~54%) and third-party sites reporting 59-65%.

YES means: at least one LLM's Humanity's Last Exam accuracy, as measured by a source credible enough to settle the market, reaches or exceeds 60% at any point before Dec 31, 2026. - [2026-09-22] The official benchmark administrators themselves (Center for AI Safety and Scale AI, the creators/owners of HLE and operators of agi.safe.ai) published on their own site, lastexam.ai, that "we compare the performance of current models on HLE-Diamond without tools," with tool-augmented results also shown; GPT-6 Astra scored 59.9% without tools on HLE-Diamond and, per the same primary release's tool-use table, reached 82.9% "with tools (web+code)" — a result generated and published directly by CAIS/Scale AI, not a third party. Source: lastexam.ai/blog/hle-diamond. - [2026-09-22] The same primary HLE-Diamond release shows Claude Opus 5.5 at 54.6% without tools and 73.9% with tools, and Claude Fable 5.1 at 50.7% without tools and 72.3% with tools — meaning three separate frontier models from two labs exceed 60% once tool use (web search and code execution) is permitted, on a dataset explicitly built and scored by the benchmark's official custodians. Source: lastexam.ai/blog/hle-diamond. - [2026-09-22] Scale AI's own leaderboard page for this dataset states plainly that "the strongest model we tested lands at 60.6%, well below the noise ceiling," confirming (modulo minor rounding differences from the blog's 59.9%) that the benchmark owners themselves regard a ~60% no-tools score as already achieved on their newest official variant. Source: labs.scale.com/leaderboard/hle-diamond. - [2026-09-22] Independent evaluator Artificial Analysis — which runs its own inference rather than relying on vendor self-reports — separately confirms Claude Opus 5.5 scoring 61.4% on the original (non-Diamond) HLE text-only question set, "currently has the highest Humanity's Last Exam score, with a score of 61.4% among models with published Humanity's Last Exam results," ahead of Claude Fable 5.1 at 59.1%. Source: artificialanalysis.ai/evaluations/humanitys-last-exam. - [2026-09-22] Artificial Analysis frames this as a jump from "previous best 59.1%, Claude Fable 5.1" to 61.4% for Opus 5.5, a gain achieved in three weeks between the Fable 5.1 write-up (Sept 1, 2026) and the Opus 5.5 write-up, indicating the pace of improvement remains fast enough to sustain or extend a 60%+ result through year-end even if any single score were later revised. Source: artificialanalysis.ai/articles/claude-opus-5-5. - [2026-09-23] A market-moving news event followed: coverage reported that "the contract pricing the odds of any AI achieving 'At least 75%' on Humanity's Last Exam (HLE) before the end of 2026 surged to 87% from just 14%" after the HLE-Diamond release, which "established a new state-of-the-art score and suggested that the 75% performance milestone has already been met" — implying the market itself now treats even a much higher 75% bar as near-certain, which mechanically implies the 60% bar is regarded as essentially already cleared. Source: octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds. - [2026-09-24] Trading commentary on the sibling market noted that "the contract for 'At least 65%' fell 15 percentage points on exceptionally high volume," which the article attributes to a mispricing correction rather than doubt about the underlying 82.9% figure, since "the probability of achieving at least 65% should be higher than the 87% odds for the 75% threshold" — underscoring that traders are not disputing the 60%+ facts, only reconciling relative pricing across strikes. Source: octagonai.co/news/ai-benchmark-humanity-s-last-exam-market-odds. - [2026-09-22] Anthropic's own system card for Opus 5.5, reported by multiple secondary sources, lists "Humanity's Last Exam, with tools — 67.7%," with Fable 5.1 at 65.6%, Opus 5 at 63.6%, and GPT-6 Astra at 57.2%, plus a no-tools figure of 64.4% for Opus 5.5 — a second independent lab (beyond CAIS/Scale) reporting 60%+ figures under its own vendor evaluation, converging with the CAIS/Scale HLE-Diamond and Artificial Analysis figures. Source: orcarouter.ai/blog/claude-opus-5-5-benchmark (cross-checked against vellum.ai/blog/claude-opus-5-5-benchmarks-explained). - [2026-09-17→09-30] Even discounting all tool-use and vendor-reported figures, the trend on the strictest tracker (Scale's original text-only leaderboard) had already reached the low-to-mid 50s (GPT-6 Astra ~54%) by mid-September, and the newer HLE-Diamond variant put the no-tools frontier at essentially 60% (59.9%–60.6% depending on source) just five days later — a roughly 5-6 point jump in under two weeks, suggesting continued no-tools improvement could plausibly clear a clean 60% on the original or Diamond set well before Dec 31, 2026 even without counting tool-assisted scores. BOTTOM LINE: Multiple mutually corroborating primary and independent sources — the benchmark's own creators (CAIS/Scale AI's HLE-Diamond release and leaderboard), the leading independent evaluator (Artificial Analysis, 61.4% no-tools), and Anthropic's own system card (64.4%-67.7%) — all show one or more LLMs already at or above 60% as of September 22, 2026, with tool-assisted scores reaching as high as 82.9% on an official CAIS/Scale-published table; this gives the YES case very strong evidentiary support for resolution before the Dec 31, 2026 deadline, contingent mainly on whether Kalshi's settlement source accepts HLE-Diamond/tool-assisted or vendor-reported results as valid "Humanity's Last Exam" scores. NOT FOUND: No explicit statement from Kalshi or agi.safe.ai's canonical page clarifying whether the market's "Humanity's Last Exam" strictly means the original no-tools 2,500-question text set (where the primary Scale AI leaderboard was near 54% as of mid-September) or also counts the newer CAIS/Scale-published HLE-Diamond and tool-assisted variants where 60%+ is already documented by the benchmark's own creators.

YES means: some LLM's score on Humanity's Last Exam is verified/reported at ≥60% accuracy by a credible source before Dec 31, 2026, and that reading is accepted for resolution. - [2026-09-17] The only clearly primary, protocol-controlled tracker — the Scale AI/CAIS official "Humanity's Last Exam (Text Only)" leaderboard — still puts the top model well under 60%, and a Wikipedia benchmark table drawing on the same official source lists GPT-6 Astra at just 54.7%, with the next-best entrants (MiMo-V2.6-Pro, Muse Spark 1.3, Gemini 3.8 Flash) all in the 40s. Source: en.wikipedia.org/wiki/Humanity%27s_Last_Exam. - [2026-09-23] The primary CAIS/Scale announcement of HLE-Diamond, tracked independently by a news aggregator that read the actual blog post and its Reddit source, explicitly states: "No benchmark scores or model results were disclosed in the announcement." This undercuts the colleague-reported 82.9% GPT-6 Astra figure, which appears to originate from a secondary write-up rather than the primary release. Source: aibriefs.news/card/4ea77bae-c00b-42c3-8e30-8b9d0691ea20. - [2026-09-22] Even the most detailed third-party reconstruction of the actual HLE-Diamond release table (BenchLM, which says it "mirrored" the CAIS/Scale table rather than re-running it) puts GPT-6 Astra's no-tools score at only 60.6% at "high" reasoning effort and 66.2% at "highest" effort — far below the 82.9% figure and only marginally above the threshold on a benchmark that is explicitly a different, refined dataset from the original HLE. Source: benchlm.ai/benchmarks/hle-diamond and benchlm.ai/benchmarks/hle-diamond-max. - [2026-09-22] BenchLM's own methodology page flags that HLE-Diamond, HLE-Rolling, HLE-Verified and the "stable" original HLE "use different item sets," and that "a model with search, browsing, code execution, or another tool has not taken the same test as a closed-book model" — meaning any resolution based on tool-assisted or HLE-Diamond scores is contestable as not matching what a plain reading of "Humanity's Last Exam" denotes. Source: benchlm.ai/benchmarks/hle. - [2026-09-24] llm-stats.com's own disclosure that its Claude Fable 5.1 leader figure rests on "0 verified results and 104 self-reported results" means the highest self-reported claims cannot be treated as confirmed accuracy figures a neutral adjudicator would accept without independent verification. Source: llm-stats.com/benchmarks/humanity's-last-exam. - [2026-09-17] CAIS's own framing acknowledges the original HLE set is nearing saturation and is being supplemented/replaced by "HLE-Rolling" with harder questions specifically because "benchmarks become quickly saturated, losing their utility as a measurement tool" — a signal that CAIS itself may treat sub-60% "official" scores on the original set as the operative baseline rather than endorsing tool-assisted or variant-based 60%+ claims. Source: labs.scale.com/leaderboard/humanitys_last_exam_text_only. - [2026-04-2025 baseline] CAIS's own 2025 prediction that models "could exceed 50% accuracy on HLE by the end of 2025" proved too aggressive — that mark wasn't reached until mid-2026 per the primary leaderboard — a track record of the benchmark's official numbers consistently lagging optimistic projections, which argues for caution in assuming 60% will be confirmed by the official leaderboard within the remaining three months. Source: agi.safe.ai. - [2026-08-16] Kalshi's general resolution practice, per its own Help Center and independent explainers, gives Kalshi's markets team broad discretion to interpret ambiguous technical criteria and, in genuinely disputed cases, to settle at last-traded price or via an Outcome Review Committee rather than a expansive/literal reading — as seen in recent controversial resolutions (Khamenei market, Cardi B market) where Kalshi favored a narrower interpretation over one preferred by bulls. Source: oddsshopper.com/articles/prediction-markets/can-kalshi-change-market-rules; mexc.com/news/846213. - [2026-09-24] The wide dispersion across trackers for the "same" benchmark on the same day — official Scale/CAIS ~54%, Artificial Analysis 61.4%, Anthropic's own system card 64.4-67.7%, pricepertoken 59.1%, llm-stats 65% (self-reported), BenchLM 65% — shows no consensus on whether 60% has been "achieved" under any single agreed protocol, giving a resolution committee ample room to conclude the threshold has not been unambiguously met on the standard, tool-free, original HLE test. BOTTOM LINE: The only clean, primary, protocol-controlled source (Scale AI/CAIS's official leaderboard) still shows top scores around 54-55%, not 60%; the widely cited 60%+ and 82.9% figures rest on tool-assisted runs, a distinct "HLE-Diamond" dataset the primary announcement didn't even disclose scores for, or self-reported/unverified trackers — giving a resolution committee credible grounds to treat the 60% bar as not yet cleared under a strict reading of "Humanity's Last Exam." NOT FOUND: No primary CAIS/Scale AI statement confirming any model has crossed 60% on the original, unmodified, tool-free HLE text-only leaderboard; no official confirmation of the specific 82.9% GPT-6 Astra HLE-Diamond figure from CAIS or Scale AI directly (the primary announcement disclosed no scores); no Kalshi-specific rules text clarifying whether HLE-Diamond, tool-assisted, or self-reported scores would count toward this market's resolution.