Scan: Will the highest score achieved by a Google Gemini model on Humanity’s Last Exam in 2026 be 55% or higher?

1. Planner Fable 5.1 · Fermi decomposition + tool plan

Tools chosen: polymarket_direct resolution_source polymarket_related kalshi_related claude_news article_search gdelt_news wikipedia code_execution

Sub-questions (Fermi decomposition)

What is the current highest HLE Accuracy for any Google Gemini model on the official agi.safe.ai leaderboard, and when was it last updated?
What has been the year-over-year rate of improvement in top HLE scores (across all labs and for Gemini specifically) since the benchmark launched in Jan 2025?
Does the official leaderboard include tool-augmented / Deep Research configurations, or only no-tools text-only evaluations, and what do Gemini models score in each mode?
What Gemini model releases (e.g., Gemini 3.x, Gemini Ultra, Deep Think) are expected before Dec 31, 2026, and what HLE scores has Google self-reported for them?
How quickly does the official leaderboard incorporate newly released frontier models, and is there risk it stops updating or excludes Google models?
What are the highest HLE scores achieved by competing frontier models (OpenAI, Anthropic, xAI) to gauge how close the frontier is to 55%?
How do neighboring Polymarket threshold markets (e.g., 50%, 60%, 65%) and analogous OpenAI/xAI HLE markets price this trajectory?

Planner reasoning

This is a Polymarket question about whether a Gemini model will hit ≥55% on Humanity's Last Exam per the official agi.safe.ai leaderboard by end of 2026. Key uncertainties are the current best Gemini score on the official leaderboard, the rate of improvement over the past year, whether Scale AI/CAIS leaderboard updates promptly with new Gemini releases (and whether they score with tools), and competitor trajectories as a proxy for feasibility. Cross-market prices on neighboring thresholds and other labs' HLE markets will help calibrate.

2. Research Parallel tool calls

tool	status	time	items	summary
polymarket_direct	OK	0.4s	1	## This Market's Polymarket Data Will the highest score achieved by a Google Gemini model on Humanity’s Last Exam in 2026 be 55% or higher? - Current price (probability): 46.50% - 7-day price change: +20.00% - 30-day price change: -7.00% - Total volume: $29,971 (USD notional) - Price range: 26.
resolution_source	OK	4.4s	1	already_resolved=unclear; matches_criteria=True; last_updated=01/28/2026 (most recent news item shown: Nature publication). The fetched pages show general information about Humanity's Last Exam (news updates, methodology, dataset design) but do not display an actual leaderboard table with model name
polymarket_related	OK	0.6s	24	Polymarket search over 3 keyword(s), kept 24 matches. keyword "Humanity's Last Exam": 8 markets \| keyword 'HLE': 8 markets \| keyword 'Gemini': 8 markets
kalshi_related	OK	3.4s	1	1 related markets / summaries. keyword "Humanity's Last Exam": ok \| keyword 'Gemini': no matches \| keyword 'HLE': no matches
claude_news	OK	50.8s	11	Here are the key findings on Gemini's HLE performance and the competitive landscape as of mid/late 2026: Google's self-reported Gemini HLE scores (trajectory): - Gemini 3 Pro (Nov 18, 2025 launch): "It demonstrates PhD-level reasoning with top scores on Humanity's Last Exam (37.5% without the
article_search	OK	0.8s	27	Fetched 27 unique articles across 4 queries (lookback=120d). "Humanity's Last Exam Gemini score": 10 articles \| 'HLE leaderboard new record': 10 articles \| "Gemini 3 Deep Think benchmark Humanity's Last Exam": 10 articles \| 'Google Gemini next model release 2026': 10 articles
gdelt_news	OK	43.7s	20	GDELT: 20 articles across 2 queries (lookback=90d). "Humanity's Last Exam Gemini": 10 hits \| 'HLE benchmark record': 10 hits
wikipedia	OK	0.4s	2	Fetched 2 Wikipedia entries (0 missing pages).
code_execution	OK	65.4s	0	Key quantitative findings - Linear extrapolation (naive): Fitting a straight line to the industry‑wide top no‑tools scores (9%→25%→37.5%, R²=0.99) gives a slope of ~2.7 pts/month, implying ~75% by Dec 2026. The Gemini‑only line (9%→21%→37.5%/41%) gives an even steeper slope (~2.9 pts/mo, R²

3. Evidence Brief Sonnet 5 · 6482 chars

# Event Will the highest score achieved by a Google Gemini model on Humanity's Last Exam (HLE) reach ≥55% by Dec 31, 2026, per the official agi.safe.ai / Scale AI leaderboard? # Outcomes to forecast - Yes (≥55% HLE accuracy achieved by any Gemini model) - No (highest Gemini score stays below 55%) # Kalshi market anchor No kalshi_direct data was returned for this ticker (kalshi_related search found no matching HLE/Gemini markets). Using Polymarket as the closest live proxy: current YES price **46.5%**, up sharply from 7 days ago (+20pp swing — was ~26.5% low), but down 7pp over 30 days; range 26.5%–59%; volume ~$30K over 56 days. Market has been volatile, reflecting uncertainty around whether newer scores count and leaderboard lag. # Resolution source Fetched agi.safe.ai and labs.scale.com/leaderboard/humanitys_last_exam (status 200, last content update ~01/28/2026 per news timestamp). Pages confirm the leaderboard exists and is the designated resolution source, but the fetched excerpt did not render the live table/scores. Secondary tool (claude_news) captured a table snapshot from ~Aug 16, 2026 showing Gemini 3.1 Pro Preview (thinking high) at 46.44% — currently the #1 model on the official leaderboard, ahead of GPT-5.4 (44.32%) and Muse Spark (40.56%). Condition (≥55%) is **not yet met** as of the latest confirmed leaderboard read. # Sub-question answers 1. **Current highest official Gemini HLE score** — 46.44% (Gemini 3.1 Pro Preview, thinking-high), #1 on Scale AI leaderboard as of ~Aug 16, 2026 [claude_news/intuitionlabs.ai]. 2. **YoY improvement rate** — Top scores rose from single digits (Jan 2025, GPT-4o 2.7%) to 37.5% (Gemini 3 Pro, Nov 2025) to 46–48% (self-reported, Feb 2026) — roughly 3x in ~13 months industry-wide, decelerating (S-curve behavior per code_execution logistic fit). 3. **Tool-augmented vs no-tools** — Official leaderboard entries cited are no-tools text scores. Google's self-reported blog figures (37.5%, 41%, 44.4%, 48.4%) are also explicitly "without tools." No confirmed tool-augmented Gemini leaderboard entry found; Monte Carlo analysis notes tool-augmented scores would push P(Yes) much higher (~90%+) if counted, but resolution criteria reference the official no-tools leaderboard. 4. **Upcoming Gemini releases** — Gemini 3.1 (Feb 2026), 3.6/3.7/3.8 Flash (through Sep 2026) already released; self-reported peak is Gemini 3 Deep Think updated to 48.4% (~Feb 2026) [blog.google]. Further Pro/Ultra/Deep Think updates plausible before Dec 2026 given rapid cadence. 5. **Leaderboard update speed/risk** — Leaderboard appears to lag self-reported/press scores by weeks-to-months (46.44% official vs 48.4% self-reported around same period); no evidence of exclusion risk for Google, but update cadence introduces resolution timing risk near year-end. 6. **Competing frontier scores** — GPT-5.4 Pro 44.32%, Muse Spark 40.56% on official leaderboard (Aug 2026); other aggregators (pricepertoken, llm-stats) claim Claude Fable 5/5.1 at 55.5–65% and GPT-6 Astra ~54.7%, but these appear to be non-official/lagging sources not matching agi.safe.ai's slower-moving table — discrepancy unresolved, flagged as a data-source risk. 7. **Neighboring markets** — Gemini 50% bucket priced 84.2% YES (likely by 2026), 55% at 46.5% (this market), 60% at 15%. Grok's parallel ladder: 45%→76.3%, 50%→57.5%, 55%→32%, 60%→21.5%. Market structure implies consensus expects Gemini to clear 50% but is roughly a coinflip on 55%. # Key facts (high-confidence, factual) 1. [blog.google] Gemini 3 Pro (Nov 2025): 37.5% no-tools HLE. 2. [blog.google] Gemini 3 Deep Think (updated ~Feb 2026): 48.4% no-tools — Google's highest self-reported figure. 3. [aicerts.ai] Gemini 3.1 Deep Think (Feb 19, 2026): 44.4% no-tools. 4. [labs.scale.com/claude_news] Official leaderboard (~Aug 2026): Gemini 3.1 Pro Preview 46.44%, currently #1. 5. [Wikipedia] HLE launched Jan 2025 by Center for AI Safety + Scale AI, 2,500 questions. # Cross-market signals - Kalshi related: no directly relevant markets found. - Polymarket: this market 46.5% YES (volatile, +20pp last week); 50% threshold 84.2% YES; 60% threshold 15% YES — consensus clusters around Gemini landing in 50–55% range by EOY 2026. - Sportsbook implied: N/A. # Analyst opinions and speculation - code_execution Monte Carlo: from 41% baseline, P(≥55%)≈86%; from 37.5% baseline, ≈76%; from 45% "with tools" baseline, ≈94%. Logistic (saturation) model far more conservative: caps at ~41–60%, reflecting HLE's designed resistance to easy gains. - Blended synthesized estimate from code_execution: ~50–60% probability, highly sensitive to whether tool-augmented scores count (they likely don't per resolution wording, which favors the more conservative estimate). - Commentators (Digg/Wang tweet) note controversy over benchmark-gaming/conflicts of interest (e.g., Muse Spark), suggesting some skepticism about rapidly rising scores generally. # Directional lean per outcome - **Yes**: Rapid Gemini release cadence (3.1→3.6→3.7→3.8 Flash within months); official leaderboard already at 46.44%, needs only +8.6pp; historical per-release gains average ~10pts; more Pro/Ultra/Deep Think releases expected before Dec 2026. - **No**: Official leaderboard (resolution source) lags self-reported figures and sits at 46.44%, still ~8.5pp short; S-curve/saturation dynamics suggest diminishing returns as questions get harder; strict no-tools interpretation excludes higher tool-augmented scores; Polymarket pricing this near-coinflip (46.5%) despite 50% bucket priced much higher (84%), implying market believes 50-54% is the modal outcome, not ≥55%. # Gaps / unknowns - No live, verified leaderboard read beyond ~Aug 2026 snapshot; true current state (Sep–Dec 2026) unknown. - Discrepancy between official Scale leaderboard (~46%) and third-party aggregators claiming 55-65% for non-Gemini models unresolved — unclear if these reflect a different/updated official table. - No Kalshi-direct price available for direct calibration; relying on Polymarket proxy. # Calibration anchors - Polymarket proxy price (used as anchor in absence of Kalshi direct): 46.5% YES. - Neighboring buckets: 50% threshold priced 84.2% YES; 60% threshold priced 15% YES — implies market's modal Gemini EOY score sits between 50-59%, consistent with ~45-50% probability mass on ≥55%. - Precedent: benchmark scores nearly doubled in ~9 months (37.5%→46-48%) but rate is decelerating (logistic fit implies ceiling near 41-60%).

4. Forecasters 5 draws · Fable 5.1 + GPT-5.6 Sol

claude-fable-5-1

43% conf

Yes	45%
No	55%

The official Scale AI leaderboard (resolution source) shows Gemini 3.1 Pro Preview at 46.44% as of ~Aug 2026, ~8.6pp short of the 55% threshold, and Google's own best self-reported no-tools figure (48.4% Deep Think, Feb 2026) has not been surpassed in roughly six months, indicating deceleration consistent with the brief's logistic fit. A Yes requires a new flagship (Gemini 4 / Deep Think update) to land a ~10pp jump AND be posted on the lagging official table before Dec 31, which is plausible given Google's cadence but far from assured. The neighboring-bucket structure (50% at 84%, 60% at 15%) implies the market's modal outcome is 50–54%, consistent with the Polymarket proxy anchor of 46.5%. Unresolved reports of competitors at 55–65% on third-party aggregators offer some upside, but leaderboard lag and the strict no-tools criterion offset it, so I sit slightly below the anchor.

claude-fable-5-1

42% conf

Yes	47%
No	53%

The official Scale/CAIS leaderboard shows Gemini 3.1 Pro Preview at 46.44% (no-tools) as of the latest confirmed read, ~8.6pp short of the 55% threshold, with Google's best self-reported figure at 48.4% (Gemini 3 Deep Think). Google's release cadence makes a major Pro/Deep Think/Gemini 4-class update before year-end plausible, and prior generational jumps (~10-16pp) could clear 55%, but the S-curve deceleration on HLE, the strict no-tools resolution basis, and the leaderboard's weeks-to-months lag against self-reported scores are material headwinds. Neighboring markets (50% at 84%, 60% at 15%) imply the modal expectation sits in the 50-54% range, consistent with the Polymarket proxy of 46.5% YES. With no Kalshi-direct price and no brief-specific information the market hasn't priced, I stay close to the anchor.

claude-fable-5-1

42% conf

Yes	48%
No	52%

The official Scale AI/CAIS leaderboard (resolution source) shows Gemini 3.1 Pro Preview at 46.44% as of ~Aug 2026, ~8.6pp short of the 55% threshold, with Google's best self-reported no-tools figure (Gemini 3 Deep Think) at 48.4%. Historical per-generation gains of roughly 7–10pp and Google's pattern of major Gemini releases in Nov/Dec make a jump to ≥55% plausible before year-end, but S-curve saturation on HLE, the no-tools restriction, and the leaderboard's weeks-to-months lag (a December release may not be scored by Dec 31) cut the other way. The Polymarket proxy at 46.5% YES, with the 50% bucket at 84% and 60% at 15%, implies a modal EOY score in the 50–54% range and a near-coinflip on 55%. I stay close to that anchor with a marginal tilt toward Yes for the expected late-year flagship/Deep Think release.

gpt-5.6-sol

55% conf

Yes	45%
No	55%

The 46.44% official Gemini score and 48.4% self-reported score indicate that another substantial 6.6–8.6 percentage-point gain is required, pointing slightly toward No. Rapid historical improvement and Google's frequent release cadence make such a gain plausible, but decelerating benchmark progress and the absence of a confirmed upcoming Pro or Deep Think release temper the case for Yes. The official no-tools leaderboard and its publication lag create additional downside, especially for improvements arriving near year-end, while unofficial scores from other models do not establish that Gemini has crossed the threshold. I therefore stay close to the 46.5% Polymarket proxy anchor but shade Yes down to 45%, consistent with neighboring markets suggesting 50–54% is the modal range.

gpt-5.6-sol

58% conf

Yes	48%
No	52%

The latest confirmed official score is 46.44%, so Gemini needs an additional 8.56 percentage points before year-end, which favors No given diminishing benchmark gains. Rapid release cadence and the 48.4% self-reported Deep Think result make another material improvement plausible, while the 84.2% price for clearing 50% suggests Gemini will likely approach the threshold. However, the official no-tools leaderboard can lag new results by weeks or months, creating extra risk that a late qualifying score will not appear in time. I therefore place Yes at 48%, slightly above the 46.5% Polymarket proxy anchor but still near a coin flip.

6. Final plain mean of 5 draws

47% (48% confidence)

Yes: 47% No: 53%

Mean of 5 draws (0.45, 0.47, 0.48, 0.45, 0.48). The official Scale AI leaderboard (resolution source) shows Gemini 3.1 Pro Preview at 46.44% as of ~Aug 2026, ~8.6pp short of the 55% threshold, and Google's own best self-reported no-tools figure (48.4% Deep Think, Feb 2026) has not been surpassed in roughly six months, indicating deceleration consistent with the brief's logistic fit. A Yes requires a new flagship (Gemini 4 / Deep Think update) to land a ~10pp jump AND be posted on the lagging official table before Dec 31, which is plausible given Google's cadence but far from assured. The neighboring-bucket structure (50% at 84%, 60% at 15%) implies the market's modal outcome is 50–54%, consistent with the Polymarket proxy anchor of 46.5%. Unresolved reports of competitors at 55–65% on third-party aggregators offer some upside, but leaderboard lag and the strict no-tools criterion offset it, so I sit slightly below the anchor.

Pipeline Timing

Total pipeline time: 143.1s

Per-tool research timings shown in the Research section above.