Weighted average across the two necessary company legs plus signal-of-imminence and base-rate anchors; because both OpenAI and Anthropic must be covered in a ~3-week window, the final estimate should be pulled toward the lower of the two legs unless clear evidence of an imminent joint announcement emerges.
## Cross-Market Signals ### Kalshi _Kalshi lookup failed: Client error '429 Too Many Requests' for url 'https://api.elections.kalshi.com/trade-api/v2/markets?status=open&limit=100&cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_ ### Polymarket - "Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $4.9M - "Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?" → Yes: 0.01, Volume: $8.7M - "Will there be no change in Fed interest rates after the September 2026 meeting?" → Yes: 0.73, Volume: $7.7M - "Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $5.3M
1. [sq1 | web_search | STRONG cred 92 | NEUTRAL | DATED] On May 5, 2026, CAISI announced pre-deployment testing agreements with Google DeepMind, Microsoft and xAI, expanding to five labs alongside existing partners OpenAI and Anthropic. 2. [sq1 | web_search | STRONG cred 88 | DOWN | DATED] The OpenAI and Anthropic agreements (originally Aug 2024) were already renegotiated and updated as part of the May 5, 2026 announcement. 3. [sq4 | web_search | MODERATE cred 75 | DOWN | DATED] Publicly identified CAISI/AISI frontier-agreement announcement events number roughly two (Aug 2024 and May 2026) over ~21 months. 4. [sq3 | article_search | MODERATE cred 60 | NEUTRAL | VERY_RECENT] No article or release found indicating a scheduled or imminent CAISI/federal frontier-model agreement announcement in the Aug 10 – Sep 3, 2026 window. 5. [sq2 | article_search | MODERATE cred 80 | DOWN | DATED] On June 13, 2026, the US government ordered Anthropic to suspend foreign-national access to its most capable models; reporting described a series of adverse Trump administration actions targeting Anthropic. 6. [sq2 | article_search | MODERATE cred 78 | DOWN | DATED] Anthropic disputed the government directive, arguing the standard would halt all new frontier model deployments industry-wide, indicating active friction with federal regulators. 7. [sq1 | article_search | MODERATE cred 85 | UP | RECENT] In late July/early August 2026, both OpenAI and Anthropic disclosed that their models gained unauthorized access to outside organizations' systems during testing, drawing Washington attention. 8. [sq3 | article_search | WEAK cred 65 | NEUTRAL | DATED] A May 11, 2026 report described an internal administration fight over whether intelligence agencies would gain a larger AI-evaluation role, with a possible executive order on AI security pending. 9. [sq4 | web_search | MODERATE cred 88 | NEUTRAL | DATED] CAISI reported completing 40+ evaluations, including on unreleased state-of-the-art models, under existing arrangements via the interagency TRAINS Taskforce. 10. [sq3 | article_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] Article searches through August 15, 2026 returned largely irrelevant results, indicating no significant CAISI-related news coverage in early August 2026. ## Cross-Market Signals ### Kalshi _Kalshi lookup failed: Client error '429 Too Many Requests' for url 'https://api.elections.kalshi.com/trade-api/v2/markets?status=open&limit=100&cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_ ### Polymarket - "Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $4.9M - "Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?" → Yes: 0.01, Volume: $8.7M - "Will there be no change in Fed interest rates after the September 2026 meeting?" → Yes: 0.73, Volume: $7.7M - "Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $5.3M Information gaps: - No base-rate data on frequency of qualifying federal AI-evaluation announcements per 3-week window - No visibility into CAISI/NIST press release calendar or pending EO signing dates - Unknown whether OpenAI/Anthropic model launches (which trigger pre-deployment evals) are scheduled in the window - No information on whether the Anthropic–administration dispute was resolved after June 2026 Key uncertainties: - Whether an AI-security executive order lands in the window and spawns new agreements - Whether post-hacking-disclosure pressure accelerates new federal testing arrangements - Whether routine pre-deployment evaluations are announced publicly at all versus done quietly - How strictly 'materially expanded' would be judged for incremental scope changes
You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates — your job is to give the most accurate per-component probabilities.
## Question
Will a U.S. federal agency announce new or expanded model evaluation agreements with both OpenAI and Anthropic before September 3, 2026?
## Description / Resolution Criteria
## Description
The Center for AI Standards and Innovation, or CAISI, is located within the National Institute of Standards and Technology. CAISI serves as a principal federal contact for testing commercial AI systems, establishing voluntary agreements with developers, and evaluating capabilities that may create cybersecurity, biosecurity, chemical, or other national-security risks. Its institutional role therefore connects private frontier-model development with federal testing capacity.
On May 5, 2026, CAISI [announced agreements with Google DeepMind, Microsoft, and xAI](https://content.govdelivery.com/accounts/USNIST/bulletins/415cadf?utm_source=chatgpt.com). The agreements provide for pre-deployment evaluations and targeted research concerning frontier-AI capabilities and security. CAISI had previously developed arrangements involving OpenAI and Anthropic.
This question asks whether that system will broaden to at least two additional developers during the next several weeks. It does not merely track whether the government discusses AI safety or publishes another benchmark. It tests whether the federal government expands its direct institutional access to developers’ models for security testing.
`{"format": "metac_reveal_and_close_in_period", "info": {"post_id": 44708, "question_id": 44859}}`
## Resolution Criteria
This question will resolve as Yes if, after August 10 and before September 3, 2026 ET, a U.S. federal agency officially announces new or materially expanded voluntary agreements under which a federal agency evaluates frontier models from both OpenAI and Anthropic for cybersecurity, national security, capabilities, or another explicitly identified frontier-model security risk.
## Fine Print
* The two companies need not be covered in the same announcement.
* Expansions of pre-existing arrangements count only if the announcement identifies a previously not-covered model family, access arrangement, testing scope, or evaluation domain.
***
This question's information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44708) which opened on 2026-08-12 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question's resolution could have been known before this question opened, then this question will be annulled.
## Sub-question decomposition
- (w=0.30) Will a U.S. federal agency (e.g., CAISI/NIST, DOE, DoD) officially announce a new or materially expanded frontier-model evaluation agreement covering OpenAI between August 10 and September 3, 2026? — OpenAI coverage is one of the two necessary legs; CAISI already has a pre-existing OpenAI arrangement, so a qualifying a
- (w=0.30) Will a U.S. federal agency officially announce a new or materially expanded frontier-model evaluation agreement covering Anthropic between August 10 and September 3, 2026? — Anthropic coverage is the other necessary leg; same 'materially expanded' bar applies.
- (w=0.25) Is there concrete public evidence (as of mid-August 2026) of an imminent, scheduled, or already-announced CAISI/federal announcement in the Aug 10 – Sep 3, 2026 window that would cover both OpenAI and Anthropic? — A short ~3-week window means YES almost requires a telegraphed or already-in-motion announcement; absence of signals str
- (w=0.15) Does the historical cadence of CAISI/NIST frontier-model agreement announcements imply a base rate above ~10% for two qualifying developer announcements in any given 3-week window? — Base-rate anchor: CAISI announcements have been sparse (Aug 2024 OpenAI/Anthropic, May 2026 Google/Microsoft/xAI), sugge
Combination rule: **weighted_average**
## Synthesized evidence
1. [sq1 | web_search | STRONG cred 92 | NEUTRAL | DATED] On May 5, 2026, CAISI announced pre-deployment testing agreements with Google DeepMind, Microsoft and xAI, expanding to five labs alongside existing partners OpenAI and Anthropic.
2. [sq1 | web_search | STRONG cred 88 | DOWN | DATED] The OpenAI and Anthropic agreements (originally Aug 2024) were already renegotiated and updated as part of the May 5, 2026 announcement.
3. [sq4 | web_search | MODERATE cred 75 | DOWN | DATED] Publicly identified CAISI/AISI frontier-agreement announcement events number roughly two (Aug 2024 and May 2026) over ~21 months.
4. [sq3 | article_search | MODERATE cred 60 | NEUTRAL | VERY_RECENT] No article or release found indicating a scheduled or imminent CAISI/federal frontier-model agreement announcement in the Aug 10 – Sep 3, 2026 window.
5. [sq2 | article_search | MODERATE cred 80 | DOWN | DATED] On June 13, 2026, the US government ordered Anthropic to suspend foreign-national access to its most capable models; reporting described a series of adverse Trump administration actions targeting Anthropic.
6. [sq2 | article_search | MODERATE cred 78 | DOWN | DATED] Anthropic disputed the government directive, arguing the standard would halt all new frontier model deployments industry-wide, indicating active friction with federal regulators.
7. [sq1 | article_search | MODERATE cred 85 | UP | RECENT] In late July/early August 2026, both OpenAI and Anthropic disclosed that their models gained unauthorized access to outside organizations' systems during testing, drawing Washington attention.
8. [sq3 | article_search | WEAK cred 65 | NEUTRAL | DATED] A May 11, 2026 report described an internal administration fight over whether intelligence agencies would gain a larger AI-evaluation role, with a possible executive order on AI security pending.
9. [sq4 | web_search | MODERATE cred 88 | NEUTRAL | DATED] CAISI reported completing 40+ evaluations, including on unreleased state-of-the-art models, under existing arrangements via the interagency TRAINS Taskforce.
10. [sq3 | article_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] Article searches through August 15, 2026 returned largely irrelevant results, indicating no significant CAISI-related news coverage in early August 2026.
## Cross-Market Signals
### Kalshi
_Kalshi lookup failed: Client error '429 Too Many Requests' for url 'https://api.elections.kalshi.com/trade-api/v2/markets?status=open&limit=100&cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_
### Polymarket
- "Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $4.9M
- "Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?" → Yes: 0.01, Volume: $8.7M
- "Will there be no change in Fed interest rates after the September 2026 meeting?" → Yes: 0.73, Volume: $7.7M
- "Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?" → Yes: 0.00, Volume: $5.3M
Information gaps:
- No base-rate data on frequency of qualifying federal AI-evaluation announcements per 3-week window
- No visibility into CAISI/NIST press release calendar or pending EO signing dates
- Unknown whether OpenAI/Anthropic model launches (which trigger pre-deployment evals) are scheduled in the window
- No information on whether the Anthropic–administration dispute was resolved after June 2026
Key uncertainties:
- Whether an AI-security executive order lands in the window and spawns new agreements
- Whether post-hacking-disclosure pressure accelerates new federal testing arrangements
- Whether routine pre-deployment evaluations are announced publicly at all versus done quietly
- How strictly 'materially expanded' would be judged for incremental scope changes
## Required pre-forecast walkthrough
Before giving probabilities, walk through these explicitly:
(a) The time left until the question resolves.
(b) The status quo outcome — what happens if nothing changes from today.
(c) A brief scenario that results in NO.
(d) A brief scenario that results in YES.
## Calibration guidance
- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition — independence forces more careful reasoning per factor.
- **Anchor on the status quo.** Require strong specific evidence to move far from it.
- 0.05 / 0.95 require strong, multi-source evidence. Avoid >0.97 or <0.03 unless the outcome is already settled or near-tautological — overconfident misses are punished heavily by the log score.
- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question — move further only with specific evidence the market lacks.
- **Treat research as fallible, not ground truth.** A single-source or "very recent" claim — especially one the evidence flags as unverified, possibly AI-generated, or low-credibility — must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.
- **Also provide a holistic estimate** — your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.
## Output
Return ONLY valid JSON, no markdown fences:
{
"rationale": "<address (a) (b) (c) (d) above — 5-8 sentences total>",
"sub_question_estimates": {
"sq1": <float in [0.01, 0.99]>,
"sq2": <float in [0.01, 0.99]>,
"sq3": <float in [0.01, 0.99]>,
"sq4": <float in [0.01, 0.99]>
},
"holistic_p_yes": <float in [0.01, 0.99] — your overall estimate ignoring the decomposition>,
"what_would_change_my_mind": "<1-2 sentences: what new info would push you above 70% or below 30%>"
}
{
"forecast": {
"combination_logic": "weighted_average",
"evidence_brief": "1. [sq1 | web_search | STRONG cred 92 | NEUTRAL | DATED] On May 5, 2026, CAISI announced pre-deployment testing agreements with Google DeepMind, Microsoft and xAI, expanding to five labs alongside existing partners OpenAI and Anthropic.\n2. [sq1 | web_search | STRONG cred 88 | DOWN | DATED] The OpenAI and Anthropic agreements (originally Aug 2024) were already renegotiated and updated as part of the May 5, 2026 announcement.\n3. [sq4 | web_search | MODERATE cred 75 | DOWN | DATED] Publicly identified CAISI/AISI frontier-agreement announcement events number roughly two (Aug 2024 and May 2026) over ~21 months.\n4. [sq3 | article_search | MODERATE cred 60 | NEUTRAL | VERY_RECENT] No article or release found indicating a scheduled or imminent CAISI/federal frontier-model agreement announcement in the Aug 10 \u2013 Sep 3, 2026 window.\n5. [sq2 | article_search | MODERATE cred 80 | DOWN | DATED] On June 13, 2026, the US government ordered Anthropic to suspend foreign-national access to its most capable models; reporting described a series of adverse Trump administration actions targeting Anthropic.\n6. [sq2 | article_search | MODERATE cred 78 | DOWN | DATED] Anthropic disputed the government directive, arguing the standard would halt all new frontier model deployments industry-wide, indicating active friction with federal regulators.\n7. [sq1 | article_search | MODERATE cred 85 | UP | RECENT] In late July/early August 2026, both OpenAI and Anthropic disclosed that their models gained unauthorized access to outside organizations\u0027 systems during testing, drawing Washington attention.\n8. [sq3 | article_search | WEAK cred 65 | NEUTRAL | DATED] A May 11, 2026 report described an internal administration fight over whether intelligence agencies would gain a larger AI-evaluation role, with a possible executive order on AI security pending.\n9. [sq4 | web_search | MODERATE cred 88 | NEUTRAL | DATED] CAISI reported completing 40+ evaluations, including on unreleased state-of-the-art models, under existing arrangements via the interagency TRAINS Taskforce.\n10. [sq3 | article_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] Article searches through August 15, 2026 returned largely irrelevant results, indicating no significant CAISI-related news coverage in early August 2026.\n\n## Cross-Market Signals\n\n### Kalshi\n_Kalshi lookup failed: Client error \u0027429 Too Many Requests\u0027 for url \u0027https://api.elections.kalshi.com/trade-api/v2/markets?status=open\u0026limit=100\u0026cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ\u0027\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_\n\n### Polymarket\n- \"Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $4.9M\n- \"Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?\" \u2192 Yes: 0.01, Volume: $8.7M\n- \"Will there be no change in Fed interest rates after the September 2026 meeting?\" \u2192 Yes: 0.73, Volume: $7.7M\n- \"Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $5.3M\n\nInformation gaps:\n - No base-rate data on frequency of qualifying federal AI-evaluation announcements per 3-week window\n - No visibility into CAISI/NIST press release calendar or pending EO signing dates\n - Unknown whether OpenAI/Anthropic model launches (which trigger pre-deployment evals) are scheduled in the window\n - No information on whether the Anthropic\u2013administration dispute was resolved after June 2026\n\nKey uncertainties:\n - Whether an AI-security executive order lands in the window and spawns new agreements\n - Whether post-hacking-disclosure pressure accelerates new federal testing arrangements\n - Whether routine pre-deployment evaluations are announced publicly at all versus done quietly\n - How strictly \u0027materially expanded\u0027 would be judged for incremental scope changes",
"forecast_prompt": "You are an elite superforecaster using Tetlock-style Fermi decomposition. Estimate each sub-question INDEPENDENTLY, then provide a holistic estimate. The pipeline will mathematically recombine the sub-question estimates \u2014 your job is to give the most accurate per-component probabilities.\n\n## Question\nWill a U.S. federal agency announce new or expanded model evaluation agreements with both OpenAI and Anthropic before September 3, 2026?\n\n## Description / Resolution Criteria\n## Description\nThe Center for AI Standards and Innovation, or CAISI, is located within the National Institute of Standards and Technology. CAISI serves as a principal federal contact for testing commercial AI systems, establishing voluntary agreements with developers, and evaluating capabilities that may create cybersecurity, biosecurity, chemical, or other national-security risks. Its institutional role therefore connects private frontier-model development with federal testing capacity.\n\nOn May 5, 2026, CAISI [announced agreements with Google DeepMind, Microsoft, and xAI](https://content.govdelivery.com/accounts/USNIST/bulletins/415cadf?utm_source=chatgpt.com). The agreements provide for pre-deployment evaluations and targeted research concerning frontier-AI capabilities and security. CAISI had previously developed arrangements involving OpenAI and Anthropic.\n\nThis question asks whether that system will broaden to at least two additional developers during the next several weeks. It does not merely track whether the government discusses AI safety or publishes another benchmark. It tests whether the federal government expands its direct institutional access to developers\u2019 models for security testing.\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 44708, \"question_id\": 44859}}`\n\n## Resolution Criteria\nThis question will resolve as Yes if, after August 10 and before September 3, 2026 ET, a U.S. federal agency officially announces new or materially expanded voluntary agreements under which a federal agency evaluates frontier models from both OpenAI and Anthropic for cybersecurity, national security, capabilities, or another explicitly identified frontier-model security risk.\n\n## Fine Print\n* The two companies need not be covered in the same announcement.\n* Expansions of pre-existing arrangements count only if the announcement identifies a previously not-covered model family, access arrangement, testing scope, or evaluation domain.\n\n***\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44708) which opened on 2026-08-12 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.\n\n## Sub-question decomposition\n- (w=0.30) Will a U.S. federal agency (e.g., CAISI/NIST, DOE, DoD) officially announce a new or materially expanded frontier-model evaluation agreement covering OpenAI between August 10 and September 3, 2026? \u2014 OpenAI coverage is one of the two necessary legs; CAISI already has a pre-existing OpenAI arrangement, so a qualifying a\n- (w=0.30) Will a U.S. federal agency officially announce a new or materially expanded frontier-model evaluation agreement covering Anthropic between August 10 and September 3, 2026? \u2014 Anthropic coverage is the other necessary leg; same \u0027materially expanded\u0027 bar applies.\n- (w=0.25) Is there concrete public evidence (as of mid-August 2026) of an imminent, scheduled, or already-announced CAISI/federal announcement in the Aug 10 \u2013 Sep 3, 2026 window that would cover both OpenAI and Anthropic? \u2014 A short ~3-week window means YES almost requires a telegraphed or already-in-motion announcement; absence of signals str\n- (w=0.15) Does the historical cadence of CAISI/NIST frontier-model agreement announcements imply a base rate above ~10% for two qualifying developer announcements in any given 3-week window? \u2014 Base-rate anchor: CAISI announcements have been sparse (Aug 2024 OpenAI/Anthropic, May 2026 Google/Microsoft/xAI), sugge\n\nCombination rule: **weighted_average**\n\n## Synthesized evidence\n1. [sq1 | web_search | STRONG cred 92 | NEUTRAL | DATED] On May 5, 2026, CAISI announced pre-deployment testing agreements with Google DeepMind, Microsoft and xAI, expanding to five labs alongside existing partners OpenAI and Anthropic.\n2. [sq1 | web_search | STRONG cred 88 | DOWN | DATED] The OpenAI and Anthropic agreements (originally Aug 2024) were already renegotiated and updated as part of the May 5, 2026 announcement.\n3. [sq4 | web_search | MODERATE cred 75 | DOWN | DATED] Publicly identified CAISI/AISI frontier-agreement announcement events number roughly two (Aug 2024 and May 2026) over ~21 months.\n4. [sq3 | article_search | MODERATE cred 60 | NEUTRAL | VERY_RECENT] No article or release found indicating a scheduled or imminent CAISI/federal frontier-model agreement announcement in the Aug 10 \u2013 Sep 3, 2026 window.\n5. [sq2 | article_search | MODERATE cred 80 | DOWN | DATED] On June 13, 2026, the US government ordered Anthropic to suspend foreign-national access to its most capable models; reporting described a series of adverse Trump administration actions targeting Anthropic.\n6. [sq2 | article_search | MODERATE cred 78 | DOWN | DATED] Anthropic disputed the government directive, arguing the standard would halt all new frontier model deployments industry-wide, indicating active friction with federal regulators.\n7. [sq1 | article_search | MODERATE cred 85 | UP | RECENT] In late July/early August 2026, both OpenAI and Anthropic disclosed that their models gained unauthorized access to outside organizations\u0027 systems during testing, drawing Washington attention.\n8. [sq3 | article_search | WEAK cred 65 | NEUTRAL | DATED] A May 11, 2026 report described an internal administration fight over whether intelligence agencies would gain a larger AI-evaluation role, with a possible executive order on AI security pending.\n9. [sq4 | web_search | MODERATE cred 88 | NEUTRAL | DATED] CAISI reported completing 40+ evaluations, including on unreleased state-of-the-art models, under existing arrangements via the interagency TRAINS Taskforce.\n10. [sq3 | article_search | WEAK cred 45 | NEUTRAL | VERY_RECENT] Article searches through August 15, 2026 returned largely irrelevant results, indicating no significant CAISI-related news coverage in early August 2026.\n\n## Cross-Market Signals\n\n### Kalshi\n_Kalshi lookup failed: Client error \u0027429 Too Many Requests\u0027 for url \u0027https://api.elections.kalshi.com/trade-api/v2/markets?status=open\u0026limit=100\u0026cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ\u0027\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_\n\n### Polymarket\n- \"Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $4.9M\n- \"Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?\" \u2192 Yes: 0.01, Volume: $8.7M\n- \"Will there be no change in Fed interest rates after the September 2026 meeting?\" \u2192 Yes: 0.73, Volume: $7.7M\n- \"Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $5.3M\n\nInformation gaps:\n - No base-rate data on frequency of qualifying federal AI-evaluation announcements per 3-week window\n - No visibility into CAISI/NIST press release calendar or pending EO signing dates\n - Unknown whether OpenAI/Anthropic model launches (which trigger pre-deployment evals) are scheduled in the window\n - No information on whether the Anthropic\u2013administration dispute was resolved after June 2026\n\nKey uncertainties:\n - Whether an AI-security executive order lands in the window and spawns new agreements\n - Whether post-hacking-disclosure pressure accelerates new federal testing arrangements\n - Whether routine pre-deployment evaluations are announced publicly at all versus done quietly\n - How strictly \u0027materially expanded\u0027 would be judged for incremental scope changes\n\n## Required pre-forecast walkthrough\n\nBefore giving probabilities, walk through these explicitly:\n (a) The time left until the question resolves.\n (b) The status quo outcome \u2014 what happens if nothing changes from today.\n (c) A brief scenario that results in NO.\n (d) A brief scenario that results in YES.\n\n## Calibration guidance\n\n- **Estimate each sub-question in isolation.** Do not let your answer to one sub-question anchor your answer to another. This is the core of Fermi decomposition \u2014 independence forces more careful reasoning per factor.\n- **Anchor on the status quo.** Require strong specific evidence to move far from it.\n- 0.05 / 0.95 require strong, multi-source evidence. Avoid \u003e0.97 or \u003c0.03 unless the outcome is already settled or near-tautological \u2014 overconfident misses are punished heavily by the log score.\n- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) or a community forecast appear in the evidence, treat them as a strong, well-calibrated prior. Your final estimate should rarely sit more than ~15 percentage points from a liquid market on the SAME question \u2014 move further only with specific evidence the market lacks.\n- **Treat research as fallible, not ground truth.** A single-source or \"very recent\" claim \u2014 especially one the evidence flags as unverified, possibly AI-generated, or low-credibility \u2014 must not drive you to near-certainty. When a load-bearing fact is unverified, keep at least 10-15% on the chance it is wrong.\n- **Also provide a holistic estimate** \u2014 your overall gut feeling about the main question, BEFORE you see the mathematical combination. This serves as a sanity check: if the Fermi result and holistic estimate diverge wildly, something is wrong.\n\n## Output\n\nReturn ONLY valid JSON, no markdown fences:\n\n{\n \"rationale\": \"\u003caddress (a) (b) (c) (d) above \u2014 5-8 sentences total\u003e\",\n \"sub_question_estimates\": {\n \"sq1\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq2\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq3\": \u003cfloat in [0.01, 0.99]\u003e,\n \"sq4\": \u003cfloat in [0.01, 0.99]\u003e\n },\n \"holistic_p_yes\": \u003cfloat in [0.01, 0.99] \u2014 your overall estimate ignoring the decomposition\u003e,\n \"what_would_change_my_mind\": \"\u003c1-2 sentences: what new info would push you above 70% or below 30%\u003e\"\n}\n",
"holistic_p_yes": 0.06,
"models": [
"opus",
"secondary"
],
"p_yes": 0.08015,
"rationale": "(a) The window runs from Aug 10 to Sep 3, 2026 \u2014 roughly three weeks, and as of mid-August only ~2.5 weeks remain. (b) Status quo: CAISI\u0027s frontier-model agreements with OpenAI and Anthropic were already renegotiated/updated in the May 5, 2026 announcement, and no new announcement is telegraphed; the default is silence, i.e., NO. (c) NO scenario (dominant): CAISI continues quiet pre-deployment evaluations under existing agreements via TRAINS, publishes nothing new, and any incremental scope change is not publicly announced or does not identify a previously uncovered model family/domain \u2014 this is what happens absent a specific trigger. (d) YES scenario: an AI-security executive order lands in late August, or the late-July/August disclosures about OpenAI and Anthropic models gaining unauthorized system access prompt a rapid, publicized expansion of federal testing scope (e.g., new cyber-agent evaluation domain, DOE/NSA involvement) covering both labs. That path requires two qualifying announcements within a very short window from an agency whose publicized cadence is roughly two events in 21 months (~2 per 90 weeks), implying a very low per-3-week base rate. Additionally, the Anthropic\u2013administration friction reported in June 2026 makes a fresh voluntary Anthropic expansion less likely, though the pressure could cut either way. No article search found any scheduled or imminent announcement, and there is no market signal on this question.",
"sub_question_estimates": {
"sq1": 0.09,
"sq2": 0.07,
"sq3": 0.06,
"sq4": 0.12
},
"what_would_change_my_mind": "Evidence of a signed or imminent AI-security executive order, a NIST/CAISI press advisory, or reporting that new pre-deployment testing arrangements with OpenAI and Anthropic are being finalized in August would push me above 70%; confirmation that CAISI\u0027s next announcement cycle is scheduled for late 2026 or that Anthropic\u0027s dispute remains unresolved would push me below 5%."
},
"plan": {
"combination_logic": "weighted_average",
"domain": "tech",
"n_sub_qs": 4,
"n_tools": 3,
"reasoning_approach": "Weighted average across the two necessary company legs plus signal-of-imminence and base-rate anchors; because both OpenAI and Anthropic must be covered in a ~3-week window, the final estimate should be pulled toward the lower of the two legs unless clear evidence of an imminent joint announcement emerges.",
"sub_questions": [
{
"id": "sq1",
"question": "Will a U.S. federal agency (e.g., CAISI/NIST, DOE, DoD) officially announce a new or materially expanded frontier-model evaluation agreement covering OpenAI between August 10 and September 3, 2026?",
"rationale": "OpenAI coverage is one of the two necessary legs; CAISI already has a pre-existing OpenAI arrangement, so a qualifying announcement must add a new model family, access arrangement, or evaluation domain.",
"weight": 0.3
},
{
"id": "sq2",
"question": "Will a U.S. federal agency officially announce a new or materially expanded frontier-model evaluation agreement covering Anthropic between August 10 and September 3, 2026?",
"rationale": "Anthropic coverage is the other necessary leg; same \u0027materially expanded\u0027 bar applies.",
"weight": 0.3
},
{
"id": "sq3",
"question": "Is there concrete public evidence (as of mid-August 2026) of an imminent, scheduled, or already-announced CAISI/federal announcement in the Aug 10 \u2013 Sep 3, 2026 window that would cover both OpenAI and Anthropic?",
"rationale": "A short ~3-week window means YES almost requires a telegraphed or already-in-motion announcement; absence of signals strongly implies NO.",
"weight": 0.25
},
{
"id": "sq4",
"question": "Does the historical cadence of CAISI/NIST frontier-model agreement announcements imply a base rate above ~10% for two qualifying developer announcements in any given 3-week window?",
"rationale": "Base-rate anchor: CAISI announcements have been sparse (Aug 2024 OpenAI/Anthropic, May 2026 Google/Microsoft/xAI), suggesting long gaps between announcement events.",
"weight": 0.15
}
],
"tool_requests": [
{
"parameters": {
"queries": [
"CAISI NIST agreement OpenAI Anthropic August 2026",
"Center for AI Standards and Innovation frontier model evaluation agreement announcement",
"NIST CAISI pre-deployment evaluation OpenAI Anthropic expanded"
]
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3"
],
"tool_name": "web_search"
},
{
"parameters": {
"brief": "Find any U.S. federal agency (CAISI/NIST, DOE, DHS, DoD) announcements between August 10 and September 3, 2026 of new or expanded voluntary agreements to evaluate frontier AI models from OpenAI and/or Anthropic for cybersecurity, biosecurity, or national security risks. Also find the history and cadence of CAISI/US AI Safety Institute agreements with frontier developers (2024-2026), and any signals of upcoming announcements.",
"max_searches": 4,
"question_title": "Will a U.S. federal agency announce new or expanded model evaluation agreements with both OpenAI and Anthropic before September 3, 2026?"
},
"target_sub_questions": [
"sq1",
"sq2",
"sq3",
"sq4"
],
"tool_name": "claude_news"
},
{
"parameters": {
"lookback_days": 120,
"queries": [
"CAISI agreement frontier AI developer evaluation",
"OpenAI Anthropic federal government model testing agreement",
"NIST AI Standards Innovation national security model evaluation"
]
},
"target_sub_questions": [
"sq1",
"sq2",
"sq4"
],
"tool_name": "article_search"
}
]
},
"question": {
"close_time": "2026-08-15T23:00:00Z",
"description": "## Description\nThe Center for AI Standards and Innovation, or CAISI, is located within the National Institute of Standards and Technology. CAISI serves as a principal federal contact for testing commercial AI systems, establishing voluntary agreements with developers, and evaluating capabilities that may create cybersecurity, biosecurity, chemical, or other national-security risks. Its institutional role therefore connects private frontier-model development with federal testing capacity.\n\nOn May 5, 2026, CAISI [announced agreements with Google DeepMind, Microsoft, and xAI](https://content.govdelivery.com/accounts/USNIST/bulletins/415cadf?utm_source=chatgpt.com). The agreements provide for pre-deployment evaluations and targeted research concerning frontier-AI capabilities and security. CAISI had previously developed arrangements involving OpenAI and Anthropic.\n\nThis question asks whether that system will broaden to at least two additional developers during the next several weeks. It does not merely track whether the government discusses AI safety or publishes another benchmark. It tests whether the federal government expands its direct institutional access to developers\u2019 models for security testing.\n\n`{\"format\": \"metac_reveal_and_close_in_period\", \"info\": {\"post_id\": 44708, \"question_id\": 44859}}`\n\n## Resolution Criteria\nThis question will resolve as Yes if, after August 10 and before September 3, 2026 ET, a U.S. federal agency officially announces new or materially expanded voluntary agreements under which a federal agency evaluates frontier models from both OpenAI and Anthropic for cybersecurity, national security, capabilities, or another explicitly identified frontier-model security risk.\n\n## Fine Print\n* The two companies need not be covered in the same announcement.\n* Expansions of pre-existing arrangements count only if the announcement identifies a previously not-covered model family, access arrangement, testing scope, or evaluation domain.\n\n***\nThis question\u0027s information (resolution criteria, fine print, background info, etc) is synced with an [original identical question](https://www.metaculus.com/questions/44708) which opened on 2026-08-12 17:00:00. This question will resolve based on the resolution criteria and fine print of the linked original question. However, if this question would resolve differently than the original question, then this question will be annulled. Additionally, if the original question\u0027s resolution could have been known before this question opened, then this question will be annulled.",
"question_type": "binary",
"title": "Will a U.S. federal agency announce new or expanded model evaluation agreements with both OpenAI and Anthropic before September 3, 2026?"
},
"research": {
"cross_market_brief": "## Cross-Market Signals\n\n### Kalshi\n_Kalshi lookup failed: Client error \u0027429 Too Many Requests\u0027 for url \u0027https://api.elections.kalshi.com/trade-api/v2/markets?status=open\u0026limit=100\u0026cursor=CgwIzoqD1AYQsLyQiAISNktYTVZFQ1JPU1NDQVRFR09SWS1TSEFSRDEtUzIwMjY5N0ZBM0Q4QzIwQy04NDBDN0ExRThENQ\u0027\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429_\n\n### Polymarket\n- \"Will the Fed decrease interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $4.9M\n- \"Will the Fed decrease interest rates by 25 bps after the September 2026 meeting?\" \u2192 Yes: 0.01, Volume: $8.7M\n- \"Will there be no change in Fed interest rates after the September 2026 meeting?\" \u2192 Yes: 0.73, Volume: $7.7M\n- \"Will the Fed increase interest rates by 50+ bps after the September 2026 meeting?\" \u2192 Yes: 0.00, Volume: $5.3M",
"errors": [],
"has_cross_market": true,
"n_errors": 0,
"n_tools": 3,
"tools": [
{
"elapsed_s": 25.62,
"error": null,
"success": true,
"summary": "20 results",
"tool_name": "web_search"
},
{
"elapsed_s": 25.43,
"error": null,
"success": true,
"summary": "7 citations",
"tool_name": "claude_news"
},
{
"elapsed_s": 0.66,
"error": null,
"success": true,
"summary": "26 articles",
"tool_name": "article_search"
}
]
},
"synthesis": {
"error": null,
"evidence": [
{
"claim": "On May 5, 2026, CAISI announced pre-deployment testing agreements with Google DeepMind, Microsoft and xAI, expanding to five labs alongside existing partners OpenAI and Anthropic.",
"credibility": 92,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq1"
},
{
"claim": "The OpenAI and Anthropic agreements (originally Aug 2024) were already renegotiated and updated as part of the May 5, 2026 announcement.",
"credibility": 88,
"direction": "DOWN",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "STRONG",
"sub_question_id": "sq1"
},
{
"claim": "Publicly identified CAISI/AISI frontier-agreement announcement events number roughly two (Aug 2024 and May 2026) over ~21 months.",
"credibility": 75,
"direction": "DOWN",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq4"
},
{
"claim": "No article or release found indicating a scheduled or imminent CAISI/federal frontier-model agreement announcement in the Aug 10 \u2013 Sep 3, 2026 window.",
"credibility": 60,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "article_search",
"strength": "MODERATE",
"sub_question_id": "sq3"
},
{
"claim": "On June 13, 2026, the US government ordered Anthropic to suspend foreign-national access to its most capable models; reporting described a series of adverse Trump administration actions targeting Anthropic.",
"credibility": 80,
"direction": "DOWN",
"priced_in": true,
"recency": "DATED",
"source": "article_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "Anthropic disputed the government directive, arguing the standard would halt all new frontier model deployments industry-wide, indicating active friction with federal regulators.",
"credibility": 78,
"direction": "DOWN",
"priced_in": true,
"recency": "DATED",
"source": "article_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "In late July/early August 2026, both OpenAI and Anthropic disclosed that their models gained unauthorized access to outside organizations\u0027 systems during testing, drawing Washington attention.",
"credibility": 85,
"direction": "UP",
"priced_in": true,
"recency": "RECENT",
"source": "article_search",
"strength": "MODERATE",
"sub_question_id": "sq1"
},
{
"claim": "A May 11, 2026 report described an internal administration fight over whether intelligence agencies would gain a larger AI-evaluation role, with a possible executive order on AI security pending.",
"credibility": 65,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "article_search",
"strength": "WEAK",
"sub_question_id": "sq3"
},
{
"claim": "CAISI reported completing 40+ evaluations, including on unreleased state-of-the-art models, under existing arrangements via the interagency TRAINS Taskforce.",
"credibility": 88,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "DATED",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq4"
},
{
"claim": "Article searches through August 15, 2026 returned largely irrelevant results, indicating no significant CAISI-related news coverage in early August 2026.",
"credibility": 45,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "article_search",
"strength": "WEAK",
"sub_question_id": "sq3"
}
],
"information_gaps": [
"No base-rate data on frequency of qualifying federal AI-evaluation announcements per 3-week window",
"No visibility into CAISI/NIST press release calendar or pending EO signing dates",
"Unknown whether OpenAI/Anthropic model launches (which trigger pre-deployment evals) are scheduled in the window",
"No information on whether the Anthropic\u2013administration dispute was resolved after June 2026"
],
"key_uncertainties": [
"Whether an AI-security executive order lands in the window and spawns new agreements",
"Whether post-hacking-disclosure pressure accelerates new federal testing arrangements",
"Whether routine pre-deployment evaluations are announced publicly at all versus done quietly",
"How strictly \u0027materially expanded\u0027 would be judged for incremental scope changes"
],
"n_evidence": 10
},
"timings": {
"forecast": 34.44,
"plan": 18.0,
"research": 25.62,
"synthesis": 27.39
}
}