Anchor on the current exact-filter count (sq1), then adjust for expected net change and growth momentum (sq2, sq3) via a weighted average, mapping the blended estimate onto the multiple-choice count buckets.
## Cross-Market Signals ### No signal found
1. [sq1 | web_search | MODERATE cred 55 | NEUTRAL | RECENT] Named LLM-chatbot trials found include NCT06346496, NCT06801925, NCT07132125, but most cited trials are completed or online-only, not currently recruiting. 2. [sq1 | web_search | MODERATE cred 50 | DOWN | RECENT] The exact clinicaltrials.gov filter 'LLM-based Chatbot' is a narrow term; several relevant trials use varied intervention naming and may not match this precise filter. 3. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] No search result reported an exact current count of recruiting studies matching the specific filter link. 4. [sq2 | web_search | MODERATE cred 60 | UP | RECENT] NCT07280650 (University of Michigan) is 'NOT YET RECRUITING' with estimated start August 2026, potentially entering recruiting status near resolution date. 5. [sq2 | web_search | WEAK cred 45 | DOWN | RECENT] Some referenced trials had ethics approval or study windows ending in early 2026, suggesting they may no longer be recruiting by August 2026. 6. [sq3 | article_search | WEAK cred 30 | NEUTRAL | VERY_RECENT] Article search returned no relevant articles about clinical trial registration momentum for LLM chatbots; results were off-topic. 7. [sq3 | web_search | MODERATE cred 55 | UP | RECENT] Multiple new LLM-chatbot health trials were registered across 2025 (depression, HPV vaccine, caregivers), indicating an active but modest registration trend. 8. [sq3 | claude_news | WEAK cred 40 | NEUTRAL | RECENT] General news reflects surging interest and scrutiny of medical/mental-health AI chatbots but provides no specific clinical-trial registration counts. ## Cross-Market Signals ### No signal found Information gaps: - Exact current count on the specific filter link (unretrieved) - Historical growth rate of trials matching this exact filter - How many named trials actually use 'LLM-based Chatbot' as the intervention label - Recruiting-status transitions expected between now and Aug 2026 Key uncertainties: - Whether narrow filter matches only a handful of trials - Whether NCT07280650 flips to Recruiting by Aug 1, 2026 - Net effect of new registrations vs completed/closed trials - Sensitivity of count to exact intervention wording
You are an elite superforecaster. Estimate the probability of each option for this Metaculus multiple-choice question.
## Question
How many clinicaltrials.gov studies with an LLM-based Chatbot as the intervention/treatment will be recruiting on August 1, 2026 ?
## Description / Resolution Criteria
## Description
IEEE Spectrum: [Can AI Chatbots Reason Like Doctors?](https://spectrum.ieee.org/ai-clinical-decision-support)
`{"format": "bot_tournament_question", "info": {"hash_id": "b3031f1f43b0491c", "sheet_id": "88"}}`
## Resolution Criteria
This question resolves as the number of studies with an Intervention/treatment of *LLM-based Chatbot* that have a study status of Recruiting are listed on August 1, 2026 at this link: https://clinicaltrials.gov/search?viewType=Card&intr=LLM-based%20Chatbot&aggFilters=status:rec
## Options
- 0 or 1
- 2
- 3
- 4
- >4
## Sub-question decomposition (planner)
- (w=0.40) Is the current number of recruiting studies matching the 'LLM-based Chatbot' intervention filter on clinicaltrials.gov in the low single digits (roughly 0-3)? — The exact search filter is very narrow; establishing the current baseline count is the single most important anchor for
- (w=0.35) Will the count of recruiting LLM-chatbot trials remain roughly stable (not grow by several new studies) by August 1, 2026? — Recruiting status is transient (studies complete or start), so net change over ~9 months determines whether the count mo
- (w=0.25) Is there strong momentum in registering LLM/chatbot clinical trials that could push the count meaningfully higher by mid-2026? — LLMs in medicine is a fast-growing area; a surge in new registrations could raise the count.
## Synthesized evidence
1. [sq1 | web_search | MODERATE cred 55 | NEUTRAL | RECENT] Named LLM-chatbot trials found include NCT06346496, NCT06801925, NCT07132125, but most cited trials are completed or online-only, not currently recruiting.
2. [sq1 | web_search | MODERATE cred 50 | DOWN | RECENT] The exact clinicaltrials.gov filter 'LLM-based Chatbot' is a narrow term; several relevant trials use varied intervention naming and may not match this precise filter.
3. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] No search result reported an exact current count of recruiting studies matching the specific filter link.
4. [sq2 | web_search | MODERATE cred 60 | UP | RECENT] NCT07280650 (University of Michigan) is 'NOT YET RECRUITING' with estimated start August 2026, potentially entering recruiting status near resolution date.
5. [sq2 | web_search | WEAK cred 45 | DOWN | RECENT] Some referenced trials had ethics approval or study windows ending in early 2026, suggesting they may no longer be recruiting by August 2026.
6. [sq3 | article_search | WEAK cred 30 | NEUTRAL | VERY_RECENT] Article search returned no relevant articles about clinical trial registration momentum for LLM chatbots; results were off-topic.
7. [sq3 | web_search | MODERATE cred 55 | UP | RECENT] Multiple new LLM-chatbot health trials were registered across 2025 (depression, HPV vaccine, caregivers), indicating an active but modest registration trend.
8. [sq3 | claude_news | WEAK cred 40 | NEUTRAL | RECENT] General news reflects surging interest and scrutiny of medical/mental-health AI chatbots but provides no specific clinical-trial registration counts.
## Cross-Market Signals
### No signal found
Information gaps:
- Exact current count on the specific filter link (unretrieved)
- Historical growth rate of trials matching this exact filter
- How many named trials actually use 'LLM-based Chatbot' as the intervention label
- Recruiting-status transitions expected between now and Aug 2026
Key uncertainties:
- Whether narrow filter matches only a handful of trials
- Whether NCT07280650 flips to Recruiting by Aug 1, 2026
- Net effect of new registrations vs completed/closed trials
- Sensitivity of count to exact intervention wording
## Required pre-forecast walkthrough
Before giving probabilities, address these explicitly in your rationale:
(a) The time left until the question resolves.
(b) The status quo outcome — the option most likely if nothing changes from today.
(c) An outcome that would be SURPRISING but plausible.
## Calibration guidance
- **Status quo anchoring.** Good forecasters put extra weight on the status quo option because the world changes slowly most of the time.
- **Leave moderate probability on most options.** Good forecasters don't drop options to ~0% — leave ≥3-5% on every option to account for unexpected outcomes, unless you have strong specific evidence ruling them out.
- Avoid uniform-distribution laziness — most options are not equally likely.
- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) for these options appear in the evidence, treat them as a strong prior and stay close to them absent specific contrary evidence.
## Output
Return ONLY valid JSON, no markdown fences:
{
"rationale": "<address (a) (b) (c) above, then explain your distribution — 4-7 sentences total>",
"probabilities": {
"0 or 1": <prob>,
"2": <prob>,
"3": <prob>,
"4": <prob>,
">4": <prob>
}
}
Probabilities MUST sum to ~1.0 (renormalization is applied; close enough is fine).
{
"forecast": {
"evidence_brief": "1. [sq1 | web_search | MODERATE cred 55 | NEUTRAL | RECENT] Named LLM-chatbot trials found include NCT06346496, NCT06801925, NCT07132125, but most cited trials are completed or online-only, not currently recruiting.\n2. [sq1 | web_search | MODERATE cred 50 | DOWN | RECENT] The exact clinicaltrials.gov filter \u0027LLM-based Chatbot\u0027 is a narrow term; several relevant trials use varied intervention naming and may not match this precise filter.\n3. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] No search result reported an exact current count of recruiting studies matching the specific filter link.\n4. [sq2 | web_search | MODERATE cred 60 | UP | RECENT] NCT07280650 (University of Michigan) is \u0027NOT YET RECRUITING\u0027 with estimated start August 2026, potentially entering recruiting status near resolution date.\n5. [sq2 | web_search | WEAK cred 45 | DOWN | RECENT] Some referenced trials had ethics approval or study windows ending in early 2026, suggesting they may no longer be recruiting by August 2026.\n6. [sq3 | article_search | WEAK cred 30 | NEUTRAL | VERY_RECENT] Article search returned no relevant articles about clinical trial registration momentum for LLM chatbots; results were off-topic.\n7. [sq3 | web_search | MODERATE cred 55 | UP | RECENT] Multiple new LLM-chatbot health trials were registered across 2025 (depression, HPV vaccine, caregivers), indicating an active but modest registration trend.\n8. [sq3 | claude_news | WEAK cred 40 | NEUTRAL | RECENT] General news reflects surging interest and scrutiny of medical/mental-health AI chatbots but provides no specific clinical-trial registration counts.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - Exact current count on the specific filter link (unretrieved)\n - Historical growth rate of trials matching this exact filter\n - How many named trials actually use \u0027LLM-based Chatbot\u0027 as the intervention label\n - Recruiting-status transitions expected between now and Aug 2026\n\nKey uncertainties:\n - Whether narrow filter matches only a handful of trials\n - Whether NCT07280650 flips to Recruiting by Aug 1, 2026\n - Net effect of new registrations vs completed/closed trials\n - Sensitivity of count to exact intervention wording",
"forecast_prompt": "You are an elite superforecaster. Estimate the probability of each option for this Metaculus multiple-choice question.\n\n## Question\nHow many clinicaltrials.gov studies with an LLM-based Chatbot as the intervention/treatment will be recruiting on August 1, 2026 ?\n\n## Description / Resolution Criteria\n## Description\nIEEE Spectrum: [Can AI Chatbots Reason Like Doctors?](https://spectrum.ieee.org/ai-clinical-decision-support)\n\n`{\"format\": \"bot_tournament_question\", \"info\": {\"hash_id\": \"b3031f1f43b0491c\", \"sheet_id\": \"88\"}}`\n\n## Resolution Criteria\nThis question resolves as the number of studies with an Intervention/treatment of *LLM-based Chatbot* that have a study status of Recruiting are listed on August 1, 2026 at this link: https://clinicaltrials.gov/search?viewType=Card\u0026intr=LLM-based%20Chatbot\u0026aggFilters=status:rec\n\n## Options\n - 0 or 1\n - 2\n - 3\n - 4\n - \u003e4\n\n## Sub-question decomposition (planner)\n- (w=0.40) Is the current number of recruiting studies matching the \u0027LLM-based Chatbot\u0027 intervention filter on clinicaltrials.gov in the low single digits (roughly 0-3)? \u2014 The exact search filter is very narrow; establishing the current baseline count is the single most important anchor for \n- (w=0.35) Will the count of recruiting LLM-chatbot trials remain roughly stable (not grow by several new studies) by August 1, 2026? \u2014 Recruiting status is transient (studies complete or start), so net change over ~9 months determines whether the count mo\n- (w=0.25) Is there strong momentum in registering LLM/chatbot clinical trials that could push the count meaningfully higher by mid-2026? \u2014 LLMs in medicine is a fast-growing area; a surge in new registrations could raise the count.\n\n## Synthesized evidence\n1. [sq1 | web_search | MODERATE cred 55 | NEUTRAL | RECENT] Named LLM-chatbot trials found include NCT06346496, NCT06801925, NCT07132125, but most cited trials are completed or online-only, not currently recruiting.\n2. [sq1 | web_search | MODERATE cred 50 | DOWN | RECENT] The exact clinicaltrials.gov filter \u0027LLM-based Chatbot\u0027 is a narrow term; several relevant trials use varied intervention naming and may not match this precise filter.\n3. [sq1 | web_search | WEAK cred 40 | NEUTRAL | VERY_RECENT] No search result reported an exact current count of recruiting studies matching the specific filter link.\n4. [sq2 | web_search | MODERATE cred 60 | UP | RECENT] NCT07280650 (University of Michigan) is \u0027NOT YET RECRUITING\u0027 with estimated start August 2026, potentially entering recruiting status near resolution date.\n5. [sq2 | web_search | WEAK cred 45 | DOWN | RECENT] Some referenced trials had ethics approval or study windows ending in early 2026, suggesting they may no longer be recruiting by August 2026.\n6. [sq3 | article_search | WEAK cred 30 | NEUTRAL | VERY_RECENT] Article search returned no relevant articles about clinical trial registration momentum for LLM chatbots; results were off-topic.\n7. [sq3 | web_search | MODERATE cred 55 | UP | RECENT] Multiple new LLM-chatbot health trials were registered across 2025 (depression, HPV vaccine, caregivers), indicating an active but modest registration trend.\n8. [sq3 | claude_news | WEAK cred 40 | NEUTRAL | RECENT] General news reflects surging interest and scrutiny of medical/mental-health AI chatbots but provides no specific clinical-trial registration counts.\n\n## Cross-Market Signals\n\n### No signal found\n\nInformation gaps:\n - Exact current count on the specific filter link (unretrieved)\n - Historical growth rate of trials matching this exact filter\n - How many named trials actually use \u0027LLM-based Chatbot\u0027 as the intervention label\n - Recruiting-status transitions expected between now and Aug 2026\n\nKey uncertainties:\n - Whether narrow filter matches only a handful of trials\n - Whether NCT07280650 flips to Recruiting by Aug 1, 2026\n - Net effect of new registrations vs completed/closed trials\n - Sensitivity of count to exact intervention wording\n\n## Required pre-forecast walkthrough\n\nBefore giving probabilities, address these explicitly in your rationale:\n (a) The time left until the question resolves.\n (b) The status quo outcome \u2014 the option most likely if nothing changes from today.\n (c) An outcome that would be SURPRISING but plausible.\n\n## Calibration guidance\n\n- **Status quo anchoring.** Good forecasters put extra weight on the status quo option because the world changes slowly most of the time.\n- **Leave moderate probability on most options.** Good forecasters don\u0027t drop options to ~0% \u2014 leave \u22653-5% on every option to account for unexpected outcomes, unless you have strong specific evidence ruling them out.\n- Avoid uniform-distribution laziness \u2014 most options are not equally likely.\n- **Anchor on prediction markets.** If liquid market prices (Polymarket / Kalshi) for these options appear in the evidence, treat them as a strong prior and stay close to them absent specific contrary evidence.\n\n## Output\n\nReturn ONLY valid JSON, no markdown fences:\n\n{\n \"rationale\": \"\u003caddress (a) (b) (c) above, then explain your distribution \u2014 4-7 sentences total\u003e\",\n \"probabilities\": {\n \"0 or 1\": \u003cprob\u003e,\n \"2\": \u003cprob\u003e,\n \"3\": \u003cprob\u003e,\n \"4\": \u003cprob\u003e,\n \"\u003e4\": \u003cprob\u003e\n }\n}\n\nProbabilities MUST sum to ~1.0 (renormalization is applied; close enough is fine).\n",
"models": [
"opus"
],
"probabilities": {
"0 or 1": 0.22,
"2": 0.24,
"3": 0.21,
"4": 0.13,
"\u003e4": 0.2
},
"rationale": "(a) About 9 months remain until August 1, 2026. (b) The status quo is a low single-digit count because the exact \u0027LLM-based Chatbot\u0027 intervention filter is very narrow and many relevant trials use different naming or are completed/online-only; the most likely outcomes cluster around 2-3 recruiting studies. (c) A surprising but plausible outcome is \u003e4, driven by the strong momentum in registering LLM/chatbot medical trials during 2025 (depression, HPV, caregivers) plus trials like NCT07280650 flipping to recruiting near the resolution date. Recruiting status is transient, so some current trials may complete while new ones enter, keeping the net count modest. Given uncertainty about the exact baseline and offsetting effects of new registrations vs. completions, I spread probability across the low options with meaningful weight on higher counts to reflect active growth.",
"top_option": "2"
},
"plan": {
"combination_logic": "weighted_average",
"domain": "health",
"n_sub_qs": 3,
"n_tools": 3,
"reasoning_approach": "Anchor on the current exact-filter count (sq1), then adjust for expected net change and growth momentum (sq2, sq3) via a weighted average, mapping the blended estimate onto the multiple-choice count buckets.",
"sub_questions": [
{
"id": "sq1",
"question": "Is the current number of recruiting studies matching the \u0027LLM-based Chatbot\u0027 intervention filter on clinicaltrials.gov in the low single digits (roughly 0-3)?",
"rationale": "The exact search filter is very narrow; establishing the current baseline count is the single most important anchor for the answer.",
"weight": 0.4
},
{
"id": "sq2",
"question": "Will the count of recruiting LLM-chatbot trials remain roughly stable (not grow by several new studies) by August 1, 2026?",
"rationale": "Recruiting status is transient (studies complete or start), so net change over ~9 months determines whether the count moves off baseline.",
"weight": 0.35
},
{
"id": "sq3",
"question": "Is there strong momentum in registering LLM/chatbot clinical trials that could push the count meaningfully higher by mid-2026?",
"rationale": "LLMs in medicine is a fast-growing area; a surge in new registrations could raise the count.",
"weight": 0.25
}
],
"tool_requests": [
{
"parameters": {
"queries": [
"clinicaltrials.gov LLM-based Chatbot intervention recruiting studies",
"clinicaltrials.gov search intr=LLM-based Chatbot status recruiting",
"number of LLM chatbot clinical trials 2025 recruiting"
]
},
"target_sub_questions": [
"sq1",
"sq2"
],
"tool_name": "web_search"
},
{
"parameters": {
"brief": "Determine how many clinicaltrials.gov studies currently list an intervention/treatment matching \u0027LLM-based Chatbot\u0027 and have a Recruiting status, and assess the trend/growth rate of LLM chatbot clinical trial registrations through 2025-2026.",
"max_searches": 4,
"question_title": "How many clinicaltrials.gov studies with an LLM-based Chatbot as the intervention will be recruiting on August 1, 2026?"
},
"target_sub_questions": [
"sq1",
"sq3"
],
"tool_name": "claude_news"
},
{
"parameters": {
"lookback_days": 180,
"queries": [
"LLM chatbot clinical trial recruiting patients",
"generative AI chatbot randomized trial healthcare 2025",
"large language model medical chatbot study enrollment"
]
},
"target_sub_questions": [
"sq3"
],
"tool_name": "article_search"
}
]
},
"question": {
"close_time": "2026-07-06T03:00:00Z",
"description": "## Description\nIEEE Spectrum: [Can AI Chatbots Reason Like Doctors?](https://spectrum.ieee.org/ai-clinical-decision-support)\n\n`{\"format\": \"bot_tournament_question\", \"info\": {\"hash_id\": \"b3031f1f43b0491c\", \"sheet_id\": \"88\"}}`\n\n## Resolution Criteria\nThis question resolves as the number of studies with an Intervention/treatment of *LLM-based Chatbot* that have a study status of Recruiting are listed on August 1, 2026 at this link: https://clinicaltrials.gov/search?viewType=Card\u0026intr=LLM-based%20Chatbot\u0026aggFilters=status:rec",
"options": [
"0 or 1",
"2",
"3",
"4",
"\u003e4"
],
"question_type": "multiple_choice",
"title": "How many clinicaltrials.gov studies with an LLM-based Chatbot as the intervention/treatment will be recruiting on August 1, 2026 ?"
},
"research": {
"cross_market_brief": "## Cross-Market Signals\n\n### No signal found",
"errors": [],
"has_cross_market": true,
"n_errors": 0,
"n_tools": 3,
"tools": [
{
"elapsed_s": 58.31,
"error": null,
"success": true,
"summary": "0 results",
"tool_name": "web_search"
},
{
"elapsed_s": 42.39,
"error": null,
"success": true,
"summary": "9 citations",
"tool_name": "claude_news"
},
{
"elapsed_s": 1.06,
"error": null,
"success": true,
"summary": "25 articles",
"tool_name": "article_search"
}
]
},
"synthesis": {
"evidence": [
{
"claim": "Named LLM-chatbot trials found include NCT06346496, NCT06801925, NCT07132125, but most cited trials are completed or online-only, not currently recruiting.",
"credibility": 55,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq1"
},
{
"claim": "The exact clinicaltrials.gov filter \u0027LLM-based Chatbot\u0027 is a narrow term; several relevant trials use varied intervention naming and may not match this precise filter.",
"credibility": 50,
"direction": "DOWN",
"priced_in": false,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq1"
},
{
"claim": "No search result reported an exact current count of recruiting studies matching the specific filter link.",
"credibility": 40,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq1"
},
{
"claim": "NCT07280650 (University of Michigan) is \u0027NOT YET RECRUITING\u0027 with estimated start August 2026, potentially entering recruiting status near resolution date.",
"credibility": 60,
"direction": "UP",
"priced_in": false,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq2"
},
{
"claim": "Some referenced trials had ethics approval or study windows ending in early 2026, suggesting they may no longer be recruiting by August 2026.",
"credibility": 45,
"direction": "DOWN",
"priced_in": false,
"recency": "RECENT",
"source": "web_search",
"strength": "WEAK",
"sub_question_id": "sq2"
},
{
"claim": "Article search returned no relevant articles about clinical trial registration momentum for LLM chatbots; results were off-topic.",
"credibility": 30,
"direction": "NEUTRAL",
"priced_in": false,
"recency": "VERY_RECENT",
"source": "article_search",
"strength": "WEAK",
"sub_question_id": "sq3"
},
{
"claim": "Multiple new LLM-chatbot health trials were registered across 2025 (depression, HPV vaccine, caregivers), indicating an active but modest registration trend.",
"credibility": 55,
"direction": "UP",
"priced_in": false,
"recency": "RECENT",
"source": "web_search",
"strength": "MODERATE",
"sub_question_id": "sq3"
},
{
"claim": "General news reflects surging interest and scrutiny of medical/mental-health AI chatbots but provides no specific clinical-trial registration counts.",
"credibility": 40,
"direction": "NEUTRAL",
"priced_in": true,
"recency": "RECENT",
"source": "claude_news",
"strength": "WEAK",
"sub_question_id": "sq3"
}
],
"information_gaps": [
"Exact current count on the specific filter link (unretrieved)",
"Historical growth rate of trials matching this exact filter",
"How many named trials actually use \u0027LLM-based Chatbot\u0027 as the intervention label",
"Recruiting-status transitions expected between now and Aug 2026"
],
"key_uncertainties": [
"Whether narrow filter matches only a handful of trials",
"Whether NCT07280650 flips to Recruiting by Aug 1, 2026",
"Net effect of new registrations vs completed/closed trials",
"Sensitivity of count to exact intervention wording"
],
"n_evidence": 8
},
"timings": {
"forecast": 14.15,
"plan": 12.21,
"research": 58.31,
"synthesis": 16.76
}
}