Every scan, reasoning trace, and price follow-up produced by this agent is viewable on this site. The underlying dataset is shared for non-commercial research only; bulk export is gated behind an access password — contact the authors to request access. This page also links to the chart set we use in our ICML workshop paper. Charts are regenerated from the live database, so they will shift as more scans accumulate.

Pull the data

The full source code, including the pipeline, analysis scripts, and figure generators, lives on GitHub:

github.com/wyw2010/forecasting-agent →

All data endpoints return JSON. The primary endpoint is a single-shot dump of every scan, trace, trade, price follow-up, and P&L snapshot:

curl -H "Authorization: Bearer ACCESS_PASSWORD" https://forecast.agenticlearning.ai/api/export > export.json

Or export straight from the browser — enter the access password and download the full JSON dump:

To reproduce the analysis in the paper:

1. git clone https://github.com/wyw2010/forecasting-agent && cd forecasting-agent
2. curl -H "Authorization: Bearer ACCESS_PASSWORD" https://forecast.agenticlearning.ai/api/export > data/export_latest.json pulls the latest export (request the access password from the authors).
3. python analysis/extract_factors.py --input data/export_latest.json --output analysis/output/factors.csv extracts per-scan factors from the reasoning traces.
4. python analysis/compute_directional_accuracy.py --factors analysis/output/factors.csv --data data/export_latest.json --output analysis/output/factors_with_da.csv joins the factors with price follow-ups.
5. python analysis/generate_figures.py --out-dir out/ --min-date 2026-04-07 --data data/export_latest.json produces the headline charts below.
6. python analysis/generate_ablation_figures.py --out-dir out/ --min-date 2026-04-07 --horizon 3 produces the pipeline and tool ablations.

Headline charts

Four charts carry the paper's main story. They answer, in order: does our agent beat a simple baseline? · which stage of the pipeline did the work? · which research tool did the work? · does the critic stage actually help, or just hug the market?

1. Multi-step vs. zero-shot, by day. The y-axis is the information coefficient (IC), the correlation between each forecaster's edge (predicted minus market) and the subsequent market movement. IC is magnitude-aware (being right by 20pp counts more than being right by 2pp) and is the field-standard skill measure in quantitative forecasting. Two baselines are drawn for context: the dotted line at IC = 0 is "market-as-forecast" (a predictor that parrots the current market price has edge identically zero by construction), and the dashed gray line is mean-reversion (always predict 0.5).

Multi-step averages IC +0.17 across t+1 through t+14 vs. +0.02 for zero-shot and +0.04 for mean-reversion. The multi-step pipeline beats both baselines at every horizon, and the gap widens at longer horizons.

1b. The same chart, extended to t+30. Same forecasters and null lines as Chart 1, just a longer x-axis. Multi-step IC keeps climbing past t+14 and reaches roughly +0.62 at the very tail; zero-shot stays flat-to-negative at −0.05 to −0.10. Read the right edge with care — sample sizes drop into the double-then-single digits past t+25 (visible in the row beneath the x-axis), and the shaded 95% CI bands fan out accordingly. The interpretable region is roughly t+1 through t+22 (n ≥ 80); past that, the trajectory is suggestive, not conclusive. Mean IC across t+1 through t+30: multi-step +0.25, zero-shot −0.03, mean-reversion +0.01.

2. Per-stage IC. One line per stage of the multi-step pipeline, measured on the same scans (so this is a within-question ablation, not a separate run). The post-critic final stage (black) sits clearly above every other line at every horizon. The three ensemble perspectives (purple) and their simple average (blue) cluster at IC ≈ +0.07 to +0.11, close to the zero-shot baseline (gray). Almost all of the multi-step lift over zero-shot is contributed by the last stage.

3. Per-tool IC lift. For each of the 10 research tools, IC of scans where the planner invoked it minus IC of scans where it did not. Tools used on every scan or never used (or with fewer than 15 scans on either side) are tagged "no data." Of the contestable tools, only FRED macroeconomic data shows a consistent positive lift across horizons (+0.30 to +0.40). The other tools are noisy or cluster near zero.

4. Critic anchoring. Mean |forecast minus market price| at three stages of the pipeline. The independent ensemble (which never sees the market price) preserves a 0.16 edge. After the critic stage reveals the price, the final forecast collapses the gap to 0.10, a ~39% compression toward the market. The critic is instructed to challenge the ensemble's reasoning; in practice, given a strong numerical anchor, it pulls the forecast toward the market instead.

Supporting evidence

Diagnostic panels that back up the headline story: the ensemble's diversity collapse, the cumulative paper-trading P&L (an economic-value sanity check on the IC lift), the relationship between forecast magnitude and subsequent movement, and a regression forest plot of every non-tool factor.

Ensemble diversity collapse. Histogram of (max − min) across the three perspective forecasts on each scan. 39% of scans are degenerate (spread < 0.02): three structurally distinct prompts converge on near-identical forecasts. This is the mechanism behind the ensemble-average step's flat IC contribution above.

Paper-trading P&L. Cumulative return on the cohort, sized by the agent's final forecast against the prevailing market price. Positive overall, consistent with the IC findings, and noisy as expected on prediction markets that are close to efficient. All trades are simulated, no real capital deployed.

Edge vs. movement. Independent ensemble edge versus subsequent price movement, panelled by horizon (t+1, t+3, t+7). Correlation grows with the forecast horizon: short-term prices are noisy, longer-term movements align more with the pre-critic ensemble edge.

Factor regression forest plot. Bivariate association with directional accuracy at t+1 for every non-tool reasoning-trace factor, grouped by category (evidence quality, decomposition, ensemble, critic, market/meta). Each row is the OLS slope of DA on a standardised factor with 95% bootstrap CIs.

Directional accuracy companions

Directional accuracy (DA) asks the simpler binary question: did the market move in the direction we predicted? The IC findings hold qualitatively under DA, but with markedly smaller effect sizes, consistent with DA discarding the magnitude information IC retains. We keep DA charts here for comparison with prior prediction-market work; IC is the primary headline.

DA by horizon. Binary directional accuracy for zero-shot and multi-step at each horizon. Multi-step tracks at or above zero-shot at every horizon, but the gap is compressed compared to IC because DA throws away the magnitude information IC retains.

Per-stage DA. Same structure as the per-stage IC chart in the headline section but using binary directional accuracy. The six stages cluster more tightly under DA than under IC; the IC view reveals the post-critic stage's sharper signal that DA masks.

Per-tool DA lift. DA companion to the per-tool IC chart. FRED again leads at most horizons, confirming the IC-based ranking is robust to the choice of metric.

Charts were last regenerated from the figures_apr7plus cohort (259 V2 scans, archived excluded). The analysis scripts and chart templates are in analysis/ on GitHub; rerun them against a fresh /api/export pull to reproduce.