Backtest a trading strategy with Claude Code

The six steps

1. 01

   Install Manifold-BT and get data

   pip install manifoldbt, then one mbt.ingest() call downloads bars from a free connector into a local store. The data never leaves your machine.

2. 02

   Connect the reference

   Register https://mcp.manifoldbt.com/docs/mcp as an MCP server in Claude Code. It serves the documentation, signatures and engine rules; it runs nothing.

3. 03

   Ask for the strategy

   State the rule, the market, the period, the costs, the signal delay, and the four numbers you want back: return, Sharpe, max drawdown, fill count.

4. 04

   Let the agent read before it writes

   With the server connected the agent calls list_topics first and receives a runnable script plus the rules the engine enforces, then writes yours from that.

5. 05

   Run it locally and read the four numbers

   The script runs on your machine on the Rust core. Metrics are fractions, and the fill count counts fills, not round trips.

6. 06

   Feed errors back, then make it honest

   An engine error goes to explain_error and comes back as a cause and a fix. Then keep signal_delay=1 and fees on both sides, and only then ask for a sweep.

What goes wrong without the reference, measured

The bench gives a headless Claude Code one backtesting task, the data already in place, and one of three toolings: pandas in a bare environment, Manifold-BT installed with no documentation, or Manifold-BT plus this reference over MCP. It has to hand back four numbers, and a run counts as right only when all four match the engine within a tolerance. Eight tasks, three runs each, nothing changing between the arms except the line that says what the agent has to work with.

Two things in that chart are not obvious from the totals. First, the middle column mostly does not measure the engine: given the library and no reference, the smaller models do not use it. Sonnet wrote Manifold-BT code in 1 run out of 30 and pandas in the other 29; Haiku, 2 out of 30. An agent does not use a library it was handed, unless it can read how. Opus imports it every time and saturates the bench either way; what the reference buys it is cost, about 25% less for the same 24 correct runs.

Second, the tasks that separate the arms are not the hard calculations. They are conventions: which side of a bracket wins when a bar touches both, where a gap through the stop fills, how a period counts on a higher timeframe. The Donchian breakout with a stop and a target went 0 of 3 without the reference and 3 of 3 with it, on Haiku and on Sonnet, because those are engine semantics that no signature reveals and nobody guesses. The chart and the per-model cost live on the MCP page.

Steps 1 and 2, install and connect

pip install manifoldbt brings the Rust core in the wheel. Data comes from mbt.ingest(), one call against a free connector (Binance, Bybit, Hyperliquid, dYdX, Deribit, Dukascopy, Yahoo Finance) or your own CSV or DataFrame, into a store on your disk. Then one line registers the reference in Claude Code:

claude mcp add --transport http manifoldbt-docs https://mcp.manifoldbt.com/docs/mcp

That is a hosted HTTP endpoint with four tools: list_topics hands back a complete runnable script and the list of rules the engine enforces, search_docs answers a question about the API or the execution model, explain_error turns an engine message into a cause and a fix, and get_capabilities says what the Community tier refuses before the agent writes code around it. Nothing executes there.

Step 3, the prompt

Say the rule, the market, the period, the costs, the delay, and exactly what you want back. The costs and the delay are not decoration: an agent left to choose will pick zero fees and a fill on the same close it decided on, and the result will be a number about a market that does not exist.

Backtest a 20/50 EMA crossover on BTCUSDT, 1-hour bars, 2022-01-01 to 2025-01-01.
Long when the fast EMA is above the slow one, flat otherwise, no shorts.
5 bps taker fee on every fill, 2 bps fixed slippage, signals fill on the next bar.
Use manifoldbt. Report total return, Sharpe, max drawdown and the number of fills.

The bench found one more thing about prompts, the hard way. A negation in a task statement (“the trailing stop does NOT start at the high”) sent the documented arm from 1 of 6 to last place; rewriting the same rule as a positive sentence with one worked number took it to 6 of 6, with the reference unchanged. Say what the rule does, not what it does not do.

Steps 4 and 5, what the agent writes, and what comes back

With the server connected, the first thing a well-behaved agent does is call list_topics. It receives a script that already runs, the three variants that trip people up (stateful bands, bracket exits, higher-timeframe filters), and the rules as one-line statements. It then writes yours in that shape:

import manifoldbt as mbt
from manifoldbt.expr import col, lit, when
from manifoldbt.indicators import ema
from manifoldbt.helpers import Interval, Slippage, time_range

# What the agent writes once it has read the reference: the same shape as
# the script list_topics hands it, with your rule in place of the example.
store = mbt.ingest(provider="binance", symbol="BTCUSDT", symbol_id=1,
                   interval="1h", start="2022-01-01T00:00:00Z",
                   end="2025-01-01T00:00:00Z")

fast, slow = ema(col("close"), 20), ema(col("close"), 50)
pos = when(fast > slow, lit(1.0), lit(0.0))      # fraction of equity per bar
strategy = (mbt.Strategy.create("ema-cross")
            .signal("pos", pos)
            .size(col("pos")))

start, end = time_range("2022-01-01", "2025-01-01")   # both bounds, always
config = mbt.BacktestConfig(
    universe={"binance": ["BTCUSDT"]},
    time_range_start=start, time_range_end=end,
    bar_interval=Interval.hours(1),
    initial_capital=100_000.0,
    warmup_bars=50,                                   # >= the longest lookback
    execution=mbt.ExecutionConfig(signal_delay=1),    # bar t decides, t+1 fills
    fees=mbt.FeeConfig(taker_fee_bps=5.0, maker_fee_bps=5.0),
    slippage=Slippage.fixed_bps(2),
)
result = mbt.run(strategy, config, store)

m = result.metrics                                    # fractions, not percents
print(m["total_return"] * 100, m["sharpe"], m["max_drawdown"] * 100)
print(m["trade_stats"]["total_trades"],               # fills: entry + exit = 2
      m["trade_stats"]["round_trips"])                # completed trades

The four numbers come out of result.metrics, and three details decide whether you read them right. Returns and drawdown are fractions, not percents. Sharpe is computed on daily equity returns with the sample standard deviation, annualised by 365.25 days for crypto (set trading_days_per_year=252 for equities). And total_trades counts fills, so an entry and its exit are two; round_trips counts completed trades. The bench’s one unfixable failure lived in that last number: a multi-asset run that returned 42,570 fills instead of 1,413 because the agent rebalanced every bar, with all three performance metrics within 2% of the reference. The fill count is the one that tells you the strategy is not the one you asked for.

Step 6, when it fails, and then making it honest

The engine’s errors state a rule, not the mistake that broke it, so an agent reading one tends to change something adjacent and try again. Paste the message into explain_error and it comes back as the cause and the correction. The two that come up most: conditions combined with the words and / or instead of & / |, and “empty bar dataset for symbol” on a store reopened without its arrow_dir, which answers list_symbols() and then finds no bars.

Once it runs, keep the two settings that make it a backtest rather than a story: signal_delay=1, so bar t decides and bar t+1 fills, and fees on both sides, because a reversal is one fill whose quantity is both legs added together and it pays for both. Then ask the agent for the next question, not the next strategy: a parameter sweep over the two EMA lengths, and a walk-forward over the period. The sweep is one call across every core; on the free Community tier it is capped at 256 combinations per session, and walk-forward is Pro, $19 a month or $290 once. The agent will read that from get_capabilities before writing code the engine would refuse.

Cursor, Codex, and any other MCP client

The endpoint is plain HTTP MCP, so the workflow is the same anywhere. In Cursor, VS Code or Windsurf, the server goes in the client’s MCP configuration:

{
  "mcpServers": {
    "manifoldbt-docs": {
      "url": "https://mcp.manifoldbt.com/docs/mcp"
    }
  }
}

Then ask for the strategy as you normally would. The agent consults the reference on its own; you do not have to tell it to.

Frequently asked questions