Rust coreApache Arrow

A Rust core that answers in milliseconds

A million bars, one core, the full metric suite: 22 ms. That number is not a kernel timing, it is the whole call, and it holds because the engine is built the same way all the way down. Python describes the strategy, Rust runs it over Arrow columns, and nothing in the middle gets to slow the loop down.

>>>res = mbt.run(strategy, config, store)

One call, one crossing into native code, results back as Arrow.

22 ms
one backtest over a million bars, single core, metrics included
~20 ns
per bar, and it does not drift from a thousand bars to a million
+0.5%
what computing the full metric suite adds to a run
8.2 µs
per backtest inside a 256-combination grid, and 80 µs at 100k bars

Three decisions that set the speed

None of them is “written in Rust”. Rust is what makes the decisions payable; the decisions are about where the boundary sits, what shape the data has, and what the inner loop is allowed to do.

import manifoldbt as mbt
from manifoldbt.indicators import close, sma

fast, slow = sma(close, 10), sma(close, 50)
strategy = (
    mbt.Strategy.create("sma_cross")
    .signal("fast", fast)
    .signal("slow", slow)
    .size(mbt.when(fast > slow, 1.0, 0.0))
)

# The expression graph crosses into Rust once. Python is not in the loop,
# and the full metric suite comes back with the run.
res = mbt.run(strategy, config, store)
res.metrics["sharpe"]

Python declares, Rust executes

architecture

A strategy is not a callback. It compiles to an expression graph that crosses the boundary once and is evaluated vectorized over the whole series in Rust, then a single sequential pass handles fills, fees and equity. There is no interpreter in the hot loop, and no GIL in the way of a sweep.

Columnar all the way down

data

Apache Arrow in memory, Parquet on disk, and a zero-copy handoff back to polars or numpy. Loading is parallel as well, which matters more than it sounds: on a short backtest the data path, not the simulation, is most of the wall time.

Specialised paths, pinned to the general one

correctness

The shapes research actually runs get their own path through the engine rather than the generic one, which is where most of the speed comes from. Every one of them is asserted against the general path bit for bit, so a fast path can never quietly become a different answer.

One backtest, as the series grows

The useful property is not the headline number, it is the shape. A straight line means the cost is the data and nothing else: no size at which the engine changes behaviour, and no cliff where a longer series suddenly costs more per bar than a shorter one.

mbt.run(strategy, config, store), one core, full metric suite
05ms10ms15ms20ms0250k500k750k1Mbars in the series1.96ms10.7ms22.2ms~20 ns per bar, flat across three orders of magnitude
One SMA crossover backtest over a growing series, on a single core, with the whole metric suite computed. The line is straight because the cost is the bars: about 20 nanoseconds each, from a thousand of them to a million. The two smallest series, a thousand and ten thousand bars, come in at 0.04 ms and 0.20 ms, too close to the origin to separate here, which is rather the point. The figure includes reading the bars out of the Arrow store, so it is the end-to-end call rather than a kernel timing.

Two costs are visible in that line. A fixed one of a few tens of microseconds per call, which is the store read, the config preparation and the result construction, and a marginal one of about 20 nanoseconds per bar. On a year of minute bars the fixed part has already disappeared into the noise.

A backtest this cheap changes what you do with it. Ten milliseconds is inside the interaction loop: you can re-run on every edit, compare a dozen variants in a notebook cell, and never build the habit of guessing which parameter is worth testing.

The batch is the unit, not the backtest

Research is never one backtest. It is a few hundred of them with one number changed, then a few thousand once the question gets serious. Calling a fast engine in a Python loop throws most of that speed away: the loop re-enters native code every time, and the interpreter serialises what should run on every core at once. Here is what it costs, with the grid and the series both growing at each step.

run_sweep_lite(strategy, grid, config, store, device="cpu")
256 combinations 10k bars020ms40ms60msone by one52msone call2.1ms25x5,041 combinations 100k bars03s6s9sone by one9.9sone call0.41s24x99,856 combinations 1M bars010min20min30minone by one37minone call4min 27s8.3x
Each row grows both axes at once: twenty times the grid and ten times the series. The reference bar prices the same grid one backtest at a time, at the single-run cost measured over the same series, and charges nothing for Python itself, so a real loop is slower still. Every figure here is the CPU path on all threads.

The ratio holds at 25x while the grid grows twenty times, then collapses to 8.3x once the series reaches a million bars. Batching removes overhead, and on short series overhead is most of the cost. It cannot remove arithmetic, and by a million bars the arithmetic is nearly all of it. That boundary is exactly where the GPU path starts to matter.

# The whole grid is one native call, not a Python loop around run():
# the bars are read once and the fan-out happens below the interpreter.
sweep = mbt.run_sweep_lite(strategy, grid, config, store)

# Read a metric back as a column, not as one object per combination.
sharpe = mbt.sweep_columns(sweep, "sharpe")
WorkloadPer backtestPer bar
256 x 10k bars8.2 µs0.82 ns
5,041 x 100k bars80 µs0.81 ns
99,856 x 1M bars2.67 ms2.67 ns
The same three runs, priced differently. A bar simulated inside a sweep costs 0.8 nanoseconds across all threads at ten and a hundred thousand bars, and three times that at a million, where the series stops being something the machine can keep close to the cores. That ratio, not the core count, is what the last row of the figure is made of.

None of it counts if the numbers move

Every number on this page comes from a build that has to produce the same results as the one before it, and that is a test rather than a promise. Golden fixtures pin the equity curve, the trade list and the metric suite bit for bit, so a change that makes a run faster and different fails exactly the way a wrong answer does.

Every run also writes a manifest: the strategy definition, the data hashes, the engine version and the whole configuration. Replaying a manifest reproduces the run exactly, whenever and wherever it is executed, which is what makes a fast research loop something you can build on rather than a source of numbers you cannot get back.

FAQ

Run it against your own data

The engine ships compiled in the wheel, so there is nothing to build and no Rust to learn. Point it at a CSV, a DataFrame or an exchange connector and time it yourself.