Backtesting
Backtesting evaluates a specified strategy on historical data. Useful results require an explicit information timeline, reliable data, realistic execution and costs, and protection against look-ahead bias, selection bias and overfitting. Separate development from evaluation on unseen data; repeatedly selecting rules after inspecting a holdout compromises it. A good historical curve does not establish future profitability, and a weaker unseen result is a warning to investigate rather than conclusive proof of one cause. The number of bars is not the number of trades.
Example. A hypothetical strategy shows gross expectancy of +0.40R per trade in a test covering 500 bars; the number of bars is not the number of completed trades. If the stop distance is 25 pips, a model with one pip of spread and one pip of adverse execution at both entry and exit totals three pips, or 0.12R. Under equal-size, equal-cost assumptions, the result becomes +0.28R after those modelled costs, before any additional commission or financing. Do not deduct a spread again when actual bid/ask fill prices already include it. If half of the profit comes from just three unusually favourable events, performance on unseen data may differ substantially. The example illustrates sensitivity to assumptions, not a universal percentage by which every backtest overstates results. A weak out-of-sample result may reflect overfitting, a regime change, an implementation error or small-sample uncertainty; investigate rather than assume one cause.
A reproducible specification
Record instrument, timeframe, price source, rule version and the availability time of each input. A close-based signal does not justify an earlier fill on that bar. When OHLC cannot resolve stop/target ordering, use a documented convention or finer data instead of selecting the favourable result.
Robustness beyond one optimum
Compare neighbouring parameters, windows, costs and removal of individual extreme trades. Selecting the best variant after inspecting a holdout compromises its independence. Keep a log of all tested variants so the report does not conceal selection.
A decision-ready report
Report completed trades, period, net expectancy, dispersion, drawdowns and open positions. Distinguish balance from equity curves. Archived data, code and configuration help separate subsequent model changes from replication errors.
FAQ
How does a backtest differ from a forward test?
A backtest uses historical data; a forward test uses data arriving after the rules have been fixed. Forward testing can use a simulation or a live account. Demo execution is not real broker execution, while live testing introduces actual costs and financial risk. A demo can test the logic and workflow. A separate small-volume live stage also exposes the trader to actual account conditions and the emotions associated with real money, but it does not establish execution quality at a much larger position size.
Which costs should a backtest include?
Model the relevant spread, commission, execution differences and, where applicable, overnight financing and other charges. Use realistic quote sides and timing. Do not deduct the same spread twice if the execution-price model already includes it. Specify whether the model represents an order’s trigger price or its fill price, and how it handles gaps and insufficient liquidity.
What is overfitting and how can it be investigated?
It is fitting rules or parameters to incidental features of the development sample rather than a repeatable relationship. Use a protected holdout, walk-forward evaluation and sensible robustness checks. A weaker unseen result is a warning, but not by itself a definitive diagnosis. These checks require that evaluation data are not reused for optimisation. Regime changes, a small sample and implementation errors can also weaken the unseen result.
How much historical data is enough?
Enough to assess the specific rule’s uncertainty and relevant market conditions. Dependence, rare outcomes and the number of tested variants matter as well as trade count. A universal minimum of 100 trades per window is not a guarantee, and a test in one regime does not establish behaviour in another. Report the number of completed trades separately from bar count and quantify estimation uncertainty.
