Backtesting and forward testing — methods, evidence and pitfalls

Testing a strategy means checking how its rules behave on data and how much the result depends on the assumptions. A backtest uses history; a prospective forward test records decisions as new data arrive. Both need a clear specification, realistic cost accounting and an honest description of the sample. This lesson provides a twelve-field test card, chronological validation examples, executable calculation tools and a procedure for comparing historical simulations with later observations. The aim is to distinguish what the evidence supports from what remains uncertain—not to turn a favourable chart into a promise of future returns.

Original illustrative artwork retained. Demo uses simulated fills; actual live execution is assessed on the micro account. The written method below explains the assumptions and limitations of the pictured progression.

Record the hypothesis, rules, sample and results in Toolkit N: the strategy testing protocol. The form complements the method in this chapter with local saving, export and print.

1. Backtesting and forward testing — different questions, different answers

A good backtest does not mean that a strategy is ready to trade. Historical testing investigates the past under a specified model. Prospective testing observes decisions as new data arrive. The two stages provide different information: a backtest can supply a longer outcome sample, while forward observation can expose timing, implementation and adherence problems. Only an appropriately controlled live stage measures actual live fills and charges; demo execution remains a simulation. Using one stage as a substitute for the other discards information the missing stage was meant to provide.

Backtesting: the historical question

A backtest asks: “How would these rules have behaved over, for example, the preceding three to ten years?” Its answer is only as useful as the data, the execution and cost model, and the validity of the research design. It is retrospective:

• It can be fast. Five years of data may take minutes in a simple test, although detailed tick modelling and large parameter searches can take much longer.
• It can produce hundreds or thousands of observations, depending on signal frequency and the historical period.
• It permits comparisons between parameter variants. That flexibility is also a major source of selection bias and overfitting.
• It is sensitive to bad ticks, missing candles, incorrect timestamps and unrealistic costs. A large sample does not repair a systematic data or model error.

A backtest does not establish that a strategy works now. A regime represented in 2018–2022 need not persist through 2024–2026. Likewise, results from a particular historical yen trend cannot simply be transferred to a different monetary-policy or intervention environment. Such examples illustrate a change in conditions, not an assertion that a named strategy must succeed or fail in a particular period. Document the regime exposure rather than treating the date range as sufficient evidence.

Forward testing: the prospective question

A prospective test asks: “How do the frozen rules behave as new observations arrive, and does their implementation match the intended process?” It runs forward in real time:

• It takes time. At five trades per week, 100 trades require about 20 weeks, roughly five months, before allowing for holidays and skipped opportunities.
• The sample may remain small. A 30–80-trade pilot is an example of limited evidence, not a verified description of every retail trader's practice.
• Future observations are unavailable at the moment of a genuine live decision. That removes one route for look-ahead, but later relabelling, revised data or retrospective edits can still contaminate the recorded evaluation.
• It can expose spread changes, latency, rejected requests and slippage. The evidence depends on the mode: live records measure live execution, while demo and paper modes do not fully reproduce it.
• It reveals human implementation errors: wrong quantities, delayed decisions, missed signals and discretionary intervention in an open position.

A short forward test does not settle a long-term edge claim. It is particularly valuable for identifying a gap between historical assumptions and current implementation. The longer-run inference still depends on sample size, dependence, regime coverage and uncertainty, not on the label “forward”.

Three testing modes: manual review, replay and automation

“Backtesting” can refer to three different activities. Separate them before comparing their evidence:

• Manual chart review. You scroll through history and record decisions. It helps identify unclear rules and boundary cases. An ordinary review with later candles visible is vulnerable to hindsight, selective pacing and memory of the outcome; it should not be presented as an unbiased performance sample. A carefully designed blinded manual protocol is a different, more demanding exercise.
• Bar-by-bar replay. A tool reveals history sequentially while hiding later observations. Examples include chart replay, visual tester modes and dedicated replay applications, subject to their actual data and timing behaviour. Replay can test consistency of discretionary classification and response timing if the pacing is controlled. It still does not establish real execution costs. Rewinding after seeing the outcome and replacing the original decision contaminates that run; record an exploratory replay separately.
• Automated historical testing. Code applies rules to a dataset. This is an efficient route to large, reproducible samples, but it introduces all the hazards in section 2: information leakage, execution assumptions, costs and repeated selection. Automation reproduces a flawed rule very consistently too.

The sequence matters. Replay before coding can reveal instructions that two people interpret differently. A tester cannot supply the missing definition; it only implements whichever interpretation the programmer chose. Freeze the strategy specification and version before its evaluation, as described in article 15.3, not after inspecting the result.

What each stage can and cannot reveal

Historical work provides context and sample depth; prospective work checks whether the process survives contact with new data and the intended operating environment. A fast backtest and a 90-day observation stage impose very different waiting costs, but elapsed time is not a reason to omit the latter. Neither a ten-minute computation nor a three-month pilot guarantees that the necessary evidence has been collected.

Reconciling historical and prospective signals

A particularly useful diagnostic is a signal-by-signal comparison over the same dates. Export both sources into a common schema, for example:

timestamp_utc, broker_server_time, pair, setup_id, direction, entry_rule_hash,
entry_price, sl_price, tp_price, trigger_bar_close, source

Preserve enough additional fields to distinguish signal time, actual information availability, order requests and fills. After the first 30–60 days of observation, run the frozen historical implementation over the archived data from those same days. Compare the two inventories. An illustrative discrepancy threshold of 5–10% can trigger investigation, but smaller differences may matter if they systematically favour the backtest. Define the denominator and whether you are matching signals, requests or fills before reporting a percentage.

Potential causes include future-information leakage, repainting, different feeds, time-zone errors, inconsistent code and failed real-time order attempts. Fifty historical signals versus twelve prospective signals on the same dates is a substantial unresolved discrepancy; do not proceed until it is explained. Reading Close[0] intrabar is not automatically a bug: the error is treating its later final value as though it were known at the earlier decision.

The test card: twelve fields before interpreting the result

Without a test card, a result may be impossible to reproduce six months later. You need to know which data, implementation, cost model and selection history produced it. Complete the first nine fields before running the evaluation. The rest of this article explains why each field can change the conclusion.

Field 9 is easy to omit. A criterion written after seeing the result is a retrospective description, not a prespecified decision rule. Field 7 matters just as much: without the total trial count, the apparent winner may simply be the maximum from many searches. Field 11 is where unresolved limitations must be explicit. If an essential input in an earlier field is unknown, do not invent it; mark it unresolved and postpone the test or restrict its interpretation accordingly.

2. Testing traps — look-ahead, survivorship, curve fitting and overfitting

These are methodological hazards, not simply signs that a researcher is careless. A platform or custom script can produce a convincing report while its configuration or implementation admits a serious error. Cognitive issues such as confirmation bias and hindsight are discussed in the trading-psychology section. Here the focus is the technical design of the test and the checks needed to expose its weaknesses.

Trap 1: look-ahead bias

Look-ahead occurs when a decision uses information that would only have become available later. Common routes include:

• An intrabar decision based on the eventual close. A strategy calculates SMA(20) including the current M5 candle and says it enters when that candle closes above the average. At second 240 of the five-minute candle, the final close is still unknown. Reading the current Close[0] is not itself future leakage; interpreting a later final close as the value available at second 240 is. Verify what your actual tester mode and code return at each callback rather than assuming that a mode named “Open prices only” supports the intended intrabar rule.
• A historical marker confirmed later. ZigZag, swing-point tools, fractals, manually selected pitchfork anchors and some custom channels may mark an earlier bar using information arriving afterwards. A swing high drawn ten bars back today may not have been confirmed on that bar. The indicator's name does not establish the exact behaviour; the implementation and confirmation timestamp do.
• Macroeconomic data indexed by reference period rather than release time. A value labelled “March CPI” may only be published in April. A strategy using it before its actual release is using unavailable information. Verify the issuer's release calendar, the provider's timestamp convention and the vintage available then. Later revisions are another source of leakage.

Detection: observe the frozen strategy on incoming data, then reconcile its original signals with a historical reconstruction of the same dates. A disagreement is a problem to diagnose, not proof of look-ahead by itself. Other explanations include a different feed, repainting, incorrect time conversion and failed real-time order attempts. First align time and data; distinguish signals from requests and fills; then inspect the strategy's information flow. Rewriting a correct indicator before checking the input bars can create a new problem without explaining the old one.

Implementations that deserve an explicit information-timing check include:

• ZigZag, fractals and pitchfork constructions. Determine when a pivot or anchor becomes knowable and whether the displayed earlier point can move. A correctly delayed confirmed signal is different from pretending the pivot was known at its origin.
• Heikin-Ashi, SuperTrend, Hull moving averages and custom channels. None is inherently fraudulent. Test the actual implementation for changes to completed-bar signals. Heikin-Ashi's synthetic candle prices also need separate treatment: they are not automatically executable market prices.
• Closed-source marketplace indicators. You cannot inspect their internal logic. Under a policy requiring source-level validation, reject them for that research workflow. Black-box timing tests can reveal defects but cannot establish every internal dependency.
• iCustom() buffer reads. Identify the buffer, bar shift, unavailable-value handling and whether historical buffer values are recalculated.
• Intrabar Close[0] decisions in MT4. The current value is an evolving current-bar price, not the final closing value. The test must reproduce the same information sequence as the live rule.
• Higher-timeframe data on a lower-timeframe chart. An H4 value used by an M15 decision before that H4 bar closes is provisional. In Pine, the behaviour of request.security() depends on expression offsets, lookahead settings and historical versus real-time evaluation. Use the documented confirmed-higher-timeframe pattern where that is the intended rule; do not assume that adding [1] anywhere fixes every timing problem. Check the corresponding MQL implementation too.
• Macro series dated by period. Join by actual information availability and vintage, not by the first day of the month described by the statistic.

A practical anti-repainting check: on a live feed, record the indicator value for an identified completed bar currently ten bars back. After five new bars arrive, inspect the same timestamp, now fifteen bars back. Five new M1 bars take roughly five minutes; five H1 bars take roughly five hours, excluding session gaps. If the value changes, a historical display does not necessarily reproduce the original signal. Record when any confirmation became available.

No change on one bar is not a certificate of non-repainting. Repeat across multiple bars, boundary cases and volatility regimes; include changes after reconnection or history refresh where relevant. This is an observation protocol, not a claim that every implementation can be validated in a few minutes. Preserve the records so that the comparison is reproducible.

Trap 2: survivorship bias

Suppose a researcher uses a present-day list of currency pairs to test an earlier period. That selection can omit instruments that were tradable then but later disappeared or became unavailable. The converse implementation problem also matters: attractive historical results may depend heavily on an instrument the intended provider no longer offers under comparable conditions. USD/RUB around the changes following 2022 is an example of why availability, sanctions restrictions, liquidity and execution conditions need date- and provider-specific verification, not a universal assertion about today's access.

Marketplace selection creates a related problem. A currently visible EA with five stars and 200 positive reviews is drawn from the products still listed and noticed. Failed, withdrawn or unpopular products may be absent from the comparison. Ratings alone do not identify the full launch cohort, and they do not establish either the probability of survival or that every EA originally had the same chance of success.

Detection: reconstruct the instrument universe as it existed at the historical decision date. A test of 2018–2020 should identify what the relevant provider offered in 2018, including instruments later withdrawn. Restricting history to today's survivors uses future information at the universe-selection level. Current feasibility is a separate limitation: record which historical instruments remain available under the intended legal entity and how much of the historical result came from those no longer usable. For marketplace products, retain publication dates, version history and information about the comparison cohort rather than interpreting a current ranking as an unselected sample.

Trap 3: curve fitting

Consider SMA periods 20, 22, 25 and 30; RSI periods 11, 14 and 17; and an ATR threshold from 0.8 to 1.5 in 0.1 increments. That is 4 × 3 × 8 = 96 combinations. Searching many alternatives increases the chance of finding an attractive result even if no durable edge exists. It does not guarantee that one will look outstanding, but the selected maximum is not an ordinary single prespecified observation.

In a synthetic example, the best combination is SMA-22, RSI-14 and ATR 1.1, with +180% over three historical years. The next 60 days produce −8%. This contrast may be consistent with fitting historical noise, but the return pair alone does not prove that diagnosis; costs, implementation, regime and sampling must also be checked. What is certain is that the historical winner was selected from 96 trials, and its report must disclose that process.

Detection: inspect a parameter heatmap. An isolated favourable cell surrounded by poor results suggests sensitivity that deserves investigation. A broad plateau only indicates that the result is not tied to one setting. A plateau does not prove positive expectancy. It may still be an artefact of the same overused dataset, or simply reflect parameters whose small changes leave the trade set unchanged.

Independent evaluation is still required: a genuinely untouched OOS period, a prespecified walk-forward procedure and comparison with the baseline in section 3. Report trade counts, uncertainty and the full selection history, not just a smooth heatmap or a favourable mean. An OOS result consulted repeatedly while changing the method is no longer untouched.

Trap 4: overfitting the rules

Parameter curve fitting is one form of overfitting. Another is adding rules to remove particular historical losers. “No entries after 13:00 Europe/London on Tuesdays” may have been introduced solely because two losing trades happened in that interval; this corresponds to the Polish example's 14:00 cutoff under the usual UK–Poland offset. Removing those trades improves that sample by construction, not necessarily the underlying strategy.

Do not present a Friday filter as validated merely because three past losses occurred on Fridays. A filter proposed after inspecting losses is a new research hypothesis. It may be investigated on development data, but it needs a reason, a trial-log entry and independent evaluation before being called an improvement. Freeze it before the evaluation used to support the claim.

This example article offers a conservative working warning for more than four optimised parameters on fewer than 200 trades, and discusses the rough heuristic optimised parameters × 30 ≤ trade count: four parameters imply 120 trades, eight imply 240. These are illustrative audit heuristics, not statistical theorems or market standards. A model below the warning line can still be overfit, and one above it is not disproved solely by the ratio.

Investigate sensitivity, the number of attempted variants, dependence and independent performance. Where variants share observations, evaluate their performance differences with an appropriate paired or resampling method rather than assuming that overlapping separate intervals settle the comparison. A favourable OOS result or stable heatmap is evidence to assess, not a universal proof that overfitting has been removed.

Trap 5: under-modelled costs

An MT4 test configured with a fixed 1-pip spread applies a simplified cost regardless of the historical liquidity state. Verify the installed tester's settings and capabilities rather than assuming every platform or build uses the same default. Actual spreads can differ materially between liquid session overlaps, rollover and major releases. Fractions of a pip, several pips and brief double-digit-pip spreads are scenario magnitudes to investigate on the relevant feed, not current provider quotations or guaranteed bounds. A strategy concentrated in a thin-liquidity overnight window can look profitable under a fixed-spread assumption and fail under an appropriate variable-spread model.

The second component is slippage. An exact-price historical fill is an assumption, not a promise of live execution. The example illustrates 0.2–0.5 pip in ordinary conditions, 1–3 pips around breakouts and 5–15 pips after releases. Those ranges must not be treated as a measured distribution for your account. A gross +0.3R estimate could fall to +0.1R if additional implementation costs amount to 0.2R; the actual reduction depends on the initial risk distance, order type and measured fills.

The third component is financing. A swing strategy holding for three to eight days can incur rollover debits or credits according to the instrument, direction and provider. Do not assume that long EUR/USD always has a particular sign. In the illustrative calculation, a 3-pip nightly charge with a 20-pip initial stop distance is 0.15R per charged day. Five single-day charges total 0.75R, exceeding a +0.4R pre-financing estimate. A 60-pip stop makes the same charge 0.05R per day. Use the actual day multipliers, including relevant weekend and holiday adjustments, rather than equating five calendar nights with five identical charges. The conversion is financing cost in R = financing in pip-equivalent units / initial stop distance in pips, with position size and currency conversion handled consistently.

The fourth component is commission. Some raw-spread accounts charge separately by traded volume; an “ECN” label does not establish a required account type or its fee. For an illustrative USD 6 per standard-lot round trip, 200 trades of 0.1 lot cost 200 × 0.1 × 6 = USD 120. That is 12% of a USD 1,000 starting balance before considering any price outcomes. The example's USD 5–7 range is an example, not a current pricing survey. A scalping test that omits a material per-turn charge cannot support the intended net-performance claim.

Trap 6: data snooping

A researcher tries 30 strategies, discards 28 and publishes two attractive backtests. The individual calculations may be correct while the selection process makes their apparent significance misleading. Under 30 independent null tests at a 5% threshold, the expected number of false positives is 1.5, and the chance of at least one is about 78.5%. This does not mean exactly two or three must succeed, nor that the published strategies have a near-zero probability of working. It means that the full search, not only the survivors, belongs in the inference.

One conventional family-wise error control is the Bonferroni adjustment: for 30 tests at an overall 0.05 level, the per-test threshold is 0.05 / 30 ≈ 0.00167.^[2] It can be conservative, especially for related tests. Other approaches address different objectives, including false-discovery-rate control and the deflated Sharpe ratio.^[3] They are not interchangeable switches, and their assumptions and selection inputs must be supplied. In every case, hiding the failed trials prevents a meaningful assessment.

Detection: maintain a register of every strategy and variant tested, including rejected results. Two winners selected from 30 exploratory candidates need different interpretation from two genuinely prespecified evaluations. That is why field 7 of the test card is mandatory.

A worked error: stop and target inside the same candle

This execution ambiguity can materially inflate a backtest while appearing to be an ordinary trade result. Its importance depends on how frequently the strategy's exits lie within the same recorded bar.

The incorrect record: an H1 candle has open 1.0850, high 1.0885, low 1.0815 and close 1.0870. A long is assumed already filled at 1.0850, with a stop at 1.0820 and target at 1.0880, each 30 pips away. Both levels lie inside the candle's range. A custom test that checks the target condition first records +1R, although the OHLC data do not reveal which level was reached first. This is a defined example of an optimistic implementation, not a claim that every tester uses target-first by default.

The corrected execution model: OHLC alone cannot reconstruct the full intrabar event sequence. State how the ambiguity is handled:

• A conservative default counts a candle touching both exits as a stop-out. Label this pessimistic, not proven realistic, in field 5 of the test card.
• Inspect lower-timeframe or tick data for the ambiguous bars. M1 may still leave both exits within one bar; sufficiently detailed, correctly sided quote data can resolve some cases, but fills may still require assumptions about order state and liquidity.
• Calculate both extremes: all ambiguous trades stop-first and all target-first. If that range changes the decision, report the uncertainty instead of selecting the favourable endpoint.
• Record the number of ambiguous bars in the limitations. If stops and targets are close relative to the decision-bar range, the chosen OHLC resolution may be inadequate for the claim.

Synthetic calculation: a 200-trade test contains 14 such candles. Treating all as target-first gives expectancy +0.18R. Changing all 14 from +1R to −1R removes 28R, or 0.14R per trade, leaving +0.04R. A sufficiently detailed replay identifies nine stop-first and five target-first cases. Relative to the all-target case, nine trades lose 2R each: 18R / 200 = 0.09R, so the resolved estimate is +0.09R. The optimistic model overstates that estimate by 0.09R per trade, which may be material relative to the strategy's cost buffer. The H1 candle alone cannot establish this sequence. Report the limitation and the method used to resolve it, not whichever outcome looks better.

3. Walk-forward analysis — validation with limited data

Walk-forward analysis, or WFA, evaluates a sequence of development and later evaluation windows. Robert Pardo discusses the method systematically in The Evaluation and Optimization of Trading Strategies.^[1] Instead of selecting parameters on the entire history, develop on an in-sample window, evaluate the frozen choice on a later out-of-sample window, then move forward according to a procedure fixed in advance.

A chronological split: development, validation and OOS

A simple two-part illustration uses:

• Five calendar years: 1 January 2020–31 December 2024.
• Approximately 70% for development: 1 January 2020–30 June 2023, or three and a half years.
• Approximately 30% for final evaluation: 1 July 2023–31 December 2024, or one and a half years, first inspected only after the selected rules are frozen.

The 70/30 division is a scenario assumption, not a market standard. Choose a split that provides usable signal counts and relevant conditions on both sides. Chronology matters: for this forecasting and trading task, evaluation follows development. Randomly mixing observations across dates can leak future regime information and dependencies into the research process.

There are also three distinct roles, even when the first illustration has only two top-level date blocks:

• In-sample development: search, optimise and reject candidates here. Parameter selection is legitimate when it is disclosed as development.
• Validation for selection: a later development-stage window helps choose among candidates. Once used for that choice, it is part of selection, not an untouched final test. Allocate it explicitly within the development period if using the two-part example above.
• Final OOS evaluation: inspect it after freezing the strategy and evaluation procedure. If its result guides a new filter, it is no longer independent evidence for that revised filter; record the contamination.

A separate selection window is useful when comparing multiple candidates. It is not logically required for a single fully prespecified candidate, but the final OOS window must never quietly become the place where you choose the winner. “Let's try one more filter” after viewing its result changes the role of the data.

You may return to development after a weak OOS result, but you cannot change the rules, rerun the same period and call the new result untouched validation. Retain the first result and mark later use as development or diagnostic reuse. An identical rerun to verify reproducibility does not itself create a new hypothesis; an outcome-guided rule change does.

Rolling walk-forward analysis

One final period can be unusually favourable or difficult. Rolling evaluation examines the procedure across several successive periods, although it cannot guarantee that those periods represent all future conditions:

The four evaluation windows do not overlap in calendar time and together cover two years. That does not make them statistically independent. They are adjacent, the development windows overlap substantially and one persistent regime may span several evaluation blocks. Freeze the rolling selection procedure before inspecting the combined sequence; otherwise the procedure itself can be fitted to the apparent OOS results.

Positive net outcomes in three or four windows may justify further investigation of robustness, but the count alone is weak evidence. Under an illustrative independent fair-sign model, at least three positives out of four occur with probability 5/16 = 31.25%. Dependence changes that probability, not necessarily in a universally predictable direction. Report each window's trade count, net expectancy, uncertainty and regime exposure. One or two positive windows may suggest regime sensitivity; zero or one does not supply a convincing activation case. None of these counts replaces the prespecified statistical and operational criteria.

Walk-forward research for a low-frequency trader

At five to ten trades per month, a swing strategy produces roughly 300–600 trades in five years. A 30% OOS share contains about 90–180. Near a 60% win rate, an independent-binomial 95% interval has an approximate half-width of 7–11 percentage points, depending on count and method. That may constrain the win-rate claim, but it does not by itself establish a profitable payoff after costs.

At 50–100 trades per month, five years supply about 3,000–6,000 observations. Sampling uncertainty may become smaller, but dependence, selection, regime coverage and execution modelling remain important. A large trade count does not automatically make the cost model the only remaining issue.

At one or two trades per month, a position strategy supplies only 60–120 trades in five years, leaving roughly 18–36 in a 30% OOS segment. The resulting uncertainty can be too wide to distinguish useful expectancy from a loss. Possible responses include:

• Longer history: ten to fifteen years gives about 120–360 trades at those frequencies, with roughly 36–108 in a 30% evaluation share. Check whether the older market and execution conditions remain relevant.
• Multiple instruments: six to ten currency pairs can add observations, but shared currency events and overlapping positions create dependence. Validate the cross-instrument claim and do not count every trade as a new independent experiment.
• Block bootstrap: resample blocks of the observed outcomes to examine variation under stated dependence assumptions. This reuses the existing observations; it creates no new market evidence. It cannot turn a tiny sample into a well-established edge merely by generating many simulated paths.

The baseline: what is the strategy being compared with?

A positive average can be reported without a baseline, but a claim that the strategy's extra rules add value requires a relevant comparator. Before interpreting “+0.25R per trade”, apply the same data period, execution assumptions and cost accounting to explicitly defined alternatives:

• The trigger without the additional filters. If four filters do not improve the prespecified performance or risk objective, they may only reduce the sample and add flexibility. Compare more than the mean if the filters were intended to change drawdown or exposure.
• Random entries with matched frequency, direction policy and exit rules. Repeated simulations produce a reference distribution rather than one conveniently chosen number. Specify the eligible entry universe, randomisation procedure and seed. A strategy near its centre has not shown a distinction on that comparison; it is not a universal proof that every aspect of the strategy is random.
• Holding a specified direction over the period. For a single-pair swing strategy, this can reveal how much of the result resembles directional drift. Use comparable exposure and include financing; an unscaled hold is not automatically a fair risk comparison.
• The same procedure with the signal reversed. If both directions appear profitable, investigate timing, asymmetric exits, volatility effects and the cost model. The result is not automatically an error, but it challenges a claim that directional prediction is the sole source of value.

Record both the strategy and baseline results in field 10. Sensitivity analysis asks the same question locally: what happens when each parameter moves one step in either direction? If the apparent advantage disappears when an ATR threshold moves from 1.1 to 1.0, investigate the trade-set changes and uncertainty. A single attractive heatmap cell is not strong evidence of a robust edge.

Freezing OOS: folders, configuration hashes and the trial record

A weak OOS result followed by parameter changes and repeated inspection of that same period is development on the combined history, not untouched walk-forward evidence. A simple project structure helps make the distinction visible:

• Keep separate directories such as 01_raw_data, 02_IS_optimization and 03_OOS_locked. Add a clearly identified selection-validation subset and its boundaries where used.
• Before the first final OOS run, record full hashes of the specification, code, parameters, cost model and date boundaries in 03_OOS_locked/run_001.lock. Afterwards append the result and run metadata without replacing the original inputs.
• A changed candidate receives a new run record, such as run_002.lock, and a note that the previously inspected period is no longer untouched evidence for the revision. Do not erase the valid historical record of what the first frozen candidate produced.
• Each final evaluation window has one first inspection under the prespecified procedure. Identical reproducibility checks are distinguished from new outcome-guided variants. Plan rolling windows in advance and record when an earlier evaluation block becomes legitimate past data for a later scheduled development step.

A filename ending in .lock or a hash does not physically prevent edits or browsing. It provides an audit trail when paired with preserved files and disciplined access. An apparent lock without a record of who inspected which data is not proof of independence.

What to do after a weak OOS result

Do not add a filter merely because the result is uncomfortable. Follow a documented decision sequence:

1. Accept and retain the result. The current candidate has not met the intended validation criterion. A result can be inconclusive rather than conclusively negative; use the criterion written before the test.
2. Return to the concept. Review the hypothesis, development cycle and edge definition in the first three sections of article 15.3. Check whether the proposed mechanism still makes sense and whether implementation or data defects explain the result.
3. If the mechanism remains plausible, investigate regime dependence as a new research question. Four to six prespecified rolling windows may help. Two favourable trending windows and four unfavourable consolidating ones suggest a candidate regime interaction, not immediate permission to reactivate whenever a trend appears. Each window needs its sample size and net-expectancy interval. Three positive windows containing only eight trades each can remain deeply inconclusive. A regime filter discovered from those results needs a versioned specification and independent evaluation.
4. If the concept fails, retire the candidate. Preserve the reason and trial history, then move to another idea rather than accumulating exceptions around a rejected claim.

4. Monte Carlo analysis — one historical path is not the whole picture

A backtest records one sequence of opportunities and outcomes. A five-year strategy sample might contain 100 trades; a different market history could have produced 95 or 110, at other prices and with other outcomes. Resampling does not reconstruct all those possible markets. It asks a narrower question: how do the results vary under an explicitly chosen model based on the observations you already have?

A simulation produces a distribution instead of one point estimate. That distribution can reveal sensitivity hidden by the original ordering, but it is conditional on the model and input sample. It is not a complete measurement of real-world risk or a guarantee that future outcomes fall within its envelope.

Two basic resampling methods

Individual-trade bootstrap: take a list of, for example, 200 R outcomes — +1.5R, −1R, +2.2R and so on — and sample 200 observations with replacement. Repeat 1,000 times to construct 1,000 paths. Both the ordering and the frequency of particular outcomes can change.

This differs from a random permutation without replacement. Reordering the same fixed R outcomes leaves their sum unchanged. With constant-fraction sizing and no path-dependent restrictions, it also leaves the product of the equity factors unchanged, although drawdown can change. Do not describe permutation and bootstrap as interchangeable procedures.

Block bootstrap: sample contiguous blocks, for example ten consecutive trades at a time, rather than isolated trades. This preserves some dependence within each block but breaks relationships across the newly joined boundaries. Whether that is more appropriate depends on the strategy's observed dependencies. Block length is a parameter to investigate; compare choices such as 5, 10 and 20 rather than selecting whichever gives the most reassuring result.

What to calculate from 1,000 simulated paths

• Median terminal return: the centre of the simulated return distribution for the stated horizon. Call it annual only if the horizon and timing model genuinely represent a year.
• The fifth percentile of return: a lower-tail quantile within the simulation. It is not the worst possible or a guaranteed “worst realistic” outcome. Ties and finite-sample interpolation can change the fraction strictly below it.
• The 95th percentile of drawdown magnitude: an upper-tail summary of simulated peak-to-trough declines. Compare it with the plan's prespecified risk limits without treating it as a cap.
• The rate of crossing a defined threshold: for example, the proportion of paths whose drawdown exceeds 30%. Name the threshold and whether it is measured from the initial balance or a running peak. A 30% drawdown is not literal insolvency.

Illustrative figures, not historical results: suppose an annual model's original path shows +45% return and 12% maximum drawdown. A hypothetical 1,000-path simulation gives median return +38%, fifth-percentile return +8%, 95th-percentile drawdown magnitude 22%, and a 4% rate of drawdowns beyond 30%. That means about one in 25 paths in this model crosses the chosen boundary; it does not establish a 4% real-world ruin probability. Every percentage drawdown already assumes a position-sizing policy, so it is not a property of the entry signal alone.^[4]

Use one consistent set of metric definitions

Before running the code, define expectancy in R, profit factor, drawdown and its duration, recovery factor and any SQN-style score. Article 15.9 is the chapter's reference for formulas and denominators. Three points are essential here:

• Drawdown uses the running peak, not the eventual global maximum. Define H_t = max(equity up to t), D_t = (H_t - E_t) / H_t and maxDD = max(D_t). For 100 → 80 → 120 → 110, the largest decline from a preceding peak is 20%, from 100 to 80. Using the later 120 peak and earlier 80 trough gives 33.3%, which is not the path's drawdown.
• State the curve and sampling frequency. Mark-to-market equity including open positions differs from closed-trade balance. Tick, daily and after-trade sampling can produce different maxima. “12% drawdown” without those definitions is not directly comparable with another report or your plan.
• State the score convention. A mean-R/standard-deviation score multiplied by √N changes with N. Capped and uncapped variants are different quantities. Use the same convention and comparable N, and do not treat a score as a substitute for uncertainty or an allocation rule.

Keep other reporting conventions explicit as well. Profit factor in cash need not equal profit factor in R when the monetary risk unit varies. Sharpe requires a defined return interval, benchmark and annualisation assumptions; per-trade R is not automatically a calendar return series. Calmar requires a stated annualised return and drawdown over a comparable horizon. Include tail losses, losing-run length, drawdown duration and unresolved operational states instead of reducing the report to one ratio.

A Python block-bootstrap example

You can run the example in a local Python environment with NumPy and pandas, or in a notebook service such as Google Colab. Colab offers a browser notebook environment with usage limits; consult its current service and storage information. Create a notebook, paste the code into a cell and run it. The default input is synthetic, so no personal trading data are needed to see the output format.

To use your own journal, prepare a CSV with one numeric R column and one complete trade per row, in chronological order. Uncomment the import line and supply the file. Each R value should include the relevant costs once and use the positive monetary risk planned before entry. Remove personal identifiers before considering a cloud upload, and keep sensitive account exports local if you do not want them stored with a third-party service.

import numpy as np
import pandas as pd

def block_bootstrap_strategy(trades_R, n_simulations=1000, block_size=10,
                             risk_per_trade=0.01, seed=42):
    """Resample contiguous, non-circular blocks of net trade outcomes in R.

    trades_R: finite chronological outcomes, one complete trade per value.
    n_simulations: number of synthetic paths, not new market observations.
    block_size: contiguous block length; investigate 5, 10 and 20.
    risk_per_trade: fixed fraction of then-current equity risked per trade.
    seed: reproducibility input; also record the NumPy version.

    Returns are over len(trades_R) trades, NOT automatically annual.
    Drawdowns are sampled after each trade and stored as negative values.
    """
    trades_R = np.asarray(trades_R, dtype=float)
    if trades_R.ndim != 1 or trades_R.size == 0:
        raise ValueError("trades_R must be a nonempty one-dimensional array")
    if not np.isfinite(trades_R).all():
        raise ValueError("trades_R must contain only finite values")
    for name, value in (("n_simulations", n_simulations),
                        ("block_size", block_size)):
        if isinstance(value, bool) or not isinstance(value, (int, np.integer)):
            raise ValueError(name + " must be a positive integer")
        if value < 1:
            raise ValueError(name + " must be a positive integer")
    if not np.isfinite(risk_per_trade) or not 0 < risk_per_trade < 1:
        raise ValueError("risk_per_trade must be between zero and one")

    rng = np.random.default_rng(seed)
    n_trades = len(trades_R)
    # Match this example: cap an oversized block at the sample length.
    # A full-sample block has no resampling diversity; report that limitation.
    block_size = min(block_size, n_trades)
    n_blocks = (n_trades + block_size - 1) // block_size
    final_returns, max_drawdowns = [], []
    threshold_breaches = 0

    for _ in range(n_simulations):
        # Sample starting positions with replacement; preserve each block.
        starts = rng.integers(0, n_trades - block_size + 1, size=n_blocks)
        sample = np.concatenate(
            [trades_R[s:s + block_size] for s in starts]
        )[:n_trades]  # Keep exactly the original trade count.

        equity, equity_curve = 1.0, [1.0]
        for outcome in sample:
            # Absorb at zero if a loss exhausts equity. This is not a debt model
            # and does not assert that legal negative-balance protection applies.
            equity *= max(0.0, 1.0 + outcome * risk_per_trade)
            equity_curve.append(equity)
        equity_curve = np.asarray(equity_curve)
        peak = np.maximum.accumulate(equity_curve)
        drawdown = (equity_curve - peak) / peak
        worst = float(drawdown.min())
        final_returns.append(equity - 1.0)
        max_drawdowns.append(worst)
        if worst < -0.30:  # Strictly deeper than a 30% running-peak decline.
            threshold_breaches += 1

    return {
        "trades_per_path": n_trades,
        "effective_block_size": block_size,
        "median_return": float(np.median(final_returns)),
        "p05_return": float(np.percentile(final_returns, 5)),
        "p95_return": float(np.percentile(final_returns, 95)),
        # Negative DD: the fifth percentile is the adverse 95th-percentile magnitude.
        "p95_max_dd": float(np.percentile(max_drawdowns, 5)),
        "p99_max_dd": float(np.percentile(max_drawdowns, 1)),
        "prob_dd_over_30pct": threshold_breaches / n_simulations,
        "mean_return": float(np.mean(final_returns)),
    }

# Optional own data: one net R value per completed trade, chronological order.
# trades = pd.read_csv("my_results.csv")["R"].to_numpy(dtype=float)

# SYNTHETIC demonstration, not results of a market strategy.
# 20 values: 6 winners, total +2R, mean +0.10R; one +7.2R outlier dominates.
toy = np.array([-1.0, -1.0, 3.1, -1.0, -1.0, -1.0, 0.9, -1.0, 7.2, -1.0,
                -1.0, -0.6, 1.4, -1.0, -1.0, -1.0, 2.3, -1.0, -1.0, 0.7])
trades = np.tile(toy, 10)  # Artificial repetition, NOT 200 independent observations.
# To use your CSV, replace the line above with the optional import line.

print("NumPy version:", np.__version__)
for risk in (0.01, 0.02):
    result = block_bootstrap_strategy(trades, n_simulations=2000,
                                      block_size=10, risk_per_trade=risk, seed=42)
    print(f"Risk fraction per trade: {risk:.0%}")
    print(f"Median return over {result['trades_per_path']} trades: {result['median_return']:.2%}")
    print(f"Fifth-percentile return: {result['p05_return']:.2%}")
    print(f"Adverse 95th-percentile DD: {result['p95_max_dd']:.2%}")
    print(f"Paths with DD deeper than 30%: {result['prob_dd_over_30pct']:.2%}")

The calculation tests how the chosen sample behaves under resampling and the specified sizing policy. The 20-value toy sequence sums to +2R and has mean +0.10R; removing its +7.2R observation changes the picture materially. Repeating it ten times makes a 200-entry coding demonstration with artificial periodicity, not ten times as much independent evidence.

Verified example output with NumPy 2.5.3, seed 42 and 2,000 paths: at 1% risk, median return over 200 trades is +19.56%, fifth-percentile return −4.11%, adverse 95th-percentile drawdown −17.66%, and none of the simulated paths exceeds 30% drawdown. At 2% risk, the corresponding figures are +32.25%, −13.71%, −33.70% and 11.05% of paths beyond the threshold. The adverse 99th-percentile drawdowns are approximately −21.09% and −39.51%, respectively. These are synthetic outputs over 200 trades, not annual forecasts. Record the code and NumPy version as well as the seed: the generator does not promise an unchanged stream across all future versions. The important relationship is that percentage drawdown depends on sizing as well as the R sequence.

Record simulation criteria in field 9 of the test card before inspecting the output. One illustrative policy could require a positive median and fifth-percentile return, the adverse drawdown quantile below the plan's limit, and fewer than 1% of simulated paths crossing a 30% drawdown threshold. Those are scenario decision rules, not market standards. A failure means the candidate does not pass that model and scale under the chosen criteria; it is not an invitation to move the thresholds afterwards.

Important limitations: resampling cannot create an individual loss more extreme than the worst input outcome, although repeated losses can produce a deeper cumulative drawdown. It cannot invent an unseen liquidity crisis, new financing regime or missing execution failure. Zero threshold breaches in a finite simulation does not imply zero real-world risk. The code's after-trade curve omits intratrade equity variation, margin liquidation, overlapping positions and residual debt. Its zero-equity floor is a modelling convention, not a legal account protection.

The same percentage, a different cash amount

A 15% decline on PLN 5,000 is PLN 750; on PLN 50,000 it is PLN 7,500. These retain the original Polish scenario's currency, not an exchange-rate conversion. The same arithmetic holds in any account currency: 15% of 5,000 units is 750, and 15% of 50,000 is 7,500. The percentage is identical while the cash consequences and the owner's constraints can differ substantially.

Acceptable drawdown and position size at different account scales belong to article 15.11; daily, weekly and monthly limits belong to the plan in article 15.2. For this test, compare the simulated quantile with the limit documented before the run, not the amount you become willing to tolerate after seeing the result.

Why consider blocks rather than independent resampling?

Volatility clustering concerns the magnitude of movements. It does not automatically mean that a losing trade makes the next trade more likely to lose. Dependence in signs and outcomes is a property to investigate in your strategy, not a universal fact to assume. A shared regime, correlated pairs or one macro event may create sequences that independent resampling breaks apart.

Block sampling retains local sequences and can therefore give a more relevant sensitivity analysis when that dependence matters. It does not preserve every long-range relationship, and independent-bootstrap error is not guaranteed to underestimate risk in every dataset. Compare block lengths of 5, 10 and 20, disclose all results and examine the original dependence structure. A rising adverse drawdown quantile with longer blocks suggests sensitivity to the resampling scheme; it does not alone prove a particular dependence model. Little change in this small sensitivity exercise is also not proof that independence is harmless under all conditions.

5. Prospective testing — demo, paper trading and a limited live pilot

The following stages increase realism and the financial consequence of an error. Shortening the process does not remove the untested assumptions; it leaves more of them to be discovered after funds are exposed. The sequence is an educational framework, not a requirement that every candidate must eventually progress to live trading.

Two different meanings of “forward”

Always distinguish these in a test report:

• Historical forward holdout: the tester separates a later part of an existing dataset, selects parameters on the earlier part and evaluates them on the withheld segment. MT5 calls its historical split option Forward.^[6] This examines another historical window under the tester's model. It does not measure actual future execution.
• Prospective observation: the frozen strategy runs as observations arrive after the freeze date. Original time-stamped decisions cannot use observations that do not yet exist. Preserve those records, however, because later revisions or retrospective relabelling can still contaminate the evaluation. Specify whether orders are simulated, merely recorded on paper or actually executed live; only the last supplies actual live fills.

Record the mode and freeze date in field 4 of the test card. Without that date, “worked on new data” is indistinguishable from “worked on data already available when the rules were chosen”.

Stage 1: demo on incoming data

A provider's demo environment may resemble its live platform without sharing every feed, execution rule or constraint. Verify those differences rather than assuming identical conditions. Demo is useful because:

• Order and strategy errors do not directly lose real account capital.
• You can discover a defect after ten trades and return to development without a financial trading loss. Record the new version and keep the old observations rather than mixing the revised rule into the same evaluation.
• You can test tool integration: alerts, EA startup, rule implementation, order-state handling and journal exports.

Its limitations include:

• Simulated fills: slippage and liquidity handling can differ from live execution. Do not infer a provider's motive from a favourable demo fill; identify the measurable difference.
• Different financial stakes: simulated margin and stop-out mechanics can still exist, but the absence of a real monetary loss does not reproduce the user's reaction to funded exposure.
• Incomplete request behaviour: rejection, requote and partial-fill handling may differ. These depend on execution mode and product settings, not simply an ECN or STP marketing label.

An illustrative 30-day, 30-trade demo stage asks practical questions: did alerts fire at the intended time, did the EA miss signals, did the observed spread differ from the model, and did the operator change rules manually? Those counts are scenario thresholds, not market standards. At one signal every two weeks, the same observation target takes many months. Even 30 correct trades mainly support an infrastructure and process check; they do not establish a precise expectancy estimate.

Record prospective results from day one. Reconstructing a month of trades by hand invites omissions and makes reconciliation unreliable. The journal schema, imports and comparison with provider statements belong to article 15.9. At minimum, preserve signal time, request and fill times, executable prices, slippage, charges, setup version and the initial monetary risk. Without compatible accounting, the prospective R-multiple cannot be compared with the test's R.

Stage 2: paper trading on a real-time feed, optionally 30–60 days

A shadow or paper process records every eligible signal and calculates a hypothetical outcome using an explicit spread, slippage and fill model without placing an order. It provides control over the model and avoids relying on unexplained demo fills. It does not produce actual execution evidence, and it may not reveal whether the operator would have submitted the order under real stakes. The illustrative 30–60-day duration is optional and depends on what question remains unresolved after demo. It is not an automatic additional source of independent market observations if it covers the same signals.

Stage 3: a separately authorised limited live pilot

The worked scenario illustrates a small funded account of PLN 1,000–2,000 with orders of 0.01–0.05 standard lot. Retain those as Polish scenario amounts, not a suitable deposit or position-size recommendation. The valid size must come from stop distance, pip value, charges, margin requirements and the owner's loss budget. Even 0.01 lot can be too large for a small account or wide stop. If the provider's minimum size cannot fit the authorised budget, the correct outcome is not to run that pilot there.

The purpose of a limited live stage is to investigate actual implementation at bounded, explicitly accepted planned exposure, not to test whether it can provide income. It can measure:

• Actual execution: observed spreads, slippage, latency, rejections and any requotes under the real account's mode.
• Operational adherence: manual exits outside the rules, stop changes, ignored signals and other deviations. Define the error-rate denominator: eligible signals, submitted orders or completed trades are different populations.
• Actual financing: reconcile the debits and credits with the applicable live symbol specification rather than a demo schedule.
• Integration under live conditions: confirm the EA, alerts, logging and control procedures behave as intended.

The example's 90-day and 60–100-trade targets are illustrative planning assumptions. Prespecify how historical and prospective estimates will be compared, allowing for uncertainty in both samples. Simply asking whether the new mean lies inside the historical mean's confidence interval is not an appropriate two-sample comparison. Nor is overlap of two separate intervals mathematically equivalent to a confidence interval for their difference containing zero.

Estimate the difference with a method appropriate to the data and dependence, and define an acceptable degradation margin if the question concerns practical equivalence. A difference interval containing zero means the difference is unresolved; it does not prove the two implementations are equivalent. Alongside that comparison, require understood execution behaviour, compliance with risk limits, and a measured, acceptable rate of rule violations. A simulated drawdown percentile can be a predefined review trigger, not an inviolable future boundary. Record all these criteria before starting.

Transition criteria

The durations and counts below retain this example scenario. The actual decision rules belong in field 9 of the test card; the table is not permission to advance merely because enough days have elapsed.

The missing branch in many plans is “do not deploy”. A failed gate is a result to record, not an invitation to lower the criterion. This example timetable totals four to eight weeks of historical work, 30 demo days and 90 pilot days: 148–176 days, approximately five to six months, excluding the optional paper stage and any pauses. This arithmetic is a planning example, not a guarantee of sufficient evidence or a prediction that skipping a stage must produce a loss. Skipping it leaves its questions unanswered.

Operational feasibility: what a historical model does not test

A strategy requiring a decision at 13:30 Europe/London is not operationally feasible for someone unavailable in meetings until 16:00 in that same zone. These are the equivalents of the original Polish 14:30 and 17:00 example, not a universal event schedule. A prospective signal inventory will reveal the missed opportunities. Record them as non-executions; do not remove them and pretend that research and practice used the same opportunity set.

Article 15.5 covers work-compatible decision windows, alerts and higher timeframes. Article 15.1 covers workstation continuity, VPS monitoring and startup; article 15.3 records the strategy's timing and execution assumptions. These are different responsibilities, all relevant to implementation.

In that scenario, the historical bid-only model used 1.5 pips of cost while the observed model requires a 2.5–4-pip spread plus 0.5–1.5-pip slippage on each side: 3.5–7 pips in total. These are synthetic assumptions, not current GBP/JPY quotes. The difference in R depends on the stop distance. For example, increasing costs from 1.5 to 4.5 pips adds 3 pips; at D = 5 pips that is 0.6R, sufficient to move +0.4R to −0.2R. A demo check might reveal quote-model differences, but cannot guarantee discovery of live slippage. At one tenth of the monetary risk, the same −6R would be USD 15 rather than USD 150, only if the provider's minimum size and proportional-cost assumptions permit that scale. Thirty trades still give limited evidence about expectancy. Measured cost discrepancies can nevertheless justify correcting a model, while recognising that costs themselves vary and tail execution remains uncertain.

6. How many observations are enough? N, p-values and confidence intervals

A 30-trade backtest with 18 winners has an observed win rate of 60%. That is a valid description of the sample, not a precise measurement of the strategy's underlying win probability. Under independent, identically distributed Bernoulli trials, the two-sided 95% Clopper–Pearson interval is approximately 40.6–77.3%. The interval procedure has at least its stated coverage over repeated samples under the model; it does not assign a 95% probability to a fixed unknown parameter after this particular sample has been observed.

The central limit theorem, heavy tails and the absence of a magic count

The central limit theorem does not declare N = 30 universally sufficient. Its applicability and approximation quality depend on distributional and dependence assumptions. Financial trade outcomes can be skewed, heavy-tailed and clustered: one +5R winner or −3R gap loss can materially change a small sample's mean and dispersion. Convergence can be slow, and some models do not satisfy the usual finite-variance assumptions at all.

There is no single magic number of trades. Adequacy depends on the spread of R outcomes, dependence between trades, signal frequency, regime coverage, the number of tried variants and the precision required for the decision. Thirty entries exposed to the same weekly trend or macro event are not automatically thirty independent experiments. Always report the observation count, but alongside the interval, method and assumptions rather than as a standalone pass mark.

Win-rate intervals at different sample sizes

The table uses two-sided Clopper–Pearson 95% intervals with integer win counts and independent, constant-probability trials. Dependence or a changing win probability invalidates that simple interpretation; positive clustering often reduces effective information, but do not assume that every possible dependency has an identical effect. State how clustered or changing conditions are handled.

The comparison column uses 50% only because that is the breakeven rate in a two-outcome, fixed 1:1-payoff model without additional costs. More generally, with positive winner magnitude W and loss magnitude L, the no-additional-cost breakeven rate is L / (W + L), equivalently 1 / (1 + W/L). A 2R winner against a 1R loser gives 33.3%; a 0.5R winner against a 1R loser gives 66.7%. An additional constant average cost c raises the model's threshold to (L + c) / (W + L), provided that cost has not already been included in W and L.

The operational conclusion is not “200 trades are enough”. It is that uncertainty relative to the chosen decision threshold matters. At n = 100 the two-sided interval still includes 50%, so that particular interval criterion is unresolved. At n = 200 it excludes 50%, but a profitable strategy claim still needs the full payoff and costs. An unresolved result normally calls for continued research under the plan; it neither automatically authorises a smaller live position nor proves that an existing strategy must be retired. Safety limits and proven implementation defects remain separate reasons to pause.

A binomial test: is the win probability above 50%?

The null hypothesis here is p = 0.5, with the prespecified alternative p > 0.5. This is not the same as “no trading edge”. A strategy with 40% winners and a 3:1 payoff has positive expectancy before costs, while even 70% winners with a 0.5:1 payoff produce only a small positive buffer that costs can remove.

The one-sided p-value is the probability, under the null and its sampling assumptions, of observing at least as many wins as the sample contains. It is not the probability that the null is true, that the strategy has an edge or that its future result will be profitable. If using descriptive bands such as those in this example article, interpret them carefully:

• Below 0.01: a relatively unusual upper-tail result under this null for one prespecified test, not proof of a useful trading effect.
• 0.01–0.05: may pass a prespecified 5% one-sided test, but selection across many candidates requires an appropriate multiplicity procedure.
• 0.05–0.10: does not pass a 5% rejection threshold. The effect remains uncertain under that criterion.
• Above 0.10: insufficient evidence to reject this win-probability null at those thresholds. It does not prove random performance or the absence of a payoff-based edge.

Choose the test direction and threshold before seeing the result. When 30 variants have been searched, the smallest unadjusted p-value is a selected result; section 2 explains why the full trial history matters.

In modern SciPy, use from scipy.stats import binomtest, then binomtest(wins, n_trades, 0.5, alternative='greater').pvalue. The official API documentation records its introduction in version 1.7.0. The older binom_test was deprecated in 1.10.0 and removed in 1.12.0; use the current result-object API rather than copying obsolete snippets.

For 60 wins in 100 trades, the one-sided p-value is approximately 0.02844. For 18 wins in 30, it is approximately 0.18080, despite the same observed 60% win rate. The first passes an unadjusted, prespecified 5% one-sided test; the second does not. There is no contradiction with the table's 100-trade interval including 50%: that table uses a two-sided 95% interval. Match confidence level and test direction before comparing their decisions.

Test the R outcome, not only the win rate

At 40% winners, average win 3R and average loss 1R, expectancy before additional costs is 0.4 × 3 − 0.6 × 1 = +0.6R. At 70% winners, average win 0.5R and average loss 1R, it is only +0.05R. The second has a higher win rate but a much smaller cost buffer. Neither numerical assumption is evidence of actual strategy quality.

A one-sample t-test can examine mean net R against zero when its assumptions and approximation are appropriate. Its statistic is t = mean_R / (sample_sd_R / sqrt(N)), with N−1 degrees of freedom. It is not the complete statistical answer for selected, dependent or heavy-tailed trading outcomes; those may require a different inference design or resampling method.

For an illustrative sample mean +0.3R and sample standard deviation 1.5R, N = 30 gives t ≈ 1.10 and a one-sided p-value around 0.14. At N = 200, the same mean and dispersion give t ≈ 2.83 and a one-sided p-value around 0.003. The second is stronger evidence against that specific zero-mean null under the assumptions, not certainty or automatic live permission. The SciPy one-sample t-test API supports explicit alternatives; choose alternative='greater' only if that is the prespecified question.

The example shows why the same point estimate can support different conclusions at different sample sizes. Record both N and an appropriate interval around net expectancy. Reaching a particular N does not make the outcome certain, and repeatedly checking until significance appears introduces another selection problem.

The operational expectancy formula

Use fractions for W% and L%, and positive magnitudes for the conditional average winner and loser. Breakeven trades contribute zero but still belong in the total count. With 54% winners, average win 1.8R and average loss 1R, the result is 0.54 × 1.8 − 0.46 × 1 = 0.512R, approximately +0.51R.

These must be realised averages, not a planned target ratio. A target 1.8 times the initial stop distance does not imply that winners actually average +1.8R: partial exits, early closes and execution change the outcome. If the R series is already net of all charges, do not subtract costs again. If it is before specific charges, include those once before evaluating the net claim. A +0.2–0.3R cost can eliminate a smaller positive pre-cost estimate; the unit and denominator must make that calculation traceable.

SQN: one reporting convention, not an allocation decision

A score of the form SQN = sqrt(N) × mean(R) / std(R) combines average outcome, dispersion and sample size.^[5] With the same mean and standard deviation, the uncapped form rises with √N. A long test can therefore have a higher score without a larger per-trade edge. Identify the sample-standard-deviation convention and any cap on N; the capped variant discussed elsewhere in this chapter is not interchangeable with an uncapped score.

Article 15.9 provides the common metric definitions and their limitations. SQN does not prove that an edge is real, replace OOS evaluation or a cost model, or determine a capital allocation by itself.

Adding average winner 0.7R and average loser 1.2R gives (27 × 0.7 − 20 × 1.2) / 47 = (18.9 − 24) / 47 ≈ −0.109R. The sum is −5.1R, not a positive result. With a fixed monetary R equal to 1% of starting equity, that is −5.1% of starting equity. If every one of the 27 wins is exactly +0.7R and every loss exactly −1.2R, risking 1% of current equity each time gives 1.007^27 × 0.988^20 − 1 ≈ −5.17%. If only the two averages are known, that compounded result is not uniquely determined because individual outcome magnitudes also matter.

A positive cash result alongside a negative unweighted mean R could reflect different monetary risk weights, deposits or withdrawals, fees or a reconciliation error; investigate rather than assuming one explanation. Variable risk does not make expectancy in R uncomputable. Each trade needs its own positive R₀ fixed before entry, with R_i = net P/L_i / R₀_i. What you cannot do is convert the unweighted sum of R into the account's percentage change without the risk weights, equity path and cash-flow information. The lesson is to preserve the missing columns, not to dismiss eight months of observations because one metric was incomplete. Small or dependent sample: limit the inference.

7. Execution and cost modelling: spread, slippage, financing, commission and partial fills

Understating costs can turn a losing implementation into an apparently profitable backtest. Model the four cost components separately, then reconcile them with the fill model so that each is counted exactly once. A detailed fee spreadsheet does not compensate for impossible fills.

Spread: fixed or variable?

Tick Data Suite is one MT4 workflow for imported tick data, variable spreads and additional execution settings. Its provider describes these capabilities and markets a 99% modelling-quality figure. That figure is a tester-related indicator, not a 99% probability that the strategy or its live fills are correct. The purchase decision depends on which missing data or modelling capability your research actually needs, not on the badge.

A spread-sensitive scalp or release strategy can change sign when the model changes, while a swing strategy with a roughly 100-pip stop may be less sensitive to a small spread difference. Measure that sensitivity rather than assuming a paid tool must repay its cost. This article does not recommend a purchase or provide current subscription prices.

Slippage: static and conditional models

Slippage is the difference between the chosen reference price and the actual fill, with direction and order side specified. Comparing a historical model price with a live fill is useful only after matching the reference timestamp and quote convention. Possible models include:

• Static: a fixed adverse allowance, such as 0.5 pip on each entry and exit. Simple, but potentially inadequate for stress conditions.
• ATR-conditioned: an illustrative allowance of 0.1 × completed M5 ATR(14), converted into pips. It increases with the chosen volatility measure, but the coefficient requires calibration and may not describe tails.
• Size and depth conditioned: a model using order quantity, available depth and the venue's execution process. This needs the relevant depth and order-state data; ordinary bid/ask tick history does not supply it automatically.

The example's three-regime starting sketch — fractions of a pip in ordinary conditions, several pips around breakouts and potentially double-digit pips after releases — is illustrative, not a measured account distribution. Build your own table from matched requests and fills. With positive numbers meaning adverse slippage, a buy uses fill - reference, while a sell uses reference - fill, then divide by pip size. A long exit is a sell and a short exit is a buy. Preserve favourable fills too, along with rejected requests, and report the reference used.

A few dozen trades may reveal an ordinary implementation discrepancy, but they rarely characterise rare execution tails reliably. Record the sample period, trade sizes and conditions rather than presenting a small average as the whole slippage model.

Swap and financing

Positions held across the applicable financing boundary can receive a debit or credit. Platform symbol specifications support different calculation modes, so a displayed swap number is not necessarily “pips per lot per night”. Check whether it represents points, a currency amount, an interest calculation or another product-specific convention before converting it.

• Direction: obtain long and short rates independently. They are not necessarily equal and opposite, and both can be charges. Do not assume a permanent positive short EUR/USD swap or negative long swap from an old rate environment.
• Day multipliers: many T+2 FX instruments use a triple adjustment associated with Wednesday rollover, but the actual day and holiday exceptions depend on the instrument and provider. Model the position's calendar rather than multiplying an undated night count.
• Rate changes: financing reflects the applicable rate environment and provider terms. For historical testing, use historical schedules where available or disclose a scenario approximation. Today's table does not automatically reconstruct costs several years ago.

For a three-to-eight-day swing position, financing may consume a material share of expectancy. Convert a pip-equivalent charge into R using the initial distance, then apply the actual sequence of dated rates and multipliers. Omitting a material charge leaves the net claim unsupported; a credit should likewise be recorded with its correct sign, not assumed to cancel another cost.

Commission

Some accounts charge by traded volume in addition to spread. The example illustrates USD 5–7 per standard-lot round trip for a major, USD 6–10 for a minor and USD 8–15 for an exotic. These are scenario ranges, not current quotations, and ECN/STP labels do not determine the fee. Use the applicable account schedule, including minimum charges and tiering.

To express commission in pips, divide the round-trip commission for one lot by that lot's pip value in the same currency. For the standard EUR/USD illustration, USD 6 / USD 10 per pip = 0.6 pip per round trip. If the account is in PLN or another currency, convert both amounts consistently; converting only the numerator changes the ratio incorrectly.

At 200 monthly trades of 0.1 lot and USD 6 per standard-lot round trip, commission is USD 120 per month. Relative to USD 1,000 of starting equity, that is 12% monthly turnover cost, or USD 1,440 — 144% of that starting amount — over twelve identical months. It is an arithmetic total, not a forecast or a compounded account return. It illustrates why turnover and costs need scrutiny; it does not prove that every small-lot strategy is uneconomic.

An illustrative policy keeps annual commission within an illustrative 20–30% share of the expected strategy return. Treat any such ratio as a scenario policy, not a universal capital threshold, and define whether its denominator is gross expected profit or profit after the same commission to avoid circular reasoning. Increasing account capital alone does not improve a per-trade net edge when exposure scales proportionally. Evaluate actual net outcomes and uncertainty first.

Tick Data Suite: an MT4 tick-data workflow

The original workflow can be retained without treating the software name as a certificate of professional validation:

• Provider and licensing: review the provider’s current official site and its licensing and compatibility terms before any purchase. Site addresses and redirect behaviour can change.
• Data: Dukascopy's historical-data export is one possible source. Inspect instrument coverage, timestamps, bid/ask fields, gaps and usage terms. A source offering fine timestamp resolution does not guarantee complete executable liquidity.
• Initial setup: the example's two-to-three-hour allowance is an example, not a promise. Download a small chosen period first, confirm time conversion and symbol properties, connect the data to the documented tester workflow and verify which tick mode is actually used.
• After setup: check variable spread, commission, financing, volume constraints, margin and stop settings against the intended account. Review sample trades manually. Do not interpret “Every tick” alone as proof that ticks were recorded rather than generated.

The purpose is to investigate a difference between a simplified spread model and a more relevant recorded-data model. The tool cannot validate the strategy hypothesis, repair unavailable depth data or guarantee that future costs resemble the past. Its usefulness follows from the research requirement and the checks you perform.

Execution modelling: delay, gaps and partial fills

Cost accounting is only half the implementation model. Field 5 must also state what happens between the signal and the fill:

• Signal time and latency: after a confirmed candle close, calculation, transmission and acceptance take time. Manual decisions may take seconds; automated delays depend on the system and provider. Measure or explicitly model them. A zero-delay assumption can be material for a breakout, but its contribution cannot be declared a fixed share of cost without evidence.
• Gaps: an ordinary stop-market order cannot fill at a price that is unavailable. Model the next available executable quote under the order rules, including the possibility of delayed or failed execution. Always filling exactly at the stop understates some tail losses, especially around closures and releases.
• Partial fills: use the quantity-weighted average of the fills rather than only the first price. A partial quantity usually creates less exposure than the full request if other terms are equal, but changing fill prices and charges can alter that comparison. Under the journal convention used here, retain the original pre-order R₀ and record the unfilled fraction separately. If another implementation uses allocated filled-position risk, record that denominator as a distinct metric rather than changing it retrospectively. Partial exits from one entry decision remain one trade; counting them as separate observations changes weighting and can distort win rate.
• Requotes and rejections: a historical signal may not become a live fill. Record request outcomes and distinguish confirmed rejection from an unknown state. Fifty candidate signals and 41 executed trades can reflect nine execution failures rather than nine missing signals; reconciliation must compare like with like.

Stress each material assumption: longer delay, worse slippage, an adverse resolution of an ambiguous candle. If a plausible change removes the edge, report the sensitivity and the missing implementation evidence. Do not declare every pessimistic scenario realistic merely because it is conservative. A model assuming instant, complete fills at every reference price needs that limitation stated openly. MAE/MFE diagnostics and their sampling conventions are developed in article 15.9.

A combined cost ledger for a validation script

The following runnable teaching example expands this example pseudocode. All components use the pair's pip units. The small spread and commission tables are synthetic, not provider data. UTC hour bands are deliberately simplified and do not model daylight-saving changes or release shocks. Production research should use dated quotes or a properly specified conditional model.

Financing events supply separate long and short costs and the applicable day multiplier; the function does not assume opposite rates or guess triple swap from a plain night count. Flags identify components already included in the tested P/L. The result is the additional adjustment still to deduct, not permission to charge the entire ledger a second time.

import math

def total_cost_per_trade(pair, hour_utc, atr_pips, direction,
                         financing_events=(), *, spread_in_fills=False,
                         slippage_in_fills=False, commission_in_result=False,
                         financing_in_result=False):
    """Additional cost in pips; positive = cost, negative = net credit.

    Each financing event contains independent 'long' and 'short' daily
    pip-equivalent COSTS and the applicable 'multiplier'. A credit is negative.
    Populate dated events externally from the actual holding period and rates.
    """
    spread_table = {
        "EURUSD": {"low": 1.5, "normal": 0.7, "peak": 0.3},
        "GBPJPY": {"low": 4.0, "normal": 2.0, "peak": 1.2},
    }
    commission_table = {"EURUSD": 0.6, "GBPJPY": 0.8}  # Synthetic RT pips.
    if pair not in spread_table:
        raise ValueError("Unsupported pair: supply an explicit cost model")
    if direction not in ("long", "short"):
        raise ValueError("direction must be long or short")
    if isinstance(hour_utc, bool) or not isinstance(hour_utc, int):
        raise ValueError("hour_utc must be an integer from 0 to 23")
    if not 0 <= hour_utc <= 23:
        raise ValueError("hour_utc must be an integer from 0 to 23")
    if not math.isfinite(atr_pips) or atr_pips < 0:
        raise ValueError("atr_pips must be finite and nonnegative")
    regime = "low" if hour_utc >= 22 or hour_utc <= 6 else (
        "peak" if 8 <= hour_utc <= 16 else "normal"
    )
    spread = 0.0 if spread_in_fills else spread_table[pair][regime]
    slippage = 0.0 if slippage_in_fills else 2 * 0.1 * atr_pips
    commission = 0.0 if commission_in_result else commission_table[pair]
    financing = 0.0
    for event in financing_events:
        rate = float(event[direction])
        multiplier = float(event["multiplier"])
        if not math.isfinite(rate) or not math.isfinite(multiplier) or multiplier < 0:
            raise ValueError("Invalid financing event")
        if not financing_in_result:
            financing += rate * multiplier
    return spread + slippage + commission + financing

# Synthetic example: separate rates, not an assumption of opposite signs.
events = [{"long": 0.3, "short": 0.1, "multiplier": 3}]
print(total_cost_per_trade("EURUSD", 10, 5.0, "long", events))  # 2.8 pips
print(total_cost_per_trade("EURUSD", 10, 5.0, "long", events,
                           spread_in_fills=True))             # 2.5 pips

For a simple check, +0.4R before additional costs becomes +0.2R if those costs are 0.2R, or −0.1R if they are 0.5R. Convert the pip ledger using the relevant initial risk distance and currency convention. Without an explicit ledger and fill model, you do not know which net result the historical report represents.

8. Cherry picking and signal selection

Methodological cherry picking occurs when a researcher removes some signals and reports the attractive remainder without a valid, prespecified selection rule. It can be deliberate or unconscious. A losing full implementation may then appear profitable because unfavourable observations have disappeared from the reported sample.

Three forms of cherry picking

1. “I skip signals that do not look right.” A historical chart contains a valid signal, but the researcher dismisses it as the wrong context. If later candles are visible or remembered, “not right” may correlate with the losing outcome rather than with a rule available at the original decision time. A discretionary strategy can be studied, but it needs a defined, blinded or prospective classification protocol and retained pre-outcome decisions.

2. “I add a filter after seeing the losses.” A group of Tuesday-morning losers prompts a new Tuesday exclusion. The same historical period improves by construction. That does not establish whether Tuesdays are genuinely different. Treat the filter as a new hypothesis, record why it was proposed and validate it independently rather than presenting the selected sample as confirmation.

3. “I use only clean periods.” The researcher removes stressed history, such as March 2020, the 2008 financial crisis or the late-2018 sell-off, then reports the remaining periods as representative. These examples need accurate labels: Lehman Brothers failed in September 2008, and the event commonly called the taper tantrum was in 2013, not December 2018. A prespecified strategy may exclude observable conditions, but it cannot exclude an unexpected crisis retrospectively merely because the outcome is inconvenient. Preserve stressed periods and state what the rules would actually have done with the information available then.

How to detect selective signal handling

• Count candidates, eligible signals and executions separately. The example's 10% and 30% manual-skip warning levels are illustrative audit triggers, not probabilities that cherry picking has occurred. Even a smaller outcome-selected omission can bias a result; a high rule-based exclusion rate can be legitimate.
• Record the rejection reason and its timestamp. A filter written before evaluation differs from an improvised decision after the outcome. For intentional discretion, preserve the decision-time judgment rather than silently treating it as a mechanical filter.
• Reconstruct the complete period. Compare all eligible signals with the selectively executed subset, including difficult conditions. A performance gap calls for diagnosis: it may reflect selection, execution failure or a genuine prespecified filter effect. The gap alone does not identify the cause.

The signal inventory: a concrete control

Assign every candidate a status in a log generated automatically or reconciled from the journal. Make statuses mutually exclusive at the chosen level, and preserve separate request and fill records where a candidate proceeds to execution:

Provider or infrastructure failures after a valid signal require additional execution statuses, such as order_rejected, order_state_unknown or partial_fill. They are not automatically invalid market data. Keep the denominator explicit when reporting an error rate; otherwise unrestricted rule exclusions can make the “taken” target mathematically misleading.

Under the example's policy, manual skips above 10% stop progression until the discrepancy is understood. Either formalise a defensible reason as a new rule and evaluate the new version, or acknowledge that the intended method cannot be executed reliably under the current workflow. Do not relabel old discretionary omissions as though the new filter existed at the time. The numerical threshold is a process choice, not a boundary below which selection bias disappears.

Cherry picking in an EA sales presentation

A prudent purchasing criterion is the ability to reproduce the relevant test with your own data and cost assumptions and to understand its limitations. Source access permits deeper inspection, but a closed-source product is not automatically fraudulent; it has a validation limit that your policy may find unacceptable. A high price, smooth curve or strong rating does not fill that gap. Do not buy or deploy merely because one selected report looks convincing.

9. Common testing mistakes

Mistake 1: optimising on the entire dataset

A researcher tries 200 parameter combinations over five years and selects the one showing +180%, with no untouched evaluation. The maximum is a selected result; its size alone does not establish an edge. Use a chronological development and evaluation design, record the split in field 4 and all 200 trials in field 7. The example's 70/30 split is an example, not a universal rule. A prospective stage remains useful, but it does not retrospectively make the earlier selection an independent test.

Mistake 2: treating a small historical sample as decisive

A six-month, 24-trade backtest is still a backtest, but it may be weak evidence for a broad long-term claim. Its uncertainty can be large relative to the mean, and the period may contain only one relevant regime. Choose history that addresses the intended scope and report the regimes actually observed; “several years” is not itself a guarantee of coverage. Counts such as 200 or 500 in this article are scenario examples, not market standards. Judge precision from the distribution, dependence and decision threshold.

Mistake 3: using only one quote side without modelling the other

A long opened and closed at bid omits the entry ask and can overstate the result by approximately the spread under a simple matched-price comparison. Long entry uses ask and exit bid; short entry uses bid and exit ask. Bid-only history is not automatically unusable if a justified ask and execution model is added, but the approximation must be explicit. MT5's recorded-tick mode and an appropriately configured MT4 tick-data workflow are options to investigate, not proof of complete realism.^[6] Record bid, ask, midpoint and generated-versus-recorded data conventions in field 3.

Mistake 4: ignoring regime differences

A result from one policy, volatility or liquidity environment may not transfer to another. Do not label whole multi-year periods as uniformly low-rate, high-rate, QE or QT when conditions changed within them. Rolling evaluation can reveal differences across observed windows, but identifying a regime-dependent edge requires enough evidence and a rule that recognises the regime using information available at the time. A filter invented after seeing the favourable windows is a new research variant, not an automatic activation rule.

Mistake 5: claiming transferability without testing it

A successful-looking EUR/USD test does not establish that the same method works across currencies. If the claim is cross-pair, evaluate a prespecified set, perhaps five to eight pairs in an illustrative research design, with appropriate costs and dependence handling. If the original claim is genuinely EUR/USD-specific, failure on another pair does not by itself disprove it or prove curve fitting. The scope of the conclusion must match the scope of the evidence.

Mistake 6: confusing return with risk-adjusted performance

Suppose strategy A reports a 35% annualised return and 22% maximum drawdown, while B reports 25% and 8% over a comparable measurement horizon. A simple Calmar-style comparison gives:

• A: 35 / 22 ≈ 1.6.
• B: 25 / 8 ≈ 3.1.

B has the higher return-to-observed-drawdown ratio under those definitions. That does not establish every dimension of stability or future risk. You cannot calculate Sharpe from those two numbers alone. Sharpe requires a return series at a specified frequency, a matching reference return and an annualisation convention: (mean periodic return - reference return) / periodic return standard deviation, with any square-root-of-periods annualisation justified by the assumptions.

Dividing an annual return by a monthly standard deviation without consistent units is not an adequate Sharpe calculation. Missing inputs mean the metric is unavailable, not permission to invent an approximate value. Article 15.9 develops Sharpe, Sortino and Calmar and what each leaves unmeasured.

Mistake 7: stopping measurement after deployment

A candidate may pass historical research, demo and a limited pilot, then stop being measured once its allocation increases. Keep reconciling live implementation and comparing outcomes with the test's assumptions, uncertainty and scenario limits. A new sample has its own uncertainty; it is not enough to ask whether its mean lies inside the old sample's confidence interval. Article 15.10 defines the review rhythm and retain, observe, reduce or stop decisions. This article supplies the comparison record: metric definitions, original intervals, costs and simulated drawdown quantiles.

Mistake 8: changing the method after every losing run

Weekly parameter changes after five to ten losses can outrun the information arriving about any single version. With independent outcomes and a 52% win probability, five losses in one prespecified block have probability 0.48⁵ ≈ 2.5%. The chance of at least one such run somewhere in 100 trades is about 75%. The latter has many overlapping opportunities for the run; treating those windows as independent using 1 - (1 - q^k)^(N-k+1) overstates the exact run probability in this example.

These are model calculations, not a claim that your trade outcomes are independent. The lesson is to ask the right probability question before declaring the strategy broken. Use prespecified research review criteria rather than the appearance of the latest week. Operational failures, exceeded risk limits or a demonstrated data defect may still require an immediate pause; you do not need a statistically significant expectancy decline before protecting the account. A rule amendment creates a new version and needs its own evaluation.

Before changing a rule: check these layers in order

Do not confuse a strategy defect with an implementation or data problem. Establish what the result measures before editing the trading rules:

1. Data and execution. Reconcile the journal with provider records, confirm the version and count rule deviations. A contaminated period cannot be interpreted as clean performance of the specified strategy; it may still provide useful evidence about the operational failure.
2. Costs. Compare actual spread, slippage and commission with the card. A 0.15R expectancy reduction alongside a 0.15R cost increase may explain the discrepancy. Costs are part of the tradable strategy's viability even if the entry mechanism has not changed.
3. Regime. Check whether the conditions under which the strategy was intended to operate actually occurred and whether its eligibility filter was applied correctly.
4. Edge. Only then assess the outcome distribution and uncertainty relative to the prespecified decision rule, not merely the last dozen trades' mean.

The full “change the system or change the execution” decision table, downgrade criteria and review process belong to article 15.10. The rule here is to resolve the first three diagnostic layers before attributing the discrepancy to entry and exit logic. Any genuine rule change returns to the versioned testing process; it is not an invisible patch to the old evidence.

10. Checklist — two separate live-decision gates

The checklist contains 20 items in five categories: statistical evaluation, methodological controls, costs, prospective testing and operational discipline. There are two gates because item 17 requires actual live observations and cannot be a prerequisite for the first limited live study. In this version, review 1–16, 19 and 20 before considering that separately authorised study; review all 20 before any subsequent scaling decision. The ongoing measurement plan in item 20 should exist before the pilot, not only after it.

A score does not compensate for a missing critical requirement. “Not applicable” requires a written reason. All durations, trade counts and numerical policy thresholds here are scenario choices, not market standards. Completing the list is neither certification of profitability nor authority to place orders.

The page does not submit these answers or deliberately persist them as a research record. A browser may restore form state when revisiting; that is not evidence that the items were verified. Save the reviewed checklist with the dated test archive.

Required archive: eleven components

A deployment review needs an audit trail. Without it, six months later you may be unable to reconstruct which rules, data and assumptions produced the original result:

1. The twelve-field test card: fields 1–9 before evaluation, results and limitations afterwards, with the reproduction manifest completed and verified.
2. Historical inputs: raw market and calendar data, source, acquisition date, permissions, hashes, cleaned versions, transformation code, and a log of gaps, duplicates and revisions.
3. The strategy specification and implementation: readable rules, code, parameters, dependency and platform versions, freeze timestamp and exact commands. Freeze before OOS, not after seeing it.
4. IS, validation and OOS reports: split boundaries, complete trial history, equity curves, expectancy intervals, profit factor and drawdown by rolling window, plus comparable baseline results.
5. Simulation and stress report: illustratively 1,000 or more paths, with seed, code, block scheme, horizon, sizing, return and drawdown quantiles, threshold-breach rates and limitations. More paths are not more market observations.
6. The cost and account model: session or tick spreads, slippage assumptions, separate financing rates and day multipliers, commission, conversions, margin and symbol constraints.
7. Demo or paper exports: the complete planned observation period, illustratively 30 days, with original signal timestamps, statuses and historical reconciliation.
8. Limited-live journal, if authorised and completed: the planned period, illustratively 90 days, with each trade's R₀, net R, MAE/MFE convention, actual costs, requests, fills, incidents and before/after screenshots where useful.
9. Signal inventory: all candidates and reasons for exclusions, manual skips, invalid data and execution failures, with reproducible deviation-rate denominators.
10. Provider settings record: account type, symbol terms, leverage, minimum volume, stops and fees as applicable to the test date. Retain evidence needed to investigate an execution dispute, without exposing credentials.
11. Recoverable backups and decision record: frozen EA, scripts and templates, reviewed conclusions, unresolved limitations, ownership, recovery locations and conditions for pausing or resuming.

An illustrative directory is strategy_X/2026-Q2/{01_data,02_IS,03_OOS_locked,04_montecarlo,05_demo,06_micro_live,07_artefacts}. Keep recoverable copies on appropriate storage and protect sensitive exports. Cloud storage is a choice governed by your data policy, not a requirement. Hashes reveal differences between files; they do not prove that the underlying quotes are correct.

Completing the first gate permits a review of whether to authorise a limited live study, not an automatic order. Completing all twenty permits a later deployment or scaling review under the plan. A missing critical condition sends the project back to the unresolved stage. The equally valid final decision is not to deploy.