Summary: Part 5 of a 10-part series. Professional backtesting methods for forex: tick data requirements, walk-forward validation, Monte Carlo simulation, and common backtest pitfalls.




Title: Backtesting Methods Part 5: Tick Data, Walk-Forward and Monte Carlo

A backtest that looks profitable is often a lie. Most retail backtests suffer from three fatal flaws: survivorship bias, look-ahead bias, and overfitting. This guide provides four professional backtesting methods that expose these flaws.

1. Method 1 – Tick Data Requirement
OHLC data is insufficient for realistic backtesting. Limit orders fill differently than market orders.
  • Minimum requirement: True tick data with bid/ask spread reconstruction

  • Where to get: Dukascopy (free historical tick data), Forex Tester, QuantConnect

  • Red flag: If your platform only offers OHLC, your backtest error margin is ±30%


    • 2. Method 2 – Walk-Forward Analysis (The Gold Standard)
    Walk-forward prevents overfitting by separating optimization from testing.
    ```python
    # Walk-forward backtesting structure
    total_data_years = 5
    optimization_window_years = 3
    testing_window_years = 1

    for year in range(0, total_data_years - optimization_window_years, testing_window_years):
    optimize_on = data[year:year + optimization_window_years]
    test_on = data[year + optimization_window_years:year + optimization_window_years + testing_window_years]
    best_params = optimize(optimize_on)
    performance = backtest(test_on, best_params)
    save_performance()
    # Final result = average of all testing window performances
    ```
  • Passing standard: Testing window performance must be at least 70% of optimization window performance

  • If not passing: Your system is overfit. Reduce parameters or simplify logic.


    • 3. Method 3 – Monte Carlo Simulation for Drawdown
    Historical backtests show only one path. Monte Carlo simulates thousands of random sequences.
    ```python
    # Simplified Monte Carlo logic for drawdown estimation
    original_trades = load_all_backtest_trades() # list of P&L values
    def monte_carlo_drawdown(simulations=1000):
    results = []
    for i in range(simulations):
    shuffled_trades = random_shuffle(original_trades)
    equity_curve = cumulative_sum(shuffled_trades)
    max_dd = calculate_max_drawdown(equity_curve)
    results.append(max_dd)
    return {
    "median_dd": median(results),
    "90th_percentile_dd": percentile(results, 90),
    "worst_case_dd": max(results)
    }
    ```
  • Interpretation: If 90th percentile drawdown exceeds 25%, your system needs stricter risk controls regardless of historical results.


    • 4. Method 4 – The Six Red Flag Checklist
    Before trusting any backtest, verify these six items:

    | Check | Pass/Fail |
    |-------|-----------|
    | No survivorship bias (includes delisted pairs) | |
    | No look-ahead bias (uses only data available at trade time) | |
    | Slippage modeled (1-2 pips for forex) | |
    | Commission modeled ($3-7 per lot round-turn) | |
    | Minimum 200 trades (not candles) | |
    | Tested on at least 3 different currency pairs | |

    5. Common Backtest Pitfalls – Exact Numbers
  • Pitfall 1: Using "Open" price for entry on the same bar → your backtest is trading the future. Fix: Use "Close" of previous bar or next bar's Open.

  • Pitfall 2: Ignoring spread widening during news. Fix: Backtest around NFP releases separately and model 3-5 pip spread.

  • Pitfall 3: 100 trades backtest → 95% confidence interval width is ±15%. You need 400+ trades for ±5% accuracy.


    • 6. Backtest to Forward Test Protocol
    ```
    Step 1 – Backtest (tick data, walk-forward, Monte Carlo) → PASS/FAIL
    Step 2 – Paper trade forward test (100 trades, live market, no real money) → Compare metrics to backtest
    Step 3 – If forward test profit factor is within 80% of backtest → Pass
    Step 4 – Small real account (1% risk per trade, 50 trades) → Compare again
    Step 5 – Full deployment
    ```
    Critical rule: If forward test results differ from backtest by more than 25%, your backtest method is flawed. Stop and rebuild.

    7. Backtesting Frequency
  • Weekly: Run a rolling 6-month backtest on your active system to detect degradation

  • Quarterly: Full walk-forward re-optimization (but only if performance has decayed >15%)

  • Annually: Complete Monte Carlo stress test with black swan scenarios (e.g., 2015 Swiss Frank)


    • 8. Next Step
    Part 6 covers risk management – black swan hedges, circuit breakers, and maximum drawdown control systems.

    Reference:
  • Pardo, R. (2008). *The Evaluation and Optimization of Trading Strategies*. Wiley.

  • Aronson, D. R. (2006). *Evidence-Based Technical Analysis*. Wiley.