Advanced Grid Search Optimization for MQL4 EA: Avoid Overfitting and Boost Backtest Accuracy

Summary: This guide covers advanced grid search optimization techniques for MQL4 EAs, including parameter space design, performance surface analysis, and robustness validation with executable code examples.




Grid search remains the gold standard for deterministic EA parameter optimization in MQL4. Unlike genetic algorithms, it guarantees global optimum discovery within defined step resolutions. However, naive grid search produces overfitted strategies. This guide covers advanced techniques that maximize robustness.

Parameter Space Design Principles

For k parameters with step counts s1, s2, ..., sk, total combinations = ∏si. To maintain practical runtime while preserving resolution, apply logarithmic scaling for sensitive parameters and linear scaling for stable ones.

Mathematical Formulation of Optimization Objective

Define the objective function:
\[
F(\theta) = \frac{\mathbb{E}[R(\theta)]}{\sigma(R(\theta))} \times \text{ProfitFactor}(\theta) \times (1 - \lambda \cdot \text{DDF}(\theta))
\]
where θ = (p1, p2, ..., pn) represents parameter vector, DDF is downside deviation factor, and λ = 0.3 is the penalty coefficient. This composite metric reduces overfitting compared to using net profit alone.

Code: Two-Pass Grid Optimization with Stability Filter
```cpp
// MQL4 code - Two-pass grid optimization with robustness validation
input int MAPeriodMin = 5;
input int MAPeriodMax = 50;
input int MAPeriodStep = 5;
input double StopLossMin = 20.0;
input double StopLossMax = 100.0;
input double StopLossStep = 5.0;

struct OptimizationResult {
int MAPeriod;
double StopLoss;
double SharpeRatio;
double ProfitFactor;
double StabilityScore;
};

void TwoPassGridOptimization() {
OptimizationResult bestResults[];
int resultCount = 0;

// First pass: coarse grid
for(int ma = MAPeriodMin; ma <= MAPeriodMax; ma += MAPeriodStep) {
for(double sl = StopLossMin; sl <= StopLossMax; sl += StopLossStep) {
double sharpe = CalculateSharpe(ma, sl);
double pf = CalculateProfitFactor(ma, sl);
double stability = ComputeStabilityScore(ma, sl);

// Filter threshold
if(sharpe > 0.5 && pf > 1.3 && stability > 0.7) {
OptimizationResult res;
res.MAPeriod = ma;
res.StopLoss = sl;
res.SharpeRatio = sharpe;
res.ProfitFactor = pf;
res.StabilityScore = stability;
ArrayResize(bestResults, resultCount + 1);
bestResults[resultCount++] = res;
}
}
}

// Second pass: fine grid around top candidates
for(int i = 0; i < resultCount; i++) {
int maCenter = bestResults[i].MAPeriod;
double slCenter = bestResults[i].StopLoss;

for(int ma = maCenter - MAPeriodStep/2; ma <= maCenter + MAPeriodStep/2; ma += MAPeriodStep/4) {
for(double sl = slCenter - StopLossStep/2; sl <= slCenter + StopLossStep/2; sl += StopLossStep/4) {
ValidateCandidate(ma, sl);
}
}
}
}
```

Performance Surface Analysis

After grid optimization, compute the local smoothness metric:
\[
S_{\text{local}}(p_i) = \frac{|F(p_i) - F(p_{i-1})|}{|p_i - p_{i-1}|}
\]
High variance in S_local indicates parameter sensitivity and poor robustness. Reject parameters where max(S_local) / min(S_local) > 5.

Out-of-Sample Validation Protocol

1. Reserve 30% of historical data as out-of-sample (OOS)
2. Optimize only on 70% in-sample
3. Top 3 parameter sets must exceed in-sample Sharpe by no more than 15% on OOS
4. Re-optimize quarterly using expanding window

Avoid Future Function in MT4 Backtest

Never reference indicator values at current bar index 0. Use shift 1 for entry signals based on confirmed closed bars. Example:
```cpp
// Wrong - future leakage
double maCurrent = iMA(NULL,0,14,0,MODE_SMA,PRICE_CLOSE,0);
// Correct - robust
double maPrevious = iMA(NULL,0,14,0,MODE_SMA,PRICE_CLOSE,1);
```

Reference: MQL4 Documentation - Optimization Tester (https://docs.mql4.com/tester), "Systematic Trading" by Robert Carver, 2015.