Articles

Did the optimizer pick your result?

Count the configurations tried, export MT5 passes and understand how deflated Sharpe affects the report's class.

The chosen configuration's report tells you how that test ended. To examine the influence of selection, you also need the search that preceded it. Collect the passes, the criterion used to rank them and variants tested outside the optimizer. Then compare Sharpe with a reference that takes that search into account.

Reconstruct what could be selected

A configuration is a combination of decisions that was evaluated: parameters, entry and exit rules, market, trading hours or period. If you inspected the result to choose between alternatives, that comparison belongs to the search. Count manual changes and earlier runs that influenced the final selection too; a filename does not describe that process.

Keep a record of alternatives, dates and the selection criterion. Combinations never executed are not observed passes. In a genetic search, the theoretical space of combinations and the passes actually executed are different quantities. If records are missing, identify the scope of the available count and declare additional attempts you know about.

Read Sharpe alongside the selection process

Selecting the highest result from a search favors values lifted by fluctuation. Deflated Sharpe compares the observed Sharpe with a reference that rises as more attempts are considered. In the report it uses history length, return skewness and tails. It does not describe the probability of a future outcome or identify the cause of a curve on its own.

The public calculator lets you explore the luck reference with declared figures. Its assumptions about frequency, normal tails and no skew simplify the problem. Nearby configurations may be correlated; that dependence limits interpretation and does not justify deleting variants from the record. With the files, the report can examine information that a standalone Sharpe figure lacks.

Declared · Assume 100 independent variants and 3 years of daily returns, with 252 periods per year and a declared annual Sharpe of 1.8. The calculator puts the expected Sharpe of the best unskilled variant at 1.47. This calculation assumes no skew and normal tails; it is not a measurement of a portfolio.

Export the complete MT5 passes

When optimization finishes, open Optimization Results in the Strategy Tester. Right-click the table and choose Export to XML (MS Office Excel). Keep the Excel 2003 XML with all passes, including the ones you discarded. The linked export guide explains the file and its upload field.

Then run a single test with the chosen parameters and save its HTML report. Upload the XML under 'MT5 optimisation export', inside 'Add more files', alongside the HTML under 'Your platform report'. The XML contains summaries per pass, not the selected result's trades: the files supply different pieces of evidence.

Separate declared counts from file counts

Declared identifies the attempt count you enter. Measured identifies counts from files: XML passes, columns in a variants matrix or variants present in a report. Rigor uses the largest available count across the declaration and those files; a lower declaration cannot reduce the documented count. Its scope still depends on what you provide.

Not measured identifies a missing count when neither your declaration nor the files supplies one. The calculation then assumes a single attempt, the most favorable case, and shows the limitation. It does not treat that assumption as a measured search. If you tested variants outside the XML, include them in your declaration and keep the record explaining the total.

What can change in the class

With no count, multiplicity that looks sufficient under the most favorable assumption stays Not measured and the class cannot exceed B. Declaring the count removes that particular gap, but may reduce deflated Sharpe. Completing the field or adding the XML does not automatically raise the class.

If multiplicity is weak or fails, the class is C unless other conditions require D. Reaching A also requires satisfying the rules for significance, costs, out-of-sample evidence, data quality and the benchmark. The linked method explains the combined rules. The class summarizes the supplied evidence and its limits; it does not describe future results.

Start with your figures and keep the files

Bring the Sharpe, duration and known attempts to the figure reader; leave unknown figures blank. Explore the search in the calculator and keep the HTML, complete XML and declarations for the report. The reader and calculator need no account. With an account, the first full report is free.

FAQ

Does the XML measure every aspect of overfitting?

No. It counts included passes, but its summaries do not contain aligned series for every variant. Computing PBO requires a matrix of variant returns; the XML does not replace it. Nor can it reconstruct searches you did not supply.

Related

Put it to the test

The luck calculator is free and needs no sign-up. With an account, your first full report is free too.