Is your best setting a plateau, or a spike?
A backtest measures one combination of settings. Change the EMA period by a step, or the RSI level by three points, and you are looking at a different strategy. A parameter search runs every combination at once and draws the answer as a landscape, so you can see whether your settings sit on high ground or on a needle.
Drag the map to turn it, hover a cell to read it, and move the slider to step through the third setting. Switch to Heatmap to look straight down on the same grid.
Why a single best result is not enough
Search enough combinations and one of them will look exceptional by chance. The best cell of any grid is biased upward, because it is the highest of many noisy measurements. What matters is the ground around it. A result that holds a step either side sits on a plateau. One that collapses a step either side is a spike, and a spike is usually fitted to the particular history it was measured on.
So the map leads with the best plateau: the cell whose neighbourhood, the cell and every measured neighbour one step away, scores best on average. The best single cell is marked as well, but it is not the headline. When the two markers sit together, the best result is on high ground. When they are apart, you are looking at a spike.
How it runs
- Pick a strategy, instrument and window, with the same costs, exits and history depth a backtest would use. Or start from a finished backtest or Refine result with Check these settings, and the grid is centred on that result's own settings.
- Choose one to three settings, such as an EMA period, an RSI level or the stop and target distances, and two to nine values for each.
- See the size and the price before you run. Up to 343 combinations in one search. The credit cost updates as you change the ranges.
- Every combination is backtested and the map fills in as cells finish.
Reading the map
One setting draws a line across its values, with the trade count under each point. Two draw a heatmap, with a 3D view that turns the measure into height, so a needle and a hill are told apart at a glance. Three add a slider that steps through the third setting's values, the way the map above does.
Choose what height means: return, profit factor, expectancy, expectancy in R, maximum drawdown, win rate, or Refine's score. Hover any cell for its settings and how its neighbours compare, then open it as an ordinary backtest to see every trade. Cells with too few trades to mean anything are greyed out rather than coloured. A small sample is not a result.
The verdicts are computed, not written
- Your settings
- 7 of 8 neighbours within ±25%
- Best cell
- 2 of 8 neighbours within ±25%
- Plateau toggle
- colours every cell by its neighbourhood
Each line counts how the measure behaves one step either side, on this history. A best cell whose measure falls by more than half one step away is called what it is: a spike. And when the best plateau sits on the edge of your grid, one click starts a new search centred on it.
What this does not do
It does not predict anything. Neighbouring cells share most of their trades, so a plateau is necessary but not sufficient. Refine's check on history the variations were not chosen on is the stronger test. The settings you did not include stay fixed, so a hill on a two-setting map can be a ridge in more dimensions. And a spike narrower than one step of the grid is invisible to it. The map describes how one strategy behaved on one stretch of history, nothing more.
Questions
What is a parameter search?
A test that keeps your strategy's conditions and runs every combination of up to three of its settings — an EMA period, an RSI level, a stop distance — as its own backtest over the same history. The result is a map of how the measure you chose changes as the settings change, rather than a single number for one setup.
How is it different from Refine?
Refine changes the strategy itself: it breeds variations, swaps conditions and checks the survivors on history they were not chosen on. A parameter search changes nothing but the numbers you pick, and measures every cell of the grid rather than a sample. Use it to see whether the settings you already have sit somewhere stable.
What is a plateau, and why does the map headline it instead of the best cell?
A plateau is a cell whose neighbours — one step either way on each setting — measured much like it. The best single cell in any grid is biased upward, because it is the maximum of many noisy measurements. A result that holds across neighbouring settings is less likely to be fitted to one particular stretch of history, so the map leads with the best plateau and marks the best cell beside it.
How big can a search be, and what does it cost?
Up to three settings with two to nine values each, and up to 343 combinations in one search (seven by seven by seven). Each combination is priced in testing credits, and the total is shown before you run it, updating as you change the ranges.
Where do I run one?
In the SixOne8 web app, from the testing hub or with "Check these settings" on a finished backtest or Refine result. The map is built for a wide screen, so parameter search lives on the web app.