Backtesting is the general method for seeing how well a strategy or model would have done after the fact. It assesses the viability of a trading strategy by discovering how it would play out using historical data. If backtesting works, traders and analysts may have the confidence to employ it going forward.
Backtesting allows a trader to simulate a trading strategy using historical data to generate results and analyze risk and profitability before risking any actual capital.
A well-conducted backtest that yields positive results assures traders that the strategy is fundamentally sound and is likely to yield profits when implemented in reality. In contrast, a well-conducted backtest that yields suboptimal results will prompt traders to alter or reject the strategy.
Particularly complicated trading strategies, such as strategies implemented by automated trading systems, rely heavily on backtesting to prove their worth, as they are too arcane to evaluate otherwise. As long as a trading idea can be quantified, it can be backtested.
For backtesting to provide meaningful results, traders must develop their strategies and test them in good faith, avoiding bias as much as possible. That means the strategy should be developed without relying on the data used in backtesting.
That’s harder than it seems. Traders generally build strategies based on historical data. They must be strict about testing with different data sets from those they train their models on. Otherwise, the backtest will produce glowing results that mean nothing.
Similarly, traders must avoid data dredging, in which they test a wide range of hypothetical strategies against the same set of data, which will also produce successes that fail in real-time markets because there are many invalid strategies that would beat the market over a specific time period by chance.
One way to compensate for the tendency to data dredge or cherry-pick is to use a strategy that succeeds in the relevant, or in-sample, time period and backtest it with data from a different, or out-of-sample, time period. If in-sample and out-of-sample backtests yield similar results, then they are more likely to be proved valid.