Five Backtest Mistakes That Wipe Out Real Money
The expensive mistakes are the ones that make a backtest look better than reality. Here are the worst offenders.
A backtest that lies to you is worse than no backtest. These five mistakes are the ones that most reliably turn promising paper strategies into losing live ones.
Lookahead bias
Using information at time T that you would not actually have known until T+1. The classic case: computing today's signal from today's closing price and assuming you could trade on the close. In reality you needed the bar to finish before you knew the signal — by then the close was gone. Fix it by lagging signals by one bar.
Survivorship bias
Backtesting on a universe that excludes companies that went bankrupt or got delisted. Your strategy looks great because the dataset already filtered the losers. Use point-in-time universes whenever possible.
Ignoring costs
A strategy that wins by 5 basis points per trade is not a strategy after commissions and slippage. Always model realistic costs and assume your fills are slightly worse than the mid.
Overfitting
Tuning parameters until the backtest looks perfect. The strategy that wins on perfect parameters in-sample almost always loses out-of-sample. Reserve a holdout period and only test on it once.
Confusing backtest fills with live fills
You can backtest a strategy that trades 200 shares of a thinly traded stock at exactly the closing price every day. You cannot live-trade it. Be honest about the size your strategy can realistically deploy.