Backtesting vs Forward Testing: How to Validate a Trading Strategy

A trading strategy may produce an excellent equity curve in the past and yet be very different when faced with market data that did not exist in the time of its development.

This difference between past and future results can expose weaknesses in a trading strategy. 

Backtesting is a strategy's performance on past market data. Forward testing is a strategy's performance on new market data. None of them proves the profitability of a strategy in the future.

In this guide, we will discuss the differences between backtesting and forward testing, the information available at each step, the reasons for the differences, and how traders can use both in the validation of a trading strategy.

Backtesting vs Forward Testing: What Is the Difference?

Backtesting implies applying the defined criteria to historical price data to see how the strategy would have worked out. Forward testing means employing these very criteria without changing anything in response to newly arriving market data.

The first one is for research; the second is a reality check.

Backtesting allows the trader to test years of market behavior in a short period of time. The primary importance of backtesting is research: did this strategy show a historical edge under clearly defined assumptions?

Forward testing is what most people are looking for when they type "what is forward testing in trading". Simply put, it is the process of testing the completed strategy prospectively, often using live or continuously updated data, and documenting the results.

Forward testing checks if the behavior, signals and assumptions about the execution are still similar to those shown by the historical test.

There is one thing that is more important than anything else. Forward testing is not a substitute for backtesting, and backtesting is not a substitute for forward testing. Each answers a different question, so a good process needs both.

What Backtesting Can and Cannot Tell You

There are many worthwhile questions which a well-designed backtest can answer:

• Did the rules lead to positive or negative expectancy, an overall expected outcome per trade?
• How frequently did the strategy trade?
• What kinds of drawdowns were there during the course of trading?
• Is the performance based only on one period, one type of asset or market situation?
• How were the win rate, average win and loss, profit factor?
• Did the strategy hold up across different historical market regimes?

All these facts prove the usefulness of a backtesting trading strategy workflow. A trader neglecting backtesting is simply guessing. However, a trader completing backtesting at least knows whether further research may be useful.

The drawbacks are no less important.  A backtest cannot establish that:

• the same edge will remain for future markets.
• live fills are going to be the same as simulated fills.
• spread, slippage or latency will match the assumption.
• a trader will execute discretionary rules exactly as assumed.
• the trader will follow discretionary rules in live trading exactly as assumed.

Historical results can be misleading for several reasons. 

For instance, overfitting describes an approach where rules become too fitted to the historical data. 

Look-ahead bias implies that a calculation is done using data that wasn’t available at the time of the decision-making process.  

Unrealistic transaction costs, as well as data quality problems, can also affect the results in the wrong way. 

When a trader decides to backtest a trading strategy, the main thing they should seek for is evidence not confirmation. A successful result implies that strategy requires some more validation. This doesn't mean, however, that it works.

What Forward Testing Can and Cannot Tell You

The forward testing trading strategy involves fixed rules and parameters. Once the test starts, no changes can take place. The trading strategy is then observed while receiving the new market data.

This enables a trader to determine such factors that a backtest cannot entirely reveal:

• Presence of predicted signals in real time.
• Whether entries and exits have been correctly implemented.
• The difference between assumed and observed spreads and slippage.
• Trade frequency.
• Whether any issue exists with the trading platform, data-feed or automation.
• whether a discretionary trader follows the rules.
• how the strategy behaves in the current market regime.

Forward testing and paper trading

The concepts are frequently confused but are not the same. Forward testing is the validation process. Paper trading is one of the environments in which a forward test can be performed, demo trading on a broker's platform being another.

Performing a few demo trades without any set of rules is not a forward test. A test is considered such only if the rules have been defined first and the results are analyzed against them.

The limitations of forward testing

Forward testing makes the test more realistic but still has its own limitations:

• It takes time to gather enough data.
• A short observation period may reflect only one market regime.
• Demo fills may differ from live fills.
• Paper trading cannot simulate the stress of real money.
• A few trades that win or lose are not much evidence.

Forward testing helps to bring the strategy closer to reality. It still cannot ensure its performance in live trading.

Why a Strategy Can Pass Backtesting but Fail Forward Testing

Poor performance during forward testing does not necessarily mean one reason behind it. There are several causes, and it is up to the trader to identify which reasons apply.

1. Overfitting or Curve fitting.

The trading rules or parameters might have captured noise of the development period rather than capturing repeatable market behavior. When the new data comes, the noise disappears.

2. Look ahead bias or Data leakage.

The historical calculations might be based on the data that is not actually available at the time of the decision-making. The backtest was too smart for any trader.

3. Unrealistic execution assumptions. 

Spread, commission, slippage, latency, partial or missed fill and order execution timing are some of the factors affecting the results. It is particularly important for short term strategies, where small execution differences can materially change expectancy.

4. Changes in market regimes.

A strategy designed under trending, volatile or highly liquid circumstances might find itself facing a completely different environment in the future. While a bad regime doesn’t automatically nullify a strategy, this fact should be taken into account.

5. Implementation differences. 

The rules applied in the forward test may be somewhat different from the rules used in the historical test. This is true for both manual and automated trading strategies.

6. Trading style.

In discretionary trading, indecisiveness, missed opportunities, early exits or quiet modifications of rules cause discrepancies between theory and practice.

7. A small forward-test sample. 

A small number of trades can create a substantial short-term difference through normal outcome variation alone.

This is the reason why backtest vs forward test comparison needs to include diagnosis, not panic. There is no general threshold of an acceptable loss, because the proper expectations depend on the strategy.

How to Use Backtesting and Forward Testing Together

The proper validation process starts with less expensive and less time-consuming evidence and progresses to increasingly realistic evidence. Every step eliminates those strategies that do not deserve the next step.

Step 1: Define and lock the strategy rules

Record the entry rules, exit rules, position size, stop-loss and take-profit rules, list of eligible instruments, trading sessions, filters and any other relevant execution assumptions. The purpose is to prevent the strategy from changing each time it fails.

Step 2: Test the strategy using the historical data

Determine whether the strategy demonstrates sufficient consistency to be worth further testing. Make sure to account for costs and different market conditions.

Step 3: Out-of-sample testing using untouched data for validation

Out-of-sample testing uses untouched historical data that has been kept separate from strategy development. It is still a historical test as the data already exists. 

However, it is a question whether the strategy survives the untouched historical data that did not shape its rules.

Step 4: Use walk-forward testing whenever possible

A strategy that has its parameters periodically optimized or recalibrated can undergo a walk-forward testing. It checks whether the recalibration process stays consistent across sequential windows. 

Walk-forward testing still operates on historical data and cannot be considered a true prospective forward test.

Step 5: Perform the forward test with the frozen rulebook

Test the given strategy with frozen rules on the new market data. Don’t optimize the parameters after several unsuccessful trades. If the rules change materially, the trader has invented another version of the strategy, and needs fresh validation from the beginning.

Step 6: Compare historical and forward results

Compare equivalent metrics. The important question is not whether both tests were profitable, but how similar the approach was.

Step 7: Diagnose differences before increasing risk

If the test results diverge considerably, determine whether the cause is execution, implementation, market situation, statistics, or original strategy hypothesis.

Small live deployment is mentioned above just for clarification only. It’s the last and most realistic phase and, as such, includes real risks, therefore it comes after the earlier evidence is in place.

How Do You Know Whether Forward Testing Confirms the Backtest?

Often traders ask about how many forward tests are enough? There is no single answer in numbers. 

A fixed count of 30 trades, a three-month window, a 20% tolerance or a single profit-factor threshold ignores the fact that the right answer depends on trade frequency, holding period, market and the strategy's return distribution.

Better is to define your evaluation criteria prior to forward testing starts. Decide in advance what to measure and what would count as consistent, inconclusive or concerning.

Metrics to compare

Compare the same metrics in both tests to have equal grounds to evaluate trading strategy performance.

How long should forward testing continue?

There are two factors to consider here. The first is the number of observations. Strategies generating multiple trades daily will accumulate observations faster compared to those generating just two setups per month. 

The second is the diversity of market conditions. Even a very large number of trades conducted in very favourable conditions will only provide you with a partial picture. 

The debate in the trader circles usually revolves around whether to consider calendar time, number of trades, or number of regimes, but the right answer would be enough observations in relevant conditions, rather than a fixed number of weeks.

This is also where forward testing vs backtesting differs most in practice: one produces samples quickly, while the other earns them slowly.

If results are worse

There are three reasonable responses:

1. Continue monitoring if there is not enough data yet.
2. Investigate if execution, implementation or market conditions could explain the difference.
3. Re-examine the strategy if the loss continues on sufficient data and cannot be explained by execution or variance.

What is not a proper course of action is changing parameters to make the forward test profitable. This would just create another overfit strategy.

Conclusion

The strategy validation is the process of gathering independent data. Backtesting can demonstrate that a strategy is worth investigating further. 

Forward testing checks if that behavior is sustainable when the strategy is unable to benefit from hindsight. Both do not eliminate uncertainty.

A trader preparing for a prop firm evaluation or a funded environment needs to come with rules that have already been tested, researched and implemented consistently. 

At Audacity Capital, proper execution and risk management are important, and getting funding does not mean that a strategy works. Testing does not guarantee an edge, but helps traders identify flawed assumptions before risking money.

This article is for informational purposes only and is not investment advice. Trading involves risk, and neither testing method guarantees future results.

Frequently Asked Questions