Jim Dean![]() Sage ![]() ![]() Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA ![]() | And, the final installment re guidelines for testing, tweaking and evaluating your Strategy, in relation to: Selection of Time Periods ... for backtesting vs forward-testing: First, terminology: by "Time Periods" I'm not referring to daily vs weekly vs 5min or 30min bars, but rather the historical chunk of sequential bars that you use to run the tweaks and tests over, for your given Focus List and Strategy. "Back-testing" refers to the tests done while you are tweaking and optimizing things. "Forward-testing" refers to proof-demo tests that USE the final adjustments. These should be DIFFERENT, NON-OVERLAPPING periods of time. It is not necessary that they be chronologically sequential, nor that the back period is chronologically prior to the forward period. The labels are conveniences. The point is that they are DIFFERENT data-sets of OHLCV bars. Just the definitions alone should tell you the key point. The same kind of rationale used for separating the Focus Lists into subsets for tweaking vs proof-testing applies here. Why, you might ask, is this necessary, if you are ALSO using different symbol-lists? Isn't that sufficient? Can't you use the SAME time-period for back and forward testing (tweaking vs proving), if the FL symbols are different? NO. (well - yes you can, but it's not wise to) Reason ... even though you are using different symbol-sets, the MARKET FORCES during that timeframe tend to PUSH ALL THE SYMBOLS in the same general way ... many guru's argue that 30-50% of the average symbol's price-action is due to the influence of the overall market at that time. So ... you need to use DIFFERENT TIME-PERIODS to isolate one group from another, when doing the "tweaking" and "proving" periods. But that raises another issue ... if you tweak during a "bull" market period, and prove using a "bear" market period, then a Strategy that does well in Bull markets might appear to be over-fitted ... that is, the results of the tweak run would be a lot better than those of the prove-run ... but it would NOT have been due to over-fitting ... rather, it would have to do with differing market conditions. This is an important concern. In fact, it should carry over into your Strategy design as well! If your Strategy is focused mainly on Bull-market conditions, then you should HAVE A FILTER in place that only allows trades during those markets. For instance: GetClose($DJI) > SMA(GetClose($DJI),100) might be (very simplistic) a "bull market condition" filter. If you use this filter, then you'll avoid having the tweak and prove runs mismatched. There's another other nifty thing you might consider here ... AUTOMATIC selection of the time-periods to use for the testing and proving. The normal way is to specify particular periods for back and forward testing, via the Strategy or PS setups. Those inputs are not very versatile. However, using a trick similar to the one I showed you for creating diverse Focus Lists, you can write OmniScript "time-period filters" for your tweak and prove runs ... Let's say that you want the tweak tests to only utilize trades that start within the 2004-2006 calendar years. You can isolate these with a Filter: BarYear() >= 2004 and BarYear() <= 2006 Of course, you can fill in whatever year(s) you want. If you want to get really focused, you can build in BarMonth() rules as well. Or, if you're using intraday data to test 30min bars, for instance, you might need to define a range of BarDayOfMonth() rules. If you're unclear on how this would be utilized, please CAREFULLY re-read the prior post. |