OmniTrader Upgrade Forums
Trade Plans
No Optimization forward test
8 replies, 1628 viewings
|
OmniTrader Upgrade Forums
Trade Plans
No Optimization forward test
|
Last Activity 1/2/2025 4:50 PM 8 replies, 1628 viewings |
|
|
|
| Printer friendly version |
LSJ Legend  Posts: 515 Joined: 8/17/2006 Location: Citrus Springs, FL  User Profile |
In trying to understand the process What is the forward test using if it follows a backtest with no optimization? With a 1000 bar BT and a 1000 bar FT, e.g., the results of the FT are significantly different, and always lower performance, than the backtest. If no optimization has been done what exactly is the forward test basing trades on? | ||
| Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
I've always presumed (and depended on!) non-optimized Analysis (and SW) runs to treat BT and FT just as two "bins" of equal significance. | ||
| LSJ Legend Posts: 515 Joined: 8/17/2006 Location: Citrus Springs, FL User Profile |
I have attached files from one of my many tests. This is a daily 1000 bar BT and FT with no optimization and the relative results of BT and FT are typical of what I get. This is what prompts me to ask what is happening in a FT that is followed by a BT with no optimization. Thanks for any light you can shed.  Attached file : BTFT.jpg (355KB - 372 downloads) Attached file : Settings.jpg (140KB - 296 downloads) Attached file : Strats.jpg (107KB - 315 downloads) | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Market during latest 1000 bars (4 yrs) could be less amenable to those Strats than in the prior 1000 bars / 4 yrs, thus showing lower performance. To check, try using the date input instead, and move everything back 4yrs, so the old BT pd is now the new FT pd, and the new BT pd is 4 yrs earlier. Or ... If those Strats had been "tuned" a bit too much during development, and if the tuning was done with an emphasis on bars a few years ago and before (not at all uncommon), then the tuning process itself will have introduced "invisible optimization" to the results. The only way to be sure is to test the strats using both BT and FT ranges that are AFTER the strats were first released. [Edited by Jim Dean on 12/4/2017 4:02 PM] | ||
| LSJ Legend Posts: 515 Joined: 8/17/2006 Location: Citrus Springs, FL User Profile |
I will split up some of the tests to compare periods. | ||
| LSJ Legend Posts: 515 Joined: 8/17/2006 Location: Citrus Springs, FL User Profile |
No optimization was used and all settings are identical. The number of bars in the backtest and forward test are the same(1000). Comparing the results they are not the same, and not by rounding but by significant enough values that there seems to be something else going on. It appears OT is processing the tests differently and I am trying to find out why. This apparent anomaly shakes my confidence in these tests until I understand why. Any explanation would be appreciated. | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
| ||
| Barry Cohen Sage    Posts: 6338 Joined: 1/19/2004 User Profile |
Looking closer at your screenshots, I see that the individual strategies are actually equivalent stats, & it's only the "All Strategies" stats that are suspect. This points to the Strategy Voting settings you are using (button at the bottom of the Strategies list). I would guess that you're using a ranking method or a filter & that's what is throwing it off. Try disabling everything in the Performance tab under Strategy Voting. | ||
| LSJ Legend Posts: 515 Joined: 8/17/2006 Location: Citrus Springs, FL User Profile |
However, if all settings are the same and the data is the same, what is done differently in a FT with no backtest optimization vs. a BT with no optimization over the same data? I re-ran without any strategy voting and the results are very close now. There is a little over 1% difference in the number of trades and HR% is very nearly the same so that is a completely acceptable result. As a user and not a programmer I don't need to know why the strategy voting is different and am comfortable with the fact that is just is. Thanks Attached file : BTFT12_6.jpg (354KB - 351 downloads) Attached file : BT12_6.jpg (329KB - 355 downloads) |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
