OmniTrader Forum OmniTrader Forum
forums calendars search
today this week
 
register logon control panel Forum Rules
You are currently browsing as a guest.
You should logon to access more features
A Self-Moderated Community - ALL MEMBERS, PLEASE READ!
Vote for Members who contribute the most to your trading, and help us moderate content within the Forums.


  Current location        Thread information  
OmniTrader 2017 Upgrade Forums
Trade Plans
MECH VS OPTIMIZED
Last Activity 7/17/2018 10:46 AM
5 replies, 776 viewings

Jump to page : 1
Now viewing page 1 [25 messages per page]
 
back reply
Printer friendly version

^ Top
Stumpy

Veteran
100100
Posts: 208

Joined: 5/25/2004
Location: Australia

User Profile
 
Subject : MECH VS OPTIMIZED
Posted : 1/24/2017 10:11 AM
Post #38735

if we do a portfolio sim on an optimized strategy can we be confident of the results or have the results been compromised by curve fitting ????
^ Top
Jim Dean

Idol
200050010010010025
Posts: 2831

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: MECH VS OPTIMIZED
Posted : 1/24/2017 10:25 AM
Post #38736 - In reply to #38735

There is no definitive answer to that. There are zillions of ways of optimizing. All of them do *some* "curve-fitting" ... but not necessarily to a degree where it's worrisome. Curve-fitting is a part of every strategy ever created by anyone ... just using "standard parameters" for things like a MACD is in a sense "curve fit", since those settings are arbitrary.

The way that you determine if you have a "nasty" curve fit is by careful regimentation of development using one date range &/or symbol-list, then lock down the "tuned" values (ie turn off optimization or stop using Strat Wiz, etc), and test the resulting Strat on a different symbol list &/or date-range.

Even that method is not perfect, since usually a developer will see the results of that test, and then go back and tweak things again. As soon as you've done that, the second list+dates are "contaminated" and you need to do your confirmation test on yet a different symbol-list &/or date-range.

In the real world, this procedure is VERY rarely followed strictly and properly ... it just takes too much effort and planning and regimentation. I doubt that N does it truly strictly with their development. In fact I don't know of any vendor that guarantees they've followed those rules throughout their development cycle.

The upshot of not following the proper regimine is that the backtests will look really enticing but the actual results moving forward will usually be significantly worse. It's probably the biggest single problem with the development of trading systems.

[Edited by Jim Dean on 1/24/2017 10:33 AM]

^ Top
John J

Veteran
10010025
Posts: 233

Joined: 8/3/2010
Location: Leduc, AB

User Profile
 
Subject : RE: MECH VS OPTIMIZED
Posted : 1/24/2017 4:18 PM
Post #38742 - In reply to #38735

It's perfectly Ok to optimize if it's done in a walk-forward manner. As far as I'm concerned, it doesn't make any sense to base a strategy on data that is several years old, especially if the market was behaving very differently at the time...
^ Top
Stumpy

Veteran
100100
Posts: 208

Joined: 5/25/2004
Location: Australia

User Profile
 
Subject : RE: MECH VS OPTIMIZED
Posted : 1/24/2017 9:49 PM
Post #38744 - In reply to #38735

Thanks Guys,
If I tick the non optimized box in strategies I am assuming the ones that are not greyed out are non optimized .
Next question the strategy I am working with tests very well in portfolio sim as well as reports mode however it uses reversing signal as an exit.
I have tried all types of trailing stops but all so far have significantly reduced the performance.
In your experience is it ok to trade a system with a reversing signal as the exit or is that a bit risky?
My preference would be to have a plotted stop so I could enter a stop in the market
^ Top
Jim Dean

Idol
200050010010010025
Posts: 2831

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: MECH VS OPTIMIZED
Posted : 1/25/2017 3:58 AM
Post #38745 - In reply to #38744

If the Strategy box is not red then there is no "active" optimization going on.

My prior remarks about curve-fitting are valid. Even walk-forward testing doesn't sidestep dangers of *over* fitting. If you were to:
a. do a walkforward test on a fixed sym list & date range, and view its results
b. manually modify the strat in a small or major way to improve those results
c. repeat a & b 5-10x ... then ...
Your resulting strategy most likely would be curve-fit sufficiently so that in real trading the results would be significantly worse. *Whole books* have been written on this subject. The bottom line is: don't over-tweak because of statistical results ... ANY changes you make to a strat should be for a known, logical reason that you can "see" the reason for making the change on a chart.

Using a Reversing Signals stop (only via Orders Block or via OmniPilot) can be a viable method for some kinds of Systems and stocks whose price-action personalities are a good match to how the strategy "thinks". Some Systems (such as RTM) should NOT be used as their own exits.

The important thing is to UNDERSTAND the logic and rationale that all the components of your strat are using, and "fit" them together *sensibly*. Some folks just like to sort of randomly match up entry and exit rules ... and doing so might yield a backtest that looks good ... but that random-matchup process, itself, is a form of curve-fitting to be avoided.

[Edited by Jim Dean on 1/25/2017 4:54 AM]

^ Top
Stumpy

Veteran
100100
Posts: 208

Joined: 5/25/2004
Location: Australia

User Profile
 
Subject : RE: MECH VS OPTIMIZED
Posted : 1/25/2017 4:52 AM
Post #38746 - In reply to #38735

Thank you Jim
Jump to page : 1
Now viewing page 1 [25 messages per page]
back reply

Legend    Action      Notification  
Administrator
Forum Moderator
Registered User
Unregistered User
E-Mail this thread to a friend
Toggle e-mail notification


Nirvana Systems
For any problems or issues please contact our Webmaster at webmaster@nirvsys.com.