OmniVest Forum OmniVest Forum
forums calendars search
today this week
 
register logon control panel Forum Rules
You are currently browsing as a guest.
You should logon to access more features
A Self-Moderated Community - ALL MEMBERS, PLEASE READ!
Vote for Members who contribute the most to your trading, and help us moderate content within the Forums.


  Current location        Thread information  
OmniVest User Forums
General Discussion Topics
Evaluating Trading Strategies
Last Activity 4/15/2019 7:43 PM
15 replies, 1401 viewings

Jump to page : 1
Now viewing page 1 [25 messages per page]
 
back
Printer friendly version

^ Top
Jeff Mishlove

Member
25
Posts: 31

Joined: 10/11/2012
Location: Las Vegas, NV

User Profile
 
Subject : Evaluating Trading Strategies
Posted : 2/16/2018 4:24 PM
Post #37047

After watching Ed's revised version of the first ATM webinar, I arranged to acquire all of the available OT plugins and strategies and, then, evaluate them according to the protocol recommended by Ed at 34 minutes into the 55 min. webinar. For those who might be interested, I have placed all of the results onto a single .xls spreadsheet (attached).

The spreadsheet shows the strategies organized alphabetically. In addition to the raw data from OT, I have added some columns of my own making.

Columns B, C, and D, indicate the strategies selected by Mark Holstius for the three different market conditions. Columns E, F, and G show the strategies selected by Nirvana for the Universal Method.

Columns T, U, and V are based on three formulas from the forward testing data that I have chosen to help rank the strategies. This should be clear from the column descriptors.

Column W is highlighted in red for those strategies that also had unprofitable short trades. It is highlighted in green when the short trades were profitable. And, for all of the strategies with short trades, Column X is highlighted in blue indicating the statistics for the long-only trades of those strategies.

I used Columns P, T, U, and V for ranking the strategies. Using the Excel sort function, I was able to highlight in yellow the top strategies for each of these columns. For each of the 27 strategies that were highlighted in yellow in three of the four ranking columns, I then highlighted the name of that strategy in yellow.

I see that the strategies selected by Mark Holstius and by Nirvana do not overlap entirely with the 27 strategies selected by my rankings. So, there remains much to consider.

I still wish to go through all of these strategies to make some sort of preliminary judgments as to which will work best under various market conditions.

Jeff
Attached file : Summary Data Spreadsheet.xls (84KB - 362 downloads)

^ Top
Mark Holstius

Elite
50010010025
Posts: 744

Joined: 10/11/2012
Location: Sleepy Hollow, IL

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/16/2018 5:22 PM
Post #37048 - In reply to #37047

Outstanding, Jeff!

Thanks for taking the time to do that, and especially for sharing it with all of us.

I just downloaded your spreadsheet and plan to spend time with it over the weekend.

I'm sure ATM will help us improve a great number of things this coming year.

Mark


^ Top
Mark Holstius

Elite
50010010025
Posts: 744

Joined: 10/11/2012
Location: Sleepy Hollow, IL

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/16/2018 5:26 PM
Post #37049 - In reply to #37047

One more thing...

Would you mind sharing the date ranges you used for the BT and FT?

Mark
^ Top
Jim Dean

Elite
10002525
Posts: 1059

Joined: 10/11/2012
Location: L'ville, GA

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/16/2018 5:49 PM
Post #37050 - In reply to #37049

Yes, thanks.

Also, please share the allocation rule and the focus list info.
^ Top
Jeff Mishlove

Member
25
Posts: 31

Joined: 10/11/2012
Location: Las Vegas, NV

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/16/2018 7:10 PM
Post #37051 - In reply to #37050

I believe that all of the settings were the ones pre-programmed into the ATM profile. The backtest was 1/1/1999 to 12/31/1999. The forward was from the Backtest end date until yesterday. The focus list consisted of 186 symbols. I presume that these are from the S&P and NASDAQ 100 combined.

To my knowledge, this test did not use any of the ATM allocations. The report was generated, according to Ed's instructions, by running the to do list and then viewing the performance summary from the File > Print Reports menu. So, it was simply an opportunity to view the strategy performance over a lot of data.

I ran the process eight different times, with about 20 different strategies checked off each time (because running all of the strategies at once caused overflow problems with my system).

Jeff
^ Top
gbarber

Veteran
100100252525
Posts: 282

Joined: 12/30/2012
Location: Pearland, TX

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/16/2018 9:51 PM
Post #37052 - In reply to #37051

Another thank you for compiling all this info. It will be very helpful to all of us possibly including Nirvana if they haven't done this already.

What ATM method did you use?
^ Top
Jeff Mishlove

Member
25
Posts: 31

Joined: 10/11/2012
Location: Las Vegas, NV

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/17/2018 9:58 AM
Post #37053 - In reply to #37052

For the evaluation statistics that you see in the spreadsheet, no ATM methods at all were employed (to my knowledge). This is simply the OmniTrader report as described by Ed Downs in the first ATM webinar. As I understand it, that analysis occurs prior to the application of ATM or ATS methodology,

Jeff
^ Top
gbarber

Veteran
100100252525
Posts: 282

Joined: 12/30/2012
Location: Pearland, TX

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/17/2018 12:56 PM
Post #37054 - In reply to #37053

yes you are right. My error. I was thinking wrong. Thanks again for your work.
^ Top
Tom Helget

Member
25
Posts: 42

Joined: 10/11/2012
Location: Baldwinsville, NY

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/17/2018 3:35 PM
Post #37055 - In reply to #37054

Jeff:

SUPER JOB!

Most appreciated.

Tom Helget
^ Top
Tim Lambie

Member

Posts: 6

Joined: 10/11/2012

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/18/2018 1:52 PM
Post #37058 - In reply to #37053

Hi Jeff,

This is awesome, thank you.

I tried this as well before seeing your note and with a pretty beefy setup (5820k, 16GB RAM and SSD drive) my system seemed to give up around 10 strategies. I am curious to know if you ran the whole list without issues or, if you parsed/batched them in some way to get it done and roughly how long did the run take? My other tests have run fine until this elephant.

Cheers, Tim
^ Top
Jeff Mishlove

Member
25
Posts: 31

Joined: 10/11/2012
Location: Las Vegas, NV

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/19/2018 9:08 PM
Post #37066 - In reply to #37058

Hi Tim,

I actually tried to run all the strategies at one time -- and that overloaded my system. After that, I broke it down into 8 batches of about 20 strategies each. The whole process seemed to take about half a day.

Jeff
^ Top
SteveJ

Veteran
100
Posts: 105

Joined: 10/11/2012
Location: UK

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/26/2018 12:55 PM
Post #37087 - In reply to #37066

I'll be very grateful for some help with the "Optimize selection" feature under the ATM Strategies tab.

I thought I'd take advantage of Jeff's work (thanks Jeff) and run a number of his top strategies through the selection process for a bear market state. I included in the run strategies unlikely to be useful in the final result (ie long trending strats in a bear market state) - I wanted to see what happened!

After about 6 hours the run ended but, unexpectedly, all the strategies I selected for the run are in the list of strategies returned.

I'm probably doing something wrong or have misunderstood. Can someone help by explaining to me how to use the simulation statistics to establish which are the best / worst strategies form the run?

Steve
^ Top
Tom Helget

Member
25
Posts: 42

Joined: 10/11/2012
Location: Baldwinsville, NY

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/26/2018 3:12 PM
Post #37088 - In reply to #37087

SteveJ:

You need to check the Optimize check box for the Bear Market state only after also checking off above in their respective boxes the check boxes for the Strategies you want to consider for the Bear Market State. Be careful here because after you have Run the Analysis only the Strategies that are the winners will subsequently be checked. You can also verify this in the Simulation Statistics column at the right of the Portfolio Simulation chart by scrolling down to the Bear Market state data. Indeed, sometimes all the Strategies you selected will be "returned" in the optimization.

Note that if you now decide to evaluate, say, the Raging Bull Market state, you should un-check the Optimize check box for the Bear Market state and check the Optimize check box for the Raging Bull Market state so that in your next run only the Raging Bull Market situation is evaluated to save time (and, yes, I made that little error myself a couple of times - more or less re-doing what had already been done).

I hope that helps!

Tom Helget

[Edited by Tom Helget on 2/26/2018 4:05 PM]

^ Top
SteveJ

Veteran
100
Posts: 105

Joined: 10/11/2012
Location: UK

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/26/2018 4:03 PM
Post #37089 - In reply to #37088

Thank you Tom, that does help.
Steve
^ Top
Tim Lambie

Member

Posts: 6

Joined: 10/11/2012

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/26/2018 4:17 PM
Post #37090 - In reply to #37066

Hi Jeff,

Thanks for the details. That correlates with my experience. It appears we need some automation for the automation ;-) as big strategy and optimization runs seem to gum up the works. I don't believe this is purely about processor speed and available memory. We should all try to post where we find the capacity limits in this area to inform each other and Nirvana, avoiding some time wasting replication and perhaps getting Nirvana to address the scale/architecture issues we find in testing.

Thanks, Tim
^ Top
Tom Helget

Member
25
Posts: 42

Joined: 10/11/2012
Location: Baldwinsville, NY

User Profile
 
Subject : RE: Evaluating Trading Strategies
Posted : 2/26/2018 4:24 PM
Post #37091 - In reply to #37090

Tim:

I thoroughly agree with you.

One of my pet peeves here is that once the Run Analysis button is "pushed" you really don't know what you are letting yourself in for. Depending on what I am optimizing the run may only take an hour or two but it could easily go overnight or even several days. It certainly would be nice if some time "estimate" of the run could be given and even "paused" (like with Strategy Wizard).

Wishin' and a Hopin'

Tom Helget
Jump to page : 1
Now viewing page 1 [25 messages per page]
back

Legend    Action      Notification  
Administrator
Forum Moderator
Registered User
Unregistered User
E-Mail this thread to a friend
(un)/Freeze thread |
Toggle e-mail notification


Nirvana Systems
For any problems or issues please contact our Webmaster at webmaster@nirvsys.com.