OmniTrader Forum OmniTrader Forum
forums calendars search
today this week
 
register logon control panel Forum Rules
You are currently browsing as a guest.
You should logon to access more features
A Self-Moderated Community - ALL MEMBERS, PLEASE READ!
Vote for Members who contribute the most to your trading, and help us moderate content within the Forums.


  Current location        Thread information  
OmniTrader 2017 Upgrade Forums
ATM
Looking back - and Walking Forward
Last Activity 9/8/2019 11:15 AM
17 replies, 919 viewings

Jump to page : 1
Now viewing page 1 [25 messages per page]
 
back reply
Printer friendly version

^ Top
mholstius

Veteran
1002525
Posts: 165

Joined: 1/13/2017

User Profile
 
Subject : Looking back - and Walking Forward
Posted : 10/15/2018 7:51 PM
Post #45586

Well, there’s good news and bad news…
This year brought us ATM, an amazing tool for automated trading - and also a challenging market to go with it.

After recovering from the black swan at the start of February, the market took another swing at us this week. I guess it’s all part of the game and happens often, but those years of up markets were nice. That said, a good system should recover after surprises, and maybe we should treat this as an opportunity(?) to test the systems we’re trading.

So, did the systems recover during the lulls between the storms? I think so - and I’m convinced we’ve made some significant strides forward this year.

I’ll compare & review the systems I’ve posted in elite during the year and introduce the results of my work on a new system that walks forward each day rather than matching strategies to historical data in a back test. To keep it simple and consistent, all of the following are results in an IRA, trading Long Only, and using no leverage.

Trading my Follow The Money HD…

It traded nicely during the 4 “normal” periods this past year while using 2% of equity, and had just gotten back to break even before the drop last week;




Those subscribers who chose to use 10% of equity fared better;




And those trading the combination of FTM HD & ATM Macro & Micro would’ve done the best;
(This combination wasn’t available until June 2018)




While these are good, I wanted something that adapted to the market on a more dynamic basis.

My 5 month walk forward project used rules that were time intensive, manually selecting what to trade at the beginning of each month and then moving forward. I improved the process immensely by adding an Expectancy block to each strategy.

Using Expectancy, EVERY DAY each strategy filters the trades it sends to ATM based on the performance of the strategy in the past.

Instead of my having to calculate a manual walk forward on a monthly basis, OT does it automatically each day. This should be more robust than a system that simply fits the historical data (the usual backtest).

Daily expectancy selection, coupled with the daily ATM process of filtering and selecting the top ranked trades, makes me feel more confident that the performance should continue in the future...

Walk Forward Long Only IRA 1 Year;




Notice that it has better performance with only 40% Invested vs the Combination chart above it with 77% Invested.

Walk Forward Long Only IRA 5 years;




Walk Forward Long Only IRA since 2003;




Some interesting DD observations…

The OT results include MTM drawdowns. I prefer to use closed trade performance, since I intend to let the system trade automatically and I’m not concerned with the whipsaw type of drawdowns that often occur with RTMs: it’s common for RTMs to enter a trade on Monday, show a DD for a few days, and then close with a profit.

The chart for the closed trade equity 2003 - present looks almost identical to the one above, but the drawdowns are based on account balances after the trades have closed, not the variations in the market each day. That’s my true concern, and those closed trade DDs are usually significantly lower than the MTM DDs (they can never be greater).

A comparison of the 3 worst MTM DDs from the chart above vs the same DDs using closed trade equity;




I don’t want to be overly concerned with momentary movements in the market. I’d rather focus on the actual bottom line results of a system.

Another way to analyze closed trade DDs is to note what they are each day and calculate how many days are spent in each DD group based on the size of the DD;




There were 3,974 trade days from 2003 - present.
The table above shows how many days were spent in each of the 1% DD groups.
1) 1,748 of them were either flat or higher than the previous day (44% of the days).
2) 1,064 days were between 0 and -1% down (26.8% of the days).
3) I’ve highlighted the -2% to -3% group (359 days / 9% of the total);
4) The last column shows that on more than 90% of days the worst DD encountered was 2%
5) I’ve also highlighted that a -6% or worse DD occurred on only about 1% of the days.

We need to recognize too that while the equity curve looks smooth and steady, there are periods where the SPY is actually performing better than the system. Obviously, the system does well over time - but we need to acknowledge that it’s not going to be ahead of the SPY all the time.

I calculated the return for both the system and the SPY each day, looking back X # of days and then totaling how many days the return of the SPY was greater than the return of the system over the lookback period;




A) When looking back 1 month, over 33% of the time the SPY had a better return.
B) When looking back 3 months, over 19% of the time the SPY had a better return.
C) When looking back 6 months, over 8% of the time the SPY had a better return.

So, I won’t get upset when the system is lagging behind the SPY at some point in the future.

I hope to upload this new system to OV (and Elite) in the near future after a few small bugs in the upload process are resolved. I’m certain that it will prove to be even more profitable when combined with other systems in OV, as it only uses an average of ~40% of equity.

In the meantime, I’m going to do take advantage of the reason for developing these automatic trading systems…

I plan to spend a lot of time the next few weeks visiting our grandchildren.

Best of luck in your trading!

Mark


[Edited by mholstius on 10/15/2018 8:30 PM]

Attached file : 01 FTM IRA 1.png (118KB - 785 downloads)
Attached file : 02 FTM IRA 2.png (123KB - 781 downloads)
Attached file : 03 Combination.png (132KB - 791 downloads)
Attached file : 04 WF 1 year.png (121KB - 777 downloads)
Attached file : 05 WF 5 Years.png (125KB - 777 downloads)
Attached file : 06 WF since 2003.png (168KB - 781 downloads)
Attached file : 07 DDs 1.png (13KB - 774 downloads)
Attached file : 07 DDs 1.png (13KB - 24 downloads)
Attached file : 08 DD Groups.png (48KB - 778 downloads)
Attached file : 09 Lookback.png (22KB - 768 downloads)

^ Top
Buffalo Bill

Legend
100100100
Posts: 305

Joined: 10/3/2006
Location: Stafford, VA

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 7:06 AM
Post #45587 - In reply to #45586

Mark

I agree 100% on MTM vs actual close DD. I wish we could have OT use the one of our choice in Port Sim

Haven't messed with the exp block - how did you configure it? Where did you get the settings used? How much improvement did it produce?

OBTW yes, ATM has taken a big DD the last couple of weeks or so on my end.

[Edited by Buffalo Bill on 10/16/2018 7:07 AM]

^ Top
jpb

Veteran
100
Posts: 122

Joined: 5/11/2005
Location: Brown Deer, WI

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 8:16 AM
Post #45588 - In reply to #45586

Mark,
I'm always impressed at how well you put together your thoughts and organize your posts! I love the idea of using the Expectancy block to filter trades before ATM does its magic. I played with the Expectancy block in the past but never took it to live trading. But your results have me intrigued.

One question as we walk through the various systems from FTM to Walk Forward. You went from 2% to 10% allocation per trade with FTM and then it looks like the combination of ATM + FTM had a blended effect of slightly more than 10%. I'll assume it was higher allocation with ATM and 10% for FTM to build upon the 10% FTM chart?

So, then for Walk Forward, was that approximately 10% allocation per trade also?

Last question, for your historical drawdown chart, what method did you use to determine daily drawdown and ending equity for the day? Surely you didn't step through 3,974 days within OT. Was it as simple as using all the closed trades (ignoring anything open) and calculating change per day?

Thanks for sharing fantastic (and time consuming) research!
^ Top
Jim Dean

Idol
2000500100100100100
Posts: 2922

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 8:36 AM
Post #45589 - In reply to #45588

Questions - probably for Barry:

IIRC, Barry posted a while ago that something was not working properly re the Analysis Backtest - Forwardtest inputs, when used its ATM evaluations. So:

1. Has that been fixed yet, and if not could you please give us more details about it so we know what to be wary of in the interim?

2. Does that mean the walk forward dynamic-backtesting algorithm is also compromised somehow?

3. Are any other backward-looking evaluations such as Expectancy or Seasonality, etc compromised by that problem?

4. If not already, will it be fixed soon?

Thanks



[Edited by Jim Dean on 10/16/2018 8:37 AM]

^ Top
Barry Cohen

Sage
5000500100
Posts: 5603

Joined: 1/19/2004

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 9:58 AM
Post #45590 - In reply to #45589

IIRC, Barry posted a while ago that something was not working properly re the Analysis Backtest - Forwardtest inputs, when used its ATM evaluations.


Apologies, but I'm drawing a blank. Do you have a link or something to this?
^ Top
Jim Dean

Idol
2000500100100100100
Posts: 2922

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 10:08 AM
Post #45591 - In reply to #45590

Nope sorry. I’m sure I saw it though. Maybe in a prerelease note? It may have been Thad or some other N staff. What I recall it saying is that the BT period was always 100% of the loaded data, regardless of the input value. Or maybe Jeff said it in a webinar?

Bottom line: do all the backtest/forwardtest input options for analysis mode and portsim mode work properly now?

[Edited by Jim Dean on 10/16/2018 10:09 AM]

^ Top
jpb

Veteran
100
Posts: 122

Joined: 5/11/2005
Location: Brown Deer, WI

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 10:30 AM
Post #45592 - In reply to #45591

I remember that too. I thought it was a post in the forum based on a discussion someone had with one of the Nirvana team. I searched the forum but couldn't find it.
^ Top
jpb

Veteran
100
Posts: 122

Joined: 5/11/2005
Location: Brown Deer, WI

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 11:14 AM
Post #45593 - In reply to #45586

By the way, anyone looking for a good thread on the Expectancy Block, I found it here. And yes, the author is... Mark Holstius.

Mark, thanks for being such a great contributor over the many years!
^ Top
Barry Cohen

Sage
5000500100
Posts: 5603

Joined: 1/19/2004

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 1:57 PM
Post #45595 - In reply to #45591

My best guess is that you're referring to this issue, where the Performance Report was including all warm-up trades from the backtest statistics.

- Fixed an issue with the Performance Report including trades outside of the backtest period.

That problem began in 2J (3-13-18) & was fixed in 2N (6-1-18).

^ Top
Jim Dean

Idol
2000500100100100100
Posts: 2922

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 2:22 PM
Post #45596 - In reply to #45595

Thanks Barry:

Bottom line: do all the backtest/forwardtest input options for analysis mode and portsim mode work properly now?
^ Top
Barry Cohen

Sage
5000500100
Posts: 5603

Joined: 1/19/2004

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 3:26 PM
Post #45597 - In reply to #45596

It's hard for me to say yes to that question with 100% certainty as it's a bit of a blanket statement. Maybe there are certain situations that don't work properly when combining multiple things or in abnormal situations, I don't know. But as far as I am aware the basic functionality of back & forward test settings in the ToDo List combined with Port Sim's Test period settings do work properly. Putting it another way, I'm not aware of any issues that invalidate Mark's results above.
^ Top
Jim Dean

Idol
2000500100100100100
Posts: 2922

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/16/2018 3:45 PM
Post #45598 - In reply to #45597

Thanks!
^ Top
mholstius

Veteran
1002525
Posts: 165

Joined: 1/13/2017

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/17/2018 7:29 AM
Post #45600 - In reply to #45587

Thanks for adding some clarity, Barry.

Buffalo;

This isn’t my exact Expectancy setting, but it’s a good place to start;




The various choices are all minimums, so it helps to think through what each will actually do.
In this case, setting it above zero passes the trade if any the previous 3 trades were profitable. I could be 3 trades ago, the middle, or the most recent trade… or all 3.

A little more on DDs.

Here are the results using margin from 2003-present;




And the comparison of the 3 largest DDs MTM vs closed trade equity for the margin account;




The MDD moves from 10/08 to 8/11 and goes from 28.3% down to 18.9%

Mark


[Edited by mholstius on 10/17/2018 7:32 AM]

Attached file : expectancy01.png (46KB - 533 downloads)
Attached file : 02 Margin.png (178KB - 555 downloads)
Attached file : 03 Margin DDs.png (14KB - 520 downloads)

^ Top
mholstius

Veteran
1002525
Posts: 165

Joined: 1/13/2017

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/17/2018 7:38 AM
Post #45601 - In reply to #45588

Hi jpb,

I've attached a spreadsheet that calculates the DDs using the trades data exported from Port Sim.

Just copy the pertinent data from the Port Sim trades data export into columns A - J (green shaded area).

Calculations are done in L-X

Make sure the formulas in L-O match the amount of data you imported into A-J.

Q-X will show you each DD's info.



Gotta run...
Mark


[Edited by mholstius on 10/17/2018 7:39 AM]

Attached file : Calculate Drawdowns.xlsx (644KB - 37 downloads)
Attached file : spreadsheet.png (44KB - 524 downloads)

^ Top
mholstius

Veteran
1002525
Posts: 165

Joined: 1/13/2017

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/18/2018 8:49 AM
Post #45603 - In reply to #45601

If you downloaded the DD spreadsheet probably already realize this, but...
After letting the DD spreadsheet calculate, if you copy columns Q thru X and then “Paste Values” on another tab you can then sort that DD data by Size, Length, Date, etc.
It’s an easy way to see the distribution of large to small DDs, etc.
Marks
^ Top
jpb

Veteran
100
Posts: 122

Joined: 5/11/2005
Location: Brown Deer, WI

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 10/18/2018 9:18 AM
Post #45604 - In reply to #45603

I started dabbling with it yesterday and the thought crossed my mind... What if we combined it with the other analysis spreadsheet that analyzed strategies against market states? Would there be value in knowing the MDD of any given market state? Or the accumulated effect of micro draw downs in a given MS? Or the location of a DD event in relation to the MS timeframe?

I don't know yet what I would do with this information or how that might change or enhance ATM filters. Maybe looking the Port Sim chart with the dates from the spreadsheet might be all that's needed. And I wonder how the Expectancy Block might actually make these questions irrelevant or reduce the size of MDD%.
^ Top
arby

Member

Posts: 19

Joined: 12/24/2003

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 1/17/2019 8:15 PM
Post #45927 - In reply to #45586

Mark,

Is there any news about the release of your Elite OmniVest Expectancy portfolio?

Arby
^ Top
mholstius

Veteran
1002525
Posts: 165

Joined: 1/13/2017

User Profile
 
Subject : RE: Looking back - and Walking Forward
Posted : 1/18/2019 7:30 PM
Post #45930 - In reply to #45927

Hi Arby,

Sorry, but no news at the moment.
I'm currently working on adding AI to it, but I need to devote my time to some family concerns for a few weeks.

Wish I had more time to investigate the new VATS method Nirvana just posted... that sure looks good.

Ah well, something to look forward to when I get back.

Mark
Jump to page : 1
Now viewing page 1 [25 messages per page]
back reply

Legend    Action      Notification  
Administrator
Forum Moderator
Registered User
Unregistered User
E-Mail this thread to a friend
Toggle e-mail notification


Nirvana Systems
For any problems or issues please contact our Webmaster at webmaster@nirvsys.com.