OmniTrader Archives
OmniTrader 2017 Technical Support
Expectancy Testing
29 replies, 6660 viewings
|
OmniTrader Archives
OmniTrader 2017 Technical Support
Expectancy Testing
|
Last Activity 1/4/2023 9:49 AM 29 replies, 6660 viewings |
|
|
| Printer friendly version |
mholstius Veteran   Posts: 175 Joined: 1/13/2017  User Profile |
The Expectancy block is inserted as a filter at the end of whatever System you’re using. It utilizes a very flexible grid to choose various parameters that can be entered to determine whether the current trade will be entered based on the performance of trades that were previously signaled by the System with that particular symbol. The real benefit is that this is done as a walk forward in time – so the only requirement that compares to a “Back Test” is the need for a sufficient amount of time for the system to generate enough trades to satisfy your Expectancy rules. IMPORTANT NOTE: if you want the Expectancy Filter to use the previous 7-10 trades, that could mean that you’d need 5-6 years of trading designated as a “Back Test” period in OT in order to have a sufficient number of trades prior to the period designated as a “Forward Test” when using the Expectancy filter. To accomplish that, I’ve set my OT data periods to 5,000 and I’ve set up my “To Do List / Test Settings” so that I have 7 years of data in the “Back Test” prior to the “Forward Test” period; This is an example of the entries I used in the Expectancy block to require that the VBX3 system had to have a Min PPT of 2% over the previous 9 trades in order for the next trade to be taken (setting the Lookback to 5,000 is just to ensure there’s sufficient data); The recent OT 2017 update to PreRelease 2C enables the use of the Expectancy Block in Strategy Wizard. While my development testing in Excel took 100s of hours, now it’s possible to run multiple combinations of trade filters in mere minutes. Incredibly better! For my first tests I used a list of 435 ETFs and only used the VBX3 RTM System. My goal was to use the filter to selectively choose better trades walking forward. I was concerned that I might just be picking a “best case / spike” with my Expectancy settings, so I ran Nirvana’s excellent Strategy Wizard using a range of 1-3% in steps of 0.25% and # Trades from 6-11. The charts below show a smooth improvement in the results - with my requirement of a Minimum of 2% PPT for the previous 9 Trades in the middle. Hopefully, those settings are robust enough that future trades that generate statistics close to my settings shouldn’t have a large effect either way on the results; The unfiltered VBX System produced ~13,500 trades from 1/1/2007 – 1/1/2017. Using an Expectancy filter requiring a Min of 2% PPT in the previous 9 Trades reduced that to ~1,000 trades. A) The original PPT of 0.71 increased to 1.76 using the Expectancy Filter B) The original Avg APR of 62 increased to 162 using the Expectancy Filter C) The original Avg Win of 1.89 increased to 3.25 using the Expectancy Filter My original research on Expectancy using Excel only used PPT, and Nirvana added 3 more possibilities. I wanted to see what would happen when using Avg APR as a filter. I also wanted to test a larger range of values (0 – 200% Avg APR in steps of 20% and 0 – 20 Trades) to confirm that the results were smooth - so my Strategy Wizard run for this more extensive test took about 15 hours to run. I believe the smooth results obtained confirm that the concept has great potential; A) The original PPT of 0.71 increased to 1.24 using the Expectancy Filter B) The original Avg APR of 62 increased to 111 using the Expectancy Filter C) The original Avg Win of 1.89 increased to 2.42 using the Expectancy Filter I’m currently running more tests with various stock groups, and will post those results when I’m finished. This is a new concept that Nirvana implemented in OT 2017. It should allow us to utilize Systems that generate a lot of trades – and enable this filter to dynamically choose which trades to take walking forward based on the System’s recent individual performance with each Symbol in the list. I hope this will be dramatically more robust vs the old method of using fixed settings that were chosen to fit the data over a long period in the past. As always, please pass on your opinions and experiences – and good luck in your trading, Mark [Edited by mholstius on 1/13/2017 11:00 AM]  Attached file : 00-Test Settings.png (26KB - 1885 downloads) Attached file : 01-Expectancy Entries.png (43KB - 1967 downloads) Attached file : 02-Expectancy Filter Using Avg PPT.png (453KB - 1878 downloads) Attached file : 03-Expectancy Filter Using Avg APR.png (485KB - 1871 downloads) | ||
| EYEGUY Icon   Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
Super stuff! Might it be possible for you to post a .pdf file of the superb analysis you are posting in this thread for more detailed inspection. Also for any future results you refer to that might be forthcoming. I look forward to using the new Expectancy Filter block once I understand it a bit better. Tom Helget | ||
| kmcintyre Legend Posts: 410 Joined: 8/30/2007 Location: Valley Center, CA User Profile |
Outstanding work, as usual! I would think that these results would hold up well in HRE trading. (But I thought that about a lot of RTM strategies and OV approaches...) I really hope this translates into real world returns! So my questions are probably more towards Nirvana, but - 1)When might we expect to see all the OV RTM strategies upgraded to include the Expectancy Block? 2) When might we expect to see ARM5 AI and GA enhanced strategies in OV? 3) Mark, will you need to redo all your work picking ETF/Strategy pairs once new strategies are available? 4) As a stop gap, might we expect Expectancy filtering of trade candidates in OV soon? 5) Mark, has flexible by design turned around since I bailed? And a personal note - over many years (and many thousands of dollars) I've seen RTM strategies perform superbly in back and forward tests on historical data, only to grind to a halt at the HRE. Call me a skeptic but I think I'm going to need 6 - 12 months of HRE profits in a demo account before I'm willing to throw more hard cash at a system. Of course, if someone can come up with a logical explanation for why RTM strategies stop working every time I start trading them, I'd be very interested in hearing it. Cheers! Keith | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
OK, I have had some time to absorb your excellent presentation. Although you did not discuss the concept of the Expectancy Filter block Ed Downs did in a mailer: Thus it would seem that the Expectancy Filter Block is looking for the "hot hand" - a run of consistently profitable trades before a new trade would even be considered. The problem I have with that is like the first precept that is usually taught in statistics/probability courses. I am sure you will recognize it. Johnny has a fair penny that he has tossed ten times and ten times it has come up heads. What is the probability that on the next toss a tails will appear? Well, of course, not knowing anything and considering the string of ten heads to be a bit unlikely one is tempted to say 100%. But the real answer as we all know is 50%. No previous toss of the coin can influence any future toss as long as the coin is indeed a fair coin. So, the premise put forth by Ed is a bit under suspicion in my opinion. Just because there were several winners preceding our contemplated trade does not mean that our next trade might not go south rather quickly. Now another problem I have is with duplicating results. Ed describes the creation of a new MVX-15 EXP strategy in the same brochure that is used in conjunction with the Connors RSI and produces incredible results: Unfortunately I am not in possession of the MVX-15 Strategy Ed refers to with my 2C OT update and cannot duplicate his claim. Nor can I even replicate the strategy since his parameters for the strategy were not imparted in the brochure. So, for the moment I am a bit skeptical of the Expectancy Block Filter. Can you perhaps address any of my concerns? Thanks, Tom Helget [Edited by EYEGUY on 1/14/2017 12:25 PM] Attached file : The Expectancy Block Concept.PNG (74KB - 1773 downloads) Attached file : MVX-15 EXP Concept.PNG (63KB - 2357 downloads) Attached file : MVX-15 EXP and Connors RSI.PNG (178KB - 1832 downloads) | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Sorry, but I don't have any more information on Ed's MVX-15 than what you put up. The Expectancy Filter development work that I did and Ed has implemented in OT is very new and quite different. The concept is to use the previous performance of a System to hopefully "weed out" better System / Symbol combinations from a large pool of potential trades each day walking forward. As new as the concept is, I can only respond with some of the results from the very limited testing I've been able to do so far. There's a LOT more to be done & learned - but I'll paste a comparison of a "before and after" run I did today using OT's Portfolio Simulation; Chart A on the top is running just VBX3 using my list of 434 ETFs Chart B also uses just VBX3 and the 434 ETFs, but with a simple Volume Filter and the Expectancy Filter set to require an Average APR of 75% over the previous 6 Trades in a symbol in order to take the next trade in that symbol. As you can see, there's a considerable difference in the personality and statistics for the 2 charts - the major difference being that both the Volume and Expectancy rule are applied at the time of the trade on a walk forward basis. There's still a lot to learn, but if it holds up I'd certainly prefer to trade chart B. I have to leave shortly for the next few days, but will try to post more info next week after running more tests. Hope you have a nice weekend, Mark [Edited by mholstius on 1/14/2017 3:25 PM] Attached file : 01-Before And After.png (315KB - 1783 downloads) | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
Thanks, Chart B sure looks nice! And the statistics are nice too. For example the Max Drawdown % wonderfully went from a very undesirable 24.9% to a very tolerable 6.4%. Avg Ann ROI dropped some to 11.1% from 16.1% while the Calmar rose to 1.73 from 0.65. And the number of trades necessary to achieve these results was a mere 10% of the Chart A number reducing commission costs and trading time and effort. Even more importantly the Average Invested % went from 36.9% down to a mere 6.2%. So, thanks for showing your Portfolio Simulation statistics - I wish Ed had done that with his Connors - MVX-15 EXP setup. Also I would appreciate it if you could clarify what you mean exactly by "at the time of the trade - on a walk-forward basis". Doesn't every strategy do this even when it isn't looking at it's past performance? Thanks, Tom Helget | ||
| Stumpy Veteran Posts: 214 Joined: 5/25/2004 Location: Australia User Profile |
Attached file : PORTFOLIO SIMDocument1.docx (141KB - 441 downloads) | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Thanks for your observations. Here’s a short explanation for the “walk forward” phrase applied to a period (say 2009-2017 in my chart); 1.You’re right - the systems (strategies) always generate trades based on their individual “triggers” (RSI level, MA cross, etc.) each day. 2. In the past, we’ve used various methods to find better results looking back over a period (say 2009-2017) by choosing which of those trades to select: Iterative Strategy Selection, maximizing a particular goal (CALMAR), etc. The problem with those methods is that we’ve been looking at things with the benefit of hindsight and picking a small set of trades from the larger pool that happen to fit well together over that period (can we say curve fitting?). We’re left with the possibility that the logic / rules used to do that while having the benefit of looking back won’t work at the HRE - so in essence the HRE becomes our FT… The difference using Expectancy… We still have the same large pool of trades every day - but as each trade triggers, the decision of whether to take the trade or not is based on the parameters in the Expectancy block at the time of the trade (“walking forward” & not in hindsight). In my example, I told it to look at the previous 6 trades in a symbol using that system (a pairing of Symbol and Strategy) – and if those 6 trades didn’t produce an AvgAPR of 75% then the trade won’t be entered. For every trade, it’s looking back at a sliding window of the previous 6 trades for that System / Symbol pair (or whatever settings you’ve chosen in the Expectancy Filter). It’s not a time period (X bars), and the decision might be different for the next trade for that pair because the “last 6 trades” window and results will change as it slides forward. I’m hoping we can feed it a lot of trades made by a lot of different pairs (symbol & system / strategy) and it’ll look back at the time of each trade and decide whether that particular pair’s “personality / performance” in the past (defined by the Expectancy Filter) was good enough to take its next trade. Will it work going forward at the HRE…??? We won’t know until we settle on an Expectancy setting that looks robust and actually trade it, but the equity curves produced in the development phase are different from what we’ve used in the past. The equity curve is now determined by the results of choosing which trade to take / reject based on the recent performance of each Symbol & System on the day of each trade - vs a decision of which trades we “would’ve taken” looking back over the entire time period. We’re attempting to make each day on the chart a “HRE decision” and see how those rules work walking forward. My hope is that we’ll be able to find a robust set of rules that will continue to work going forward because we’ll have a better representation of how it worked “at the HRE” in the past. Hope that helps, Mark | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
That’s excellent! Glad to see you’re getting favorable results using the new expectancy capability. There are a lot of combinations / permutations to test and learn about… Thanks, Mark | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
My pay grade only allows me to answer 2 of your questions. ;-) 3) Mark, will you need to redo all your work picking ETF/Strategy pairs once new strategies are available? I’ll have to research that once they come out. 5) Mark, has flexible by design turned around since I bailed? A bit – The IRA’s had 10 trades: 80% HR, up ~ .6%. The Margin’s had 13 trades: 77% HR, up ~1.2% On the questions to Nirvana; I’m hoping we’ll see more good results with Expectancy and it’ll make its way into OV sometime soon… Mark | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
So as to a particular strategy to select for Expectancy Testing modification - it would appear that one should consider strategies that fire frequently within a short period of time as opposed to those that fire relatively infrequently so that the "personality of the market" would not have time to change. For example the new Connors RSI which produces around 5000 trades on the Russell 1000 over a twenty year period on a Portfolio Simulation (or roughly 5 trades per symbol over twenty years) or, to put it another way, roughly 1 trade a day on average would not be a reasonable candidate for Expectancy Training modification. Would that be a reasonable assumption? Thanks, Tom Helget [Edited by EYEGUY on 1/17/2017 11:36 AM] Attached file : CRT-3 PORTFOLIO SIMULATION.PNG (97KB - 1688 downloads) | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Why don't you try testing your theory by adding an Expectancy filter to it and then inserting various parameters in Strategy Wizard for Expectancy and see how it does? I'd certainly be interested in what you find, as I'm a bit busy running a bunch of tests on a couple of machines myself. Thanks, Mark | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
You read my mind. OK, here are my settings and the strategy: Of course it will take a while. Tom Helget [Edited by EYEGUY on 1/17/2017 12:16 PM] Attached file : CRT - 3 EF Test Settings.PNG (9KB - 1599 downloads) Attached file : CRT - 3 EF Strategy.PNG (19KB - 1596 downloads) | ||
| wolf Veteran Posts: 159 Joined: 4/16/2011 Location: Germany User Profile |
might it be possible to consider for further Portfolio Simulations Starting Balance 10.000 or 20.000 USD Leverage:1 Fixed Trade size Commission: 10 USD each trade. 100.000 USD starting Balance is a very high amount. I do not know if everyone is able to invest 100.000 Starting Balance. In Europe most Banks charge approx 10 USD each trade for an amount up to 3000 USD Brgds Wolfgang [Edited by wolf on 1/17/2017 1:19 PM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
A suggestion: I'm using a fixed FT period so that I can always go back and compare any changes I make... Also, if you want to do a bunch of runs overnight or something, you can set up Strategy Wizard to do a range of settings - BUT BE SURE that when you're in Strategy Wizard and setting up your options to choose the first button in each (Min HR, Min AvgAPR, Min AvgPPT, Min PPT) and Click "Fixed Value" and then "Checked" (for the one you're testing) or "Unchecked" (for the ones you're not). I missed that and wasted 14 hours of computer time... sigh. Mark [Edited by mholstius on 1/17/2017 1:22 PM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Feel free to run it any way you want. I think most of us use $100,000 or $10,000 just because the math is easier. If you post your results, we're really just interested in the before / after effect of using different Expectancy values. Thanks, Mark [Edited by mholstius on 1/17/2017 1:23 PM] | ||
| wolf Veteran Posts: 159 Joined: 4/16/2011 Location: Germany User Profile |
the thread you launched is one of the best I have ever seen in the OT Forum. It is a revitalization. Really amazing Wolfgang | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
I've been hanging out over in the Omnivest Forum for so long my membership here had expired... ;-) Mark | ||
| Mel Veteran Posts: 291 Joined: 12/30/2003 Location: Kensington User Profile |
| ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
I've already communicated that to Nirvana too - it's important in both the PPT and AvgAPR. FWIW: I've found you CAN set any increment you want while doing a Strategy Wizard run if that helps... Mark | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
OK, I am not really sure what I got. My result was not nearly as smooth as yours (and having done many Strategy Wizard runs I have to say that your results appeared a bit unusual to me in their clarity). But here it is for the FT PPT: What interpretation would you apply to the data? Especially concerning the drop in the FT PPT at the Min Avg PPT value of 2.25%? Also what do you make of the Number of Trades results? Would it have been better to have used a value of 2 or 5 instead of the 4 chosen?: Thanks, Tom [Edited by EYEGUY on 1/17/2017 8:55 PM] Attached file : Connors RSI Expectancy Testing PPT.PNG (20KB - 1561 downloads) Attached file : Connors RSI Number of Trades.PNG (10KB - 1558 downloads) | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Nice to see you’re experimenting – there are so many capabilities and permutations with this filter that I’m sure it’ll take a while to test them and see how they work and what effect they have on each other. I know you’ve done a lot of work, both here and on Jim Dean’s excellent TradeTight site, so I’m sure you realize this - but for the benefit of others just starting with Port Sim I’ll point out that the 2 charts you posted are looking at the same data, but you’ve just reversed the plots. Both have FT PPT as the Y axis, but the top chart has Lookback # Trades as the “Dots” and Min Avg PPT as the X axis. In your 2nd chart, you’ve kept FT PPT as the Y axis and swapped the other 2: the Dots are now Min Avg PPT and the X axis is Lookback # Of Trades. Rather than add to this thread, I’ve started another one at the link below containing more information on how I interpret the output from Port Sim runs; https://www.omnitrader.com/currentclients/otforum/thread-view.asp?threadid=12351&posts=1 Thanks for your input, Mark [Edited by mholstius on 1/22/2017 6:34 PM] | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
As I am sure you realize my plots above were from a Strategy Wizard run and not a Portfolio Simulation. Tom Helget | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
So much technology... so many terms. ;-) Mark | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
Such excellent work by any other name would smell just as sweet.... Tom Helget |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
