OmniTrader Upgrade Forums
ATM
Using ATM to Improve ATM
38 replies, 6088 viewings
|
OmniTrader Upgrade Forums
ATM
Using ATM to Improve ATM
|
Last Activity 4/21/2021 9:48 AM 38 replies, 6088 viewings |
|
|
| Printer friendly version |
mholstius Veteran   Posts: 175 Joined: 1/13/2017  User Profile |
I’ve just returned from a family vacation and watched the recordings of the first 2 weeks of Jeff Drake’s “ATM Master Course”… and the course is incredible. As I’ve said so many times before… if you don’t have ATM, you really should buy it. If you DO have ATM, Jeff’s course is unquestionably the best way to get the most from your investment. I thought I was somewhat familiar with ATM, but just the first 2 weeks of Jeff’s course have given me countless new ideas to improve my trading with ATM. ATM is a revolutionary piece of software. With the addition of this outstanding course, and the combined efforts of the talented traders in this group, I’m definitely eager to see what we’ll accomplish over the next few months… OK, on to the purpose of this thread - how to use ATM to improve our results with ATM. Trust me, it’s not as convoluted as it sounds. Jeff B started a thread on the “No Trade Market State” introduced in Jeff’s ATM course. I asked him to follow some specific rules for setting up his ATM system so that he could generate the maximum number of trades. He then posted his export of the 38,135 trades generated and I imported then into the attached spreadsheet ("Analyzing Macro And Micro ATM Trades"). Thanks, Jeff... His thread is here: https://www.omnitrader.com/currentclients/otforum/thread-view.asp?threadid=15822&posts=16 These are the steps I shared with Jeff in his thread to obtain all the raw trades the system could generate; 1) Check “Use These Strategies” in each state and then go down and check all the Strategies in the list in every market state. That way, the strategy used will show up along with the market state at the time of the trade in the export of the trades. 2) Set the allocation in each market state to 2% / trade and 100 Longs and 100 Shorts. 3) Set the account starting balance to $1M. That way a 2% allocation will allow a trade in a symbol that trades at $20,000 / share and it won’t be missing out on high priced stocks. 4) Set the Leverage at 6X and check “Use leverage to Increase The Number Of Trades”. With this, you stand less of a chance of missing trades due to hitting the equity ceiling. I do all this because when analyzing a system, I want to look at all the trades the system can potentially generate. I don’t care about ridiculously high ending equities, or big DDs. I want a large number of diverse trades to analyze before applying the excellent ranking and filtering capabilities of ATM. The nice part is that ATM itself can be used to generate the trades and label them with both the Strategies and the Market State at the time of the trade - something we’ve never been able to do before. I’ve plugged Jeff B’s trade data (38,135 trades) into the attached spreadsheet. I’ll use it to show how the analysis can help us find potentially good pairings of Strategies with Market States. Overview: There are 3 tabs in the spreadsheet; The core of the data processing is in the “Analysis” tab. The Exported raw trade data is plugged into the “Trades” tab. The Market States for each day are in the “States” tab (Macro & Micro daily states are included) #1) I’ve repeated the group of Jeff’s 21 Strategies 9 times in rows 2 thru 192 and paired each group of 21 Strategies with each of the 9 Macro & Micro Market States. #2) I’ve found that “Normalizing” the returns according to how many days were available to trade in a particular state is a useful way to compare the efficiency of the strategies (comparing Apples to Apples). #3) You can select any range of dates to be evaluated. #4) You can substitute your own groups of Strategies and Market States if you want to modify the spreadsheet to use it yourself - you just need to ensure that the formulas in the yellow shaded columns match the number of rows necessary for your own analysis. The data can be sorted in numerous ways to help isolate possibly good pairings of Strategies and States. This is a sort by Strategy; This is a similar sort by State; Can this method be used as a “double optimization” to find perfect combinations and ridiculous equity curves that probably won’t trade well at the HRE? Sure - but if done with some prudence, it can be a useful tool to improve results. The key is to input as many trades as possible using as many symbols as possible to get representative results. In particular, if you just input the trades already selected using ATM’s filtering and ranking, then you will definitely be curve fitting. However, if you input a large, representative sample of a variety of diverse trades over time then this can definitely help find robust combinations that ATM will then refine. I’ll close with 2 more plugs… I heartily recommend Jeff’s ATM course being offered by Nirvana - the first 2 weeks have been extremely informative and helpful. And as always, please consider the kids at Saint Jude Children’s Hospital; https://www.stjude.org/ Thanks - and good luck in your trading, Mark I attached "Analyzing Macro And Micro ATM Trades Ver 09" and it now appears to work correctly without any errors on opening after downloading. Not sure what was going on, but please let me know if you have any problems with it. [Edited by mholstius on 8/14/2018 5:08 AM]  Attached file : 00 Overview.png (495KB - 1700 downloads) Attached file : 01 Sort Example 01.png (460KB - 1759 downloads) Attached file : 02 Sort Example 02.png (470KB - 1686 downloads) Attached file : Analyzing Macro And Micro ATM Trades Ver 09.xlsx (3782KB - 356 downloads) | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
This is awesome! (and by the way, the spreadsheet worked fine for me) I'm going to compare it to what I've been doing with Pivot Tables but I already see that you are doing some things here that I couldn't possibly do (or don't know how to) with pivot tables. I continue to learn more with each post of yours that I read. I do have one question. You stated: In particular, if you just input the trades already selected using ATM’s filtering and ranking, then you will definitely be curve fitting. However, if you input a large, representative sample of a variety of diverse trades over time then this can definitely help find robust combinations that ATM will then refine. I take it that you mean, because we took as many trades as possible (using the $1 million, 2% allocation, and 100 long and short), the strategies had to stand on their own (no ATM optimization -- only the definition of the market state really came into play). And because of this and the size of the population over a long period of time, we can find stronger strategies to use with each market state. Then if we activate only those strategies for the given market state, we will effectively be improving our results while minimizing any curve fitting that may be leading us into a false sense of optimistic performance. Did I get that right? Thanks, Jeff B. [Edited by jpb on 8/13/2018 2:10 PM] | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
I love the breakdown between Long and Short. Looking at my Pivot Table (not posted) results compared to your analysis, I would have discarded ATM-NSP-41-MOC-In-224 from Bear and Bear due to an overall loss. However, looking at the data you've produced, I'd actually consider transforming it into a Long only strategy for this market state while in the Bear and Default MS, I would include it as a Long and Short. Is this over thinking or over optimizing in your opinion? Also, the symbol population just happened to be the original set included with ATM Macro & Micro (making it easy to compare). But with this process, I should be able to change up the symbol population and repeat the process to see that the strategies hold up in general within a given market state -- true? Thanks, Jeff B. | ||
| SteveL Veteran Posts: 262 Joined: 8/19/2005 Location: Boulder, CO User Profile |
I'm very interested in what you've done with this spreadsheet, but I can't get it open. Excel 2016 just fails to repair and offers the same "Open and Repair" suggestion over and over. I see Jeff B. was successful, so perhaps I need to install "LibreOffice" which he indicated (in another thread) that he uses. What are you using to produced the spreadsheet? Thanks. | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
All will work fine, except that if you accidentally click on an OT file, it will try to launch LibreOffice. It won't work of course and LibreOffice gracefully informs you of that. It won't break OT either or corrupt the file. So, you are safe, but it is just a consequence of file extensions. I've saved Mark's original file back as excel and uploaded it. Maybe this one will work for Excel 2016. [removed file since Mark uploaded ver 9] [Edited by jpb on 8/14/2018 7:27 AM] | ||
| SteveL Veteran Posts: 262 Joined: 8/19/2005 Location: Boulder, CO User Profile |
Thanks, but it also failed to open in Excel 2016. So, I've installed LibreOffice and both Mark's file and your file open successfully. Steve | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
I also see on the Trades tab that Mark overcame the text based fields for G/L($) and Entry Date that come out of Portfolio Simulator by using the =VALUE() function -- very nice. I've been conducting a change all with RegEx (inside of LibreOffice) to convert those to proper numbers/dates but I'm going to remember this trick. One observation while I was comparing Mark's analysis against my pivot table, specifically in the Default and Bear market state. Two strategies "ATM T3-S3 MOC In 224" and "ATM X ULT MOC In 224" both show a positive G/L(%), arrived at by summing all the G/L(%) for that strategy in that market state. However, if you sum the G/L($), you would see significant losses. T3-S3: G/L(%) 86.08% G/L($) -325,059.02 X-ULT: G/L(%) 16.52% G/L($) -69,198.08 I dug into the trades found on the Trades tab for T3-S3 and see that on 8/20/2015, 8/21/2015, and 1/5/2016, this strategy had a few bad days. Because we are using % Equity as the allocation, the quantity acquired caused significant losses in a short period of time. Usually I'll look just at the % since it normalizes value, but since the loss was significant (without looking at the data), I had originally thrown out those strategies even though they demonstrated good returns (%). But looking at the data, it tells me I forgot why we look at %. If I pulled out the trades from those 3 days, the picture changes dramatically: T3-S3: G/L(%) 352.16% G/L($) 621,076.68 I like that Mark removed the statistics that can cause distraction (at least distract me). Thanks, Jeff B. | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
I was distracted by some other issues yesterday. First, I'll try to attach a newly saved version of the spreadsheet to this reply and see if it downloads better. Mark OK, I attached "Analyzing Macro And Micro ATM Trades Ver 09" to this reply and the original post and it now appears to work correctly without any errors on opening after downloading. Not sure what was going on, but please let me know if you have any problems with it. [Edited by mholstius on 8/14/2018 5:09 AM] Attached file : Analyzing Macro And Micro ATM Trades Ver 09.xlsx (3782KB - 344 downloads) | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
I'll try to respond to your questions (in blue) before the day gets busy. Didn't mean to ignore you yesterday - just had other things to deal with. I take it that you mean, because we took as many trades as possible (using the $1 million, 2% allocation, and 100 long and short), the strategies had to stand on their own (no ATM optimization -- only the definition of the market state really came into play). And because of this and the size of the population over a long period of time, we can find stronger strategies to use with each market state. Then if we activate only those strategies for the given market state, we will effectively be improving our results while minimizing any curve fitting that may be leading us into a false sense of optimistic performance. Did I get that right? Exactly. If we use the trades already screened by ATM and then screen them even more with this, then we're just curve fitting. But, one of the beauties of ATM is that we can use it to export huge numbers of raw trades to see how the strategies work in different market states - and have the trades labeled with the market state at the time of the trade. That's extremely helpful, if done right. Also, the symbol population just happened to be the original set included with ATM Macro & Micro (making it easy to compare). But with this process, I should be able to change up the symbol population and repeat the process to see that the strategies hold up in general within a given market state -- true? Sure! You can repeat your OT and Port Sim run again with a different list that contains all new symbols, append those new additional exported trades to the Trades tab data, and then make sure the yellow shaded cells on that tab are extended to account for the new trades. When analyzed again, it will have even more data to work with to make the analysis more reliable. Nothing would need to be changed on the "Analysis" tab. It will just add the new trades to the analysis. (You might want to re-sort the "Trades" tab by entry date to speed up some calculations, but it's not strictly necessary) With ATM, we can finally begin to identify what type of market and symbol "personality" a strategy works well with. That's huge. Mark [Edited by mholstius on 8/14/2018 5:44 AM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
I really like the analysis spreadsheet data the more I look at it. I noted earlier that there is a Long vs Short. I didn't see in the spreadsheet how it determined that so I'm suspecting Mark is using a Macro to run through the data and dynamically build the Analysis sheet. Actually, we can identify the long vs short by the # of shares purchased. A short trade has a negative number of shares in the exported "Qty" field of the data. No macros in the spreadsheet. Usually I'll look just at the % since it normalizes value, but since the loss was significant (without looking at the data), I had originally thrown out those strategies even though they demonstrated good returns (%). But looking at the data, it tells me I forgot why we look at %. Yup, $ amounts vary with the size of the acount, but %'s are static. Also, I'm not sure why these had so few trades in your data...??? NSP-41: 1 trade T3-S3: 2 Trades X-MFI: 2 trades I hope others will be able to use the spreadsheet to improve things too. I used your data as a good example (thanks), but it can work for anybody. And a reminder - when you do the sorts to glean more information, be sure to sort columns A thru J. But, that's probably obvious. You can also sort the small tables for the State (L1:T11) and Strategy (L13:P34), but you have to specifically select each range before doing the sort. Also probably obvious, but wanted to point it out. Good luck - off to another busy day, Mark | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
As for why NSP-41, T3-S3 and X-MFI didn't have many trades, I'll have to look deeper. I saw that too and added it to my list to explore. In fact, I expected far more trades from some of the derivative strategies too. I might need to step through these strategy by strategy. I suspect that if two strategies fire for the same symbol that only the first one alphabetically is being taken. That might explain why the *-224 had trades while the *-324 did not. It may also explain why NSP-41, T3-S3, and X-MFI had so few. I'll report back once I've had a chance to get through that analysis. | ||
| SteveL Veteran Posts: 262 Joined: 8/19/2005 Location: Boulder, CO User Profile |
The Ver 09 spreadsheet opens without errors. Thanks. And thanks for the spreadsheet. It will be useful. | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Don't go too far, I've discovered some important info and am improving the spreadsheet today. Will be back with more later today - along with better ways to use it. Mark | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
After extensive testing over the past 5 days and discussions with Barry, I can explain why they don’t show many trades - and why this discovery(?) is so important for this kind of analysis. I had forgotten (if I ever knew it?) that OT only allows 1 trade in a symbol / day. With ATM, and especially using concurrent states, this is a critical point. Unfortunately, I have things to do today and won’t be able to illustrate what to do about this until later - but I’d suggest waiting before doing more analysis until I post the info and an updated spreadsheet. Mark | ||
| Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Also, if mult Strategies fire on one symbol, only one will control it. [Edited by Jim Dean on 8/14/2018 12:10 PM] | ||
| EYEGUY Icon  Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
I can understand your second statement as this has to do with OmniTrader and the way it works. But certainly your first statement can't be correct (or at least in the way I interpret it). On Interactive Brokers if I, say, have a position in SPY I can add to that position at any time - on the day of the original trade or any day thereafter. If this were not the case one couldn't spiral up his trades as so many strategies (like Connors' 1-2-3-4) work. And when I exit the trade my profit on each "leg" of the trade will be determined by the price at which I entered the trade. So, what am I missing here? Tom Helget | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
If you want a separately controlled (via TP) position, it needs to be via a separate brokerage account. Example: can’t trade daily bars and intraday bars at same time from same account for a single symbol. | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Sorry, but I need to add some clarity to this, as it doesn’t apply in Omnivest. In my Elite FTM portfolio, we regularly have multiple trades open in 1 symbol at IB that are controlled by multiple trade plans, with entries and / or exits on different days. Here’s a quick snag I made of 5 different positions in the live account. In each case, the 5 concurrent trades in each of the 5 symbols were controlled by 5 different strategies / trade plans; No criticism intended, just want to clarify that it happens regularly in OV coupled with IB. I'm adapting to OT rules... Mark [Edited by mholstius on 8/14/2018 2:12 PM] Attached file : Multiple Trade Plans.png (404KB - 1424 downloads) | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
So I’ll amend my statement to refer to trades in OT. (And also VT trades) Mark do you know if the OV ATM separate-server-instance tool also permits overlaps? I don’t know what “FTM” is, that that you referenced. I don’t understand how OT can be a development platform for ATM if it’s core capabilities differ that much from OV ATM. Seems like performance might differ markedly. [Edited by Jim Dean on 8/14/2018 2:17 PM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Maybe someday it will actually come under OV "rules", but at the moment it's just a convenient way to trade ATM on IB via Nirvana's servers. FTM = Follow The Money portfolio. Thanks for your input, Mark | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
[Edited by Jim Dean on 8/14/2018 2:54 PM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Mark Did you ever consider practicing law...??? ;-) | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
I was already about 75% through rerunning all the strategies so I just kept going. The spreadsheet worked great. Because of all the strategies I was testing, I had to unmerge the instructional box on the Analysis tab and then remerged it when I was done expanding the Strategy Performance section. The reason for the lack of trades for many of the 28 strategies was that trades for a symbol had been taken already for that day as suspected and confirmed. Running the strategies one by one through the market states eliminated that problem. Since I have many different variations of the same base strategy that I was running through, it was necessary to isolate each from another. Now all the results are side by side and I can see which strategies actually perform well in any given MS. And I can compare the strategy to its base to see if it actually improved performance or not. I have 2 variables to each strategy compared to the base: 1) Market on Close 2) Inactivity So, I can't contribute the impact of each change to overall improvement (or degradation) without removing the Inactivity stop and running a new set with only MOC as the variable. I'll have to look through my virtual machines to see where I have those versions -- I'm certain I still have them around. There is one problem. The number of trades is now over 275,000. The spreadsheet is almost 24 MB in size -- zipping it is only shrinking it to 22.8 MB so I can't upload it. I'll try a few more methods of compressing it and see if I can get it under the 7 MB limit for attachment. Update: Sorry, even 7zip could only compress it to 21 MB. Update 2: For those who want it, I placed it on my google drive [Edited by jpb on 8/14/2018 3:43 PM] | ||
| mholstius Veteran Posts: 175 Joined: 1/13/2017 User Profile |
Exactly what I was going to suggest and explain. Can you just put the exported trades by themselves into a spreadsheet and attach it (or send a dropbox link to it)? I've finished the updated version of the spreadsheet and could put the trades into it. Mark If not, please send me a private message and we'll work something out. | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
The prior post has a link to my google drive if you'd like the full spreadsheet. 7-zip files can be opened with most archive tools -- Peazip is one that I use on Windows. I tried straight zip but it was close to the 7MB limit. I'll put the zip file version on my google drive also for those that don't want to deal with 7zip. Attached file : Analyzing Macro And Micro ATM Trades Ver 09-Revised.csv.7z (3429KB - 325 downloads) |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
