Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA  User Profile | If you compare the results from Analysis Reports between Strat-A and Strat-B, then look at the results from Portfolio Simulator runs which are based on exactly the same Analysis, you sometimes will arrive at seemingly "conflicting" conclusions. The Analysis stat's might tell you Strat-A is "better", but PortSim might clearly show Strat-B to be more desirable. This confuses a lot of people. Hopefully some of the points made below will help clarify the different perspectives offered by these two tools ... 1. Analysis Reports are based on taking EVERY trade that is signaled by the strategy, on every signal. 2. Analysis Reports are based on a ONE SHARE position-size per trade, regardless of share-price. 3. Analysis Summary statistics are selected to be relatively independent of #1+2 limitations. 4. Analysis Reports take Slippage into account, but do NOT pass on that effect to PortSim runs. 5. Portfolio Simulator starts with the list of trades created by the Analysis Reports. 6. PortSim first figures out how much money is available for a given trade, based on spec'd rules 7. PortSim then determines which trade(s) to implement if multiple symbols fire on the same bar. 8. PortSim won't take trades that don't meet its filtering rules re min size, if inadequate funds. 9. PortSim commission calc's can differ from Analysis calc's (since they have separate inputs). The main point to "absorb" here is that PortSim results are "WEIGHTED" by the funds applied to each trade, and to some degree are "filtered" by max availability of funds and by other specified trading rules. When you look at APR and ROI and DD, from PortSim, those numbers include all the effects listed above, which "bias" the results significantly versus the "simplistic" approach that Analysis runs use. Which is "better"? Well, if you've made careful, realistic selections for your Settings in the PortSim run, then clearly that will match your actual trading practice much more closely. So, given that important caveat, the PortSim results are the ones to focus on for OVERALL strategy performance. The Analysis reports are better to use for early strategy-development, especially when the exits and/or filters are not in a final form yet. Also (at least with current OT2011 pr3P) the Analysis Test Settings value for Slippage DOES affect the Analysis reports, BUT it does NOT affect the PortSim results which are derived from the same Analysis run (and there is no way to model slippage independently in PortSim). So, it might be wise for the time being to turn off Slippage in the Ctrl-S > Test Settings panel if you are going to be comparing a lot of Analysis Reports with PortSim runs. This is, btw, not good. Slippage can be important. However, OT's method of modelling slippage is SO simplistic that it's not very accurate anyway. At a minimum, it should be a function of ATR vs price, and also be related to volume vs trade size. I've suggested this to Nirvana before, and will suggest it again. In the meantime, just be aware of it. For more information on this subject, you can check out the threads in this Room: http://tradetight.org/forums/forum-view.asp?fid=43 | |
