OmniTrader Forum OmniTrader Forum
forums calendars search
today this week
 
register logon control panel Forum Rules
You are currently browsing as a guest.
You should logon to access more features
A Self-Moderated Community - ALL MEMBERS, PLEASE READ!
Vote for Members who contribute the most to your trading, and help us moderate content within the Forums.


  Current location        Thread information  
OmniTrader 2017 Upgrade Forums
Portfolio Simulation Mode
PortSim versus Analysis Results
Last Activity 9/2/2018 1:56 PM
2 replies, 1364 viewings

Jump to page : 1
Now viewing page 1 [25 messages per page]
 
back reply
Printer friendly version

^ Top
Jim Dean

Idol
2000500100100100252525
Posts: 2884

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : PortSim versus Analysis Results
Posted : 7/19/2011 7:22 AM
Post #24609

If you compare the results from Analysis Reports between Strat-A and Strat-B, then look at the results from Portfolio Simulator runs which are based on exactly the same Analysis, you sometimes will arrive at seemingly "conflicting" conclusions. The Analysis stat's might tell you Strat-A is "better", but PortSim might clearly show Strat-B to be more desirable.

This confuses a lot of people. Hopefully some of the points made below will help clarify the different perspectives offered by these two tools ...


1. Analysis Reports are based on taking EVERY trade that is signaled by the strategy, on every signal.
2. Analysis Reports are based on a ONE SHARE position-size per trade, regardless of share-price.
3. Analysis Summary statistics are selected to be relatively independent of #1+2 limitations.
4. Analysis Reports take Slippage into account, but do NOT pass on that effect to PortSim runs.

5. Portfolio Simulator starts with the list of trades created by the Analysis Reports.
6. PortSim first figures out how much money is available for a given trade, based on spec'd rules
7. PortSim then determines which trade(s) to implement if multiple symbols fire on the same bar.
8. PortSim won't take trades that don't meet its filtering rules re min size, if inadequate funds.
9. PortSim commission calc's can differ from Analysis calc's (since they have separate inputs).


The main point to "absorb" here is that PortSim results are "WEIGHTED" by the funds applied to each trade, and to some degree are "filtered" by max availability of funds and by other specified trading rules.

When you look at APR and ROI and DD, from PortSim, those numbers include all the effects listed above, which "bias" the results significantly versus the "simplistic" approach that Analysis runs use.

Which is "better"? Well, if you've made careful, realistic selections for your Settings in the PortSim run, then clearly that will match your actual trading practice much more closely. So, given that important caveat, the PortSim results are the ones to focus on for OVERALL strategy performance.

The Analysis reports are better to use for early strategy-development, especially when the exits and/or filters are not in a final form yet.


Also (at least with current OT2011 pr3P) the Analysis Test Settings value for Slippage DOES affect the Analysis reports, BUT it does NOT affect the PortSim results which are derived from the same Analysis run (and there is no way to model slippage independently in PortSim).

So, it might be wise for the time being to turn off Slippage in the Ctrl-S > Test Settings panel if you are going to be comparing a lot of Analysis Reports with PortSim runs. This is, btw, not good. Slippage can be important. However, OT's method of modelling slippage is SO simplistic that it's not very accurate anyway. At a minimum, it should be a function of ATR vs price, and also be related to volume vs trade size. I've suggested this to Nirvana before, and will suggest it again. In the meantime, just be aware of it.


For more information on this subject, you can check out the threads in this Room:
http://tradetight.org/forums/forum-view.asp?fid=43

^ Top
Colin945

Member
25
Posts: 49

Joined: 2/26/2005
Location: Calgary

User Profile
 
Subject : RE: PortSim versus Analysis Results
Posted : 2/18/2013 2:13 PM
Post #27930 - In reply to #24609

Jim:

Partially as a result of taking the OTU training over the past 9 months I've been doing a lot of work with Strategy Wizard. Your comments below are consistent with my own observations and conclusions.

In SW I have decided to always use the PS option rather than use the performance reports. I set up PS to only use the Fixed $ amount and set that, and the portfolio equity, to allow all trades to be taken (I know how many stocks are in the list, say S&P 100). I also remove any trade selection criteria so I should be taking all trades.

The biggest issue I'm currently wrestling with is trying to understand the settings availabe for the Fast Search. There seems to be no thorough documentaion as to how it works and what and how they should be adjusted depending on the potential number of permutations that would arise from Exhaustive Search. Are you aware of any such documentation?

My feeling is that coupling SW with PS creates a very powerful tool, especially if used with appropriate quantities of in sample and out of sample data. I have found Pardo's book on the evaluation and optimization of trading strategies to be extremely useful and feel that we could benefit from the implementation of some of his evaluation approaches. Much of the raw data are already provided by the SW/PS combination.

I know that you are one of the leaders in our Club community so wonder if you have any additional insights into this subject?

Thanks for listening.

Colin
^ Top
Jim Dean

Idol
2000500100100100252525
Posts: 2884

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: PortSim versus Analysis Results
Posted : 2/18/2013 2:16 PM
Post #27931 - In reply to #27930

I just addressed that, here:
http://www.omnitrader.com/currentclients/otforum/thread-view.asp?threadid=6378
Jump to page : 1
Now viewing page 1 [25 messages per page]
back reply

Legend    Action      Notification  
Administrator
Forum Moderator
Registered User
Unregistered User
E-Mail this thread to a friend
Toggle e-mail notification


Nirvana Systems
For any problems or issues please contact our Webmaster at webmaster@nirvsys.com.