OmniTrader Forum OmniTrader Forum
forums calendars search
today this week
 
register logon control panel Forum Rules
You are currently browsing as a guest.
You should logon to access more features
A Self-Moderated Community - ALL MEMBERS, PLEASE READ!
Vote for Members who contribute the most to your trading, and help us moderate content within the Forums.


  Current location        Thread information  
OmniTrader Archives
OmniTrader 2012 Technical Support
Trade Calc allocation varies unexpectedly
Last Activity 3/18/2022 1:29 PM
11 replies, 4321 viewings

Jump to page : 1
Now viewing page 1 [25 messages per page]
 
back reply
Printer friendly version

^ Top
SteveL

Veteran
1001002525
Posts: 262

Joined: 8/19/2005
Location: Boulder, CO

User Profile
 
Subject : Trade Calc allocation varies unexpectedly
Posted : 8/13/2012 5:26 PM
Post #27031

Hello support:
I have observed that the allocations vary depending on whether only back test period or BOTH test periods are selected in Portfolio Simulator, and also vary depending upon the value set in "Optimize on xxxxx boundaries" even if the "Dynamic Optimizaiton" option is not selected.

It is not clear to me why these changes result in different allocations. And please notice that some allocations are "0" (zero) which I think is incorrect.

Here is an overview of the test sequence:
1. Create a new profile.

2. Setup symbol list.

3. Select strategy.

4. Configure test periods with "Optimize on (default) MONTHLY boundaries".

5. Adjust number of bars loaded.

6. Configure default portfolio.

7. Run analysis and portfolio simulation.

8. Select a stock with signal (AA in this example).

9. Review trade calculator results.
NOTE: Several of the allocation methods have quantity = 0. Why??

10. Review PortSim results for MONTHLY boundaries and BACK test period only.

11. Run PortSim with BOTH test periods.

12. Review PortSim results for MONTHLY boundaries and BOTH test periods.

13. Review trade calculator results after PortSim run with MONTHLY boundaries and BOTH test periods.
NOTE: All methods have allocated quantities. And some previously allocated quantities have changed.

14. Change optimization boundary to DAILY boundaries.

15. Run analysis and PortSim with optimization on DAILY boundaries and BOTH test periods.

16. Review trade calculator results with DAILY boundaries and BOTH test periods.
NOTE: Results are different than with "MONTHLY" boundaries, even though the optimization is not selected.

17. Review PortSim results for DAILY boundaries and BOTH test periods.

18. Change optimization back to MONTHLY boundaries.

19. Run analysis on MONTHLY boundaries and BOTH test periods.

20. Review trade calculator results for MONTHLY boundaries and BOTH test periods.
NOTE: The results are the same as previously for MONTHLY boundaries and BOTH test periods.



Following is the test sequence and results with snapshots:
1. Create a new profile.


2. Setup symbol list.


3. Select strategy.


4. Configure test periods with "Optimize on (default) MONTHLY boundaries".


5. Adjust number of bars loaded.


6. Configure default portfolio.


7. Run analysis and portfolio simulation.


8. Select a stock with signal (AA in this example).


9. Review trade calculator results.
NOTE: Several of the allocation methods have quantity = 0. Why??


10. Review PortSim results for MONTHLY boundaries and BACK test period only.


11. Run PortSim with BOTH test periods.


12. Review PortSim results for MONTHLY boundaries and BOTH test periods.


13. Review trade calculator results after PortSim run with MONTHLY boundaries and BOTH test periods.
NOTE: All methods have allocated quantities. And some previously allocated quantities have changed.


14. Change optimization boundary to DAILY boundaries.


15. Run analysis and PortSim with optimization on DAILY boundaries and BOTH test periods.


16. Review trade calculator results with DAILY boundaries and BOTH test periods.
NOTE: Results are different than with "MONTHLY" boundaries, even though the optimization is not selected.


17. Review PortSim results for DAILY boundaries and BOTH test periods.


18. Change optimization back to MONTHLY boundaries.


19. Run analysis on MONTHLY boundaries and BOTH test periods.


20. Review trade calculator results for MONTHLY boundaries and BOTH test periods.
NOTE: The results are the same as previously for MONTHLY boundaries and BOTH test periods.


[Edited by SteveL on 8/13/2012 5:33 PM]

Attached file : 01 Create new profile.png (26KB - 936 downloads)
Attached file : 02 Setup symbol list.png (108KB - 995 downloads)
Attached file : 03 Select strategy.png (32KB - 992 downloads)
Attached file : 04 Configure test periods with optimize on MONTHLY boundaries.png (33KB - 944 downloads)
Attached file : 05 Adjust number of bars loaded.png (53KB - 978 downloads)
Attached file : 06 Configure default portfolio.png (35KB - 961 downloads)
Attached file : 07 Run analysis and portfolio simulation.png (29KB - 965 downloads)
Attached file : 08 Select stock with signal.png (77KB - 1042 downloads)
Attached file : 09 Review trade calculator results.png (93KB - 1046 downloads)
Attached file : 10 Review PortSim results for MONTHLY boundaries and BACK test period only.png (145KB - 1039 downloads)
Attached file : 11 Run PortSim with BOTH test periods.png (36KB - 945 downloads)
Attached file : 12 Review PortSim results for MONTHLY boundaries and BOTH test periods.png (146KB - 969 downloads)
Attached file : 13 Review trade calculator results after PS run with MONGTHLY boundaries and both test periods.png (119KB - 1055 downloads)
Attached file : 14 Change optimization to DAILY boundaries.png (39KB - 952 downloads)
Attached file : 15 Run analysis and portfolio simulation with optimization on DAILY boundaries.png (35KB - 985 downloads)
Attached file : 16 Review trade calculator results after PS with DAILY boundaries and both test periods.png (95KB - 967 downloads)
Attached file : 17 Review PortSim results for DAILY boundaries and BOTH test periods.png (146KB - 995 downloads)
Attached file : 18 Change optimization back to MONTHLY boundaries.png (39KB - 978 downloads)
Attached file : 19 Run analysis on MONTHLY and BOTH test periods.png (34KB - 966 downloads)
Attached file : 20 Review trade calculator results for MONTHLY and BOTH test periods (same as original results).png (100KB - 934 downloads)

^ Top
Jim Dean

Sage
20001000
Posts: 3022

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/13/2012 5:33 PM
Post #27032 - In reply to #27031

WOW ... great doc's Steve

Let me add my voice also that we need a VERY clear explanation of how and why TC does its thing the way it does ... and a fix if necessary.

TC is "spending our money for us" ... it's the most important component in OT to be thoroughly documented, imho.

Thanks
^ Top
Barry Cohen

Sage
5000100010010010025
Posts: 6338
Online

Joined: 1/19/2004

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/14/2012 5:32 PM
Post #27037 - In reply to #27031

9. I don't know. Did every symbol in the list produce zeros for those methods?

13. Of course some of the method results have changed. You changed it to use Back Test only to Back Test + Forward Test. So additional trades are being included & changing statistics.

16. Yes, they will be different because your Back Test & Forward Test periods have changed due to the optimization boundary change. Even if your strategy might not be optimizing (which I don't know if it is or not), the boundaries changed. Dynamic optimization has nothing to do with any of this.

20. You're comparing MONTHLY boundaries and BOTH test periods to MONTHLY boundaries and BOTH test periods. Why do you think they shouldn't be the same?

^ Top
SteveL

Veteran
1001002525
Posts: 262

Joined: 8/19/2005
Location: Boulder, CO

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/14/2012 6:20 PM
Post #27038 - In reply to #27037

9. I'll rerun and see if the 0 result is found in all stocks. It doesn't seem correct for it to be true for even 1 stock. I'll get back to you on this.

13. So, the things that changed were the ones that were previously zero. For example, I don't understand how Back and Forward tests could affect "fixed risk". It is an ATR and equity based calculation. Why should it change based on the boundary for "optimization?

16. OK. Conceivable. But the differences are HUGE. Quite surprising. Am I right to assume the results should be the same at the monthly boundary? For example: Fixed Risk went from 2395 down to 95. I think the only difference is that the "boundary" used by PortSim to calculate its data ended at the end of July, vs the yesterday. Why should that affect Fixed Risk, and why such a HUGE difference. (And Fixed Risk is not the only huge difference.)

20. Sorry, I wasn't clear. I agree with you. I was simply pointing out that results were consistent.

I'll get back to you on the zero results in the first point.
^ Top
SteveL

Veteran
1001002525
Posts: 262

Joined: 8/19/2005
Location: Boulder, CO

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/14/2012 11:10 PM
Post #27039 - In reply to #27037

Hi Barry,

With PortSim set to do Back Test only, and ToDo List Test Settings "Optimize on MONTHLY boundaries" the Trade Calculator allocates quantity=0 for ALL stocks for the following allocation methods:
Fixed Risk
NN% of Equity
Optimal f

The other issue is that I don't understand how/why changing "Optimize on XXXXX boundary" would result in any allocation changes, since there is no optimization being done anywhere that I'm aware of.

Steve

[Edited by SteveL on 8/15/2012 7:06 AM]

^ Top
Barry Cohen

Sage
5000100010010010025
Posts: 6338
Online

Joined: 1/19/2004

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 9:50 AM
Post #27041 - In reply to #27039

With PortSim set to do Back Test only, and ToDo List Test Settings "Optimize on MONTHLY boundaries" the Trade Calculator allocates quantity=0 for ALL stocks for the following allocation methods:
Fixed Risk
NN% of Equity
Optimal f


You might check to see if any trades at all occurred in the back test period for those methods, maybe trades for those 3 occurred only in the forward test period.

The other issue is that I don't understand how/why changing "Optimize on XXXXX boundary" would result in any allocation changes, since there is no optimization being done anywhere that I'm aware of.


The "Optimize on boundary" setting doesnt necessarily need optimization for it to have some effect. Really all it does is tell OT where the floating back test period ends. If set to Monthly, the back test period will end on the first bar of the current month.
^ Top
SteveL

Veteran
1001002525
Posts: 262

Joined: 8/19/2005
Location: Boulder, CO

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 10:40 AM
Post #27042 - In reply to #27041

Hi Barry,
There appears to be a bug in the Trade Calculator. Can you please look into it?

In your last post, you asked if there were trades in the backtest. The answer is YES. This is Nirvana's default Breakout strategy running against the DOW 30. All of them have trades in the back test. The stock I previously chose as my example does not have a FTHR (although it does have a FTNT because it is still in trade - which was the reason I selected it for focus). Attached below are the focus list with stats, WMT chart because it has many BT and FT trades, and the PortSim results for Fixed Risk (demonstrating that it is configured).

The FixedRisk allocation is 0 (along with some other allocation methods). Let's focus on Fixed Risk. I don't think it should be zero. You'll notice in the focus list that there is an ATR value. And I'm using the default Portfolio ($50,000) setup. And FixedRisk is configured (with the default) in Portfolio Simulator. And there are NO constraints configured. What more is needed for FixedRisk allocation to be calculated? The allocation should not be zero.

If I change to use "DAILY" boundary, then there will be a value calculated for FIXED RISK.

This used to work. It stopped working for me about 1 week ago. I was using it in a much more complicated set of strategies. I put together this simple scenario to make it easy to replicate the problem.

In trying to set up the simple FIXED RISK scenario, I've observed that other methods are also act inconsistently when switching from MONTHLY to DAILY boundary. To me, there appears to be a systemic problem in the Trade Calculator with regard to how it is interacting with MONTHLY/DAILY boundaries and BACK vs BOTH (and also probably FORWARD although I have not done that variation) test periods in the Portfolio Simulator.

Thanks in advance for looking into this problem.
Steve







[Edited by SteveL on 8/15/2012 10:41 AM]

Attached file : 8-15-2012 9-25-13 AM Focus list with stats.png (99KB - 953 downloads)
Attached file : 8-15-2012 9-26-29 AM Trade calculator results.png (116KB - 942 downloads)
Attached file : 8-15-2012 9-27-31 AM PortSim showing FixedRisk setup.png (110KB - 985 downloads)

^ Top
Barry Cohen

Sage
5000100010010010025
Posts: 6338
Online

Joined: 1/19/2004

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 11:12 AM
Post #27043 - In reply to #27042

Yes, it appears to be a bug when running a forward test period with Monthly boundaries, along with Portfolio Simulation set to back test period only. I reproduced & reported it.
^ Top
Jim Dean

Sage
20001000
Posts: 3022

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 11:18 AM
Post #27044 - In reply to #27043

Thanks, Barry

I am still confused about to what degree the PS runs, if any, have to do with how the TC works when you pull a chevron down for a real trade at the HRE, using the Fixed Risk feature.

That's what Tom Helget posted as an oddity, recently - a 33% position taken as I recall.

Specifically:

1. Do the various settings in PortSim have ANY impact on how the Trade Calculator works in a live TradePlan at the HRE? If so, which settings matter and how do they affect it?

2. What OTHER settings in OT have ANY impact on how the TC works at the HRE for Fixed Risk calc's ... such as data periods or BT/FT or strat settings or timeframes, etc etc?

3. Have you determined that there is a need for a bug fix re the TC calc's at the HRE, re TomH's experience?

Thanks
^ Top
SteveL

Veteran
1001002525
Posts: 262

Joined: 8/19/2005
Location: Boulder, CO

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 11:38 AM
Post #27045 - In reply to #27044

Jim,
I used the FixedRisk calc for a couple years. I haven't been using it recently. But, I assume it works (or will work once the Trade Calculator is fixed) as it used to.

From my experimenting and observation, I believe this is how it works:
In PortSim, you configure 3 FixedRisk allocation method parms: 1) Percentage risk, 2) ATR periods, 3) Fixed Loss Level. These 3 values are used to do the Fixed Risk allocation at the HRE.

I believe your Fixed Loss Level and ATR Periods "should" (there is no enforcement) be the same as the ATR and ATR Periods set in your Fixed Loss Stop used by your strategy, else your REAL risk won't match what was assumed in the PS evaluation.

I will be surprised if any other PS settings affect the FixeRisk allocation at the HRE.

Steve
^ Top
Jim Dean

Sage
20001000
Posts: 3022

Joined: 9/21/2006
Location: L'ville, GA

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 11:45 AM
Post #27046 - In reply to #27045

Thanks Steve ...

No disrespect at all intended ... but this is such a crucial thing, I'd appreciate Barry's definitive response to my questions. Tom posted a pretty strange thing that I think needs to be clarified by a better general and official description from Nirvana re how TC works at the HRE.

It makes zero sense to me that TC at the HRE would be affected in ANY way by whatever the last PS run was ... since PS may NEVER have been run ... in fact there might not even BE a strategy active. I'm asking about how TC does its thing with a chevron-pulldown.

For instance, how in the TC does the risk-ATR get set for Fixed Risk, when used w/o a strat or a prior PS run? That seems to be a hidden value that's really important!

Sorry for doing this in your thread ... it appeared that your Q was dealt with already. I should have started another one.
^ Top
Barry Cohen

Sage
5000100010010010025
Posts: 6338
Online

Joined: 1/19/2004

User Profile
 
Subject : RE: Trade Calc allocation varies unexpectedly
Posted : 8/15/2012 2:52 PM
Post #27048 - In reply to #27044

Jim, I do not know the formulas of how the Trade Calculator determines it's values nor how Fixed Risk is calculated, but I would assume if anything changes the analysis for any particular method, that the Trade Calculator will have different results.

I have not determined that there is a bug regarding Tom's thread.

Jump to page : 1
Now viewing page 1 [25 messages per page]
back reply

Legend    Action      Notification  
Administrator
Forum Moderator
Registered User
Unregistered User
E-Mail this thread to a friend
Toggle e-mail notification


Nirvana Systems
For any problems or issues please contact our Webmaster at webmaster@nirvsys.com.