OmniVest User Forums
OmniVest Pro Tools Beta
28 days of October - Part 2
8 replies, 1038 viewings
|
OmniVest User Forums
OmniVest Pro Tools Beta
28 days of October - Part 2
|
Last Activity 10/7/2016 12:00 PM 8 replies, 1038 viewings |
|
|
|
| Printer friendly version |
kmcintyre Elite    Posts: 890 Joined: 10/11/2012 Location: Portland, OR  User Profile |
It took a while, but I collected PW/EF performance results over a 7.75 year simulation range. The Account settings were all "stock" sans the simulation period. The start date was incremented from 1/1/2007 to 1/28/2007. The end date was incremented from 10/1/2014 - 10/28/2014. The strategy universe consisted of the "stock" (archived) RTM strategies. All "stock" EFs were selected. The PW settings were (100,1,5,4). I'm attaching the Excel 2010 spreadsheet(s) containing the raw data and descriptive statistics I felt pertinent. (I'm sure some of you will be able to slice and dice the data in interesting ways if you care to.) Some observations - 1) Over 7.75 years, the mean CAGR for all EFs, over 588 runs, was 19.26%. That beat the heck out of the $SPX which generated 4.18% sans margin. Even at 200% margin, PW was a clear winner. None of the PW/EF runs lost money over 7.75 years. 2) The best performance over the 7,75 year simulation span was $961,954. The worst performance was $107,729. Quite a range... 3) The Summary sheet collects a lot of data on each EF. (Hide columns to simplify if you wish.) Column Z contains the Coefficient of Variation for Ending Equity. The higher the number, the more dispersion in ending equity. (Remember, the only reason for dispersion is the date in January, 2007 when the lever was pulled...) 4) I could have emphasized the variation in ending equity by graphing profits only. (Subtracting the $100K investment...) 5) The top ranked EF did not repeat from the simulations ran between 10/1/2012 and 10/1/2014. (The original test.) Nor did it match the winning EF from my "101 vs 25" test. Consistency is elusive. 6) The frequency distribution graphs of CAR for the various EFs seem a bit less random. I need to dig deeper, but I believe the graphs will be valuable in picking EFs. 7) Using "(c-c[20])/(GetClose("SPY")-GetClose("SPY")[20])" I might have made $31,499 or $767,846 in profits, over 7.75 years, solely based on which day in January, 2007 I pulled the slot machine lever. 8) Using "-1*CHP(14)" I might have made $15,685 or $315,431 based solely on being lucky. 9) Etc. I want more consistency. Cheers Keith  Attached file : EF Test - 7 year - 28 days - Stock RTMs.xlsx (525KB - 264 downloads) | ||
| kmcintyre Elite Posts: 890 Joined: 10/11/2012 Location: Portland, OR User Profile |
If I want to increase consistency and overcome the wild swings in performance caused by rebalance day bias, (damn, that sounds technical), why not use multiple DPs, all using the same settings, strategy universe, and EF pool, but vary the rebalance date? So if I want to ensure I get closer to the mean return of 19.26%, run 28 DPs with start dates varying between the 1st and the 28th of the month. (Of course the side effect of avoiding the depressed returns is that I also eliminate hope of achieving the manic returns...) OK - 28 x 21 DPs (days x EFs) might be overkill. How much "diversification" might 30 or 40 DPs provide? Will PB and the existing Account level settings allow me to efficiently run multiple DPs, all chasing the same stocks with the same strategies? What would the commission impact be on using massively parallel DPs, all placing smaller bets? Also on the Todo list, what would be the benefit or throwing out the X EFs with the highest CV scores. What about throwing out the Y EFs with smallest mean CAR and/or Calmar? And eventually, can I find a more optimal set of Account settings, as demonstrated over a statistically significant sampling? And someday I hope to leverage the 700+ custom strategies that kicked ass during their day in the sun. But I need a filter to automate my strategy universe selection to make that happen. (Like the simple CAR sort previously requested...) Now if I can just get all this done before I die... Cheers Keith [Edited by kmcintyre on 11/30/2014 10:55 AM] | ||
| Pete Taylor Member Posts: 28 Joined: 10/11/2012 Location: N Bradley, England User Profile |
My thoughts on your tests are as follows: I think that an option to have more frequent runs of the PW/EF and have the options of at least a weekly option, would I feel, be the key to greater consistency. By the way this is not based on any research or evidence. However if you ran your tests & had the option of a revaluation on a daily basis, then the only difference would be in theory, the difference in the first & last few days, being different. The ability to have a revaluation on a weekly basis would, I think, have to have an improvement in consistency. I seem to remember that a weekly option, rather than just monthly, was discuss in various meetings but I haven't heard anything since. I'm think this would be key to avoiding the wider variations of results you have experienced?? Regards Pete Taylor | ||
| Mel Member Posts: 20 Joined: 3/4/2014 User Profile |
If that is so, I see now way to avoid it in real trading. What you seem to be doing now is kind of like Monte Carlo analysis, which can give you a scary view of the variability of a single strategy. Mel | ||
| kmcintyre Elite Posts: 890 Joined: 10/11/2012 Location: Portland, OR User Profile |
I do not believe that simply reducing the rebalance "Interval" to daily or weekly will reduce the dispersion in returns. I think having multiple DPs, each with differing rebalance dates, would cause the Account containing the DPs to have a more predictable, consistent return. Mel, There are a few clear "outliers" in the PW/EF performance data. But that is real world. I don't want to ignore them. (In fact, I want to fully understand and embrace them.) Yes, I think what I am doing is a type of Monte Carlo analysis. Shifting the simulation date by one day allows be to collect data on many runs over essentially the same input data. What I'm measuring is the noise (randomness) of the algorithm - for the most part. Some variation comes from the calendar -> trading day transform. (Some runs are a little longer than others as measured in trading days.) Some variation comes from the price delta of the days that are not shared between runs. But the lions share of dispersion comes from the trading algorithm. And since I keep all Account settings, strategies, EFs, and PW settings constant, the dispersion is attributable to 1) the random walk of the markets and 2) the noise injected by ECA and the EFs. And a little noise can become a roar if the amplifier gain is large. Small changes in strategy selection get locked in for a month, then leverage further amplifies the noise, then compounding propagates the noise into future months, almost like a feedback loop. So each rebalance adds new noise while the old noise feeds back in amplified form. Back to the slot machine metaphor. Slot machines are designed to generate pseudo random results over the short run, but totally predictable returns over the long run. A casino manages the short-term risk by loading up on slot machines. Over a large pool of slot machines, the house gets much more predictable returns. I want to be the house. Keith | ||
| Steve2 Elite Posts: 750 Joined: 10/11/2012 Location: Annapolis, MD User Profile |
The other big source of variability in portfolio performance is equity constraints. These can cause a large ripple effect through the trade history that extends well beyond the difference in starting and ending dates. For example, in the 23 strategy static portfolio I posted in your first thread on this topic, I compared the trade histories on two consecutive days to identify the number of different positions held during the 6 year simulation period. For this analysis, I first assumed that a position matched if the starting date, ending date, and symbol matched. This showed that there were 391 different positions held between the two trade histories. This is way more than the 12 positions that one might have expected based on TPM and the one day difference in starting and ending dates. This is all due to how trades are filtered out or not filtered out due to equity constraints. If one adds number of shares to the matching criteria then there were a whopping 5,544 positions that didn't match. This, of course, is due to the difference in daily account values between the two simulations. One can effectively remove the impact of equity constraints in a simulation by increasing Buying Power to a very high number. I did this for these simulations and found that it reduced variation in daily ending equity by about 50%. So for this one set of examples, it appears that 50% of the variation is caused by equity constraints and 50% by compounding effects related to the difference in starting and ending positions. Since a dynamic portfolio is just a set of static portfolios strung together, I think that account equity constraints are a significant cause of variability and I wonder if there is truly a predictive way to periodically re-balance strategies to achieve higher returns without running a large number of simulations and looking at the distribution of returns (which is what I think you and Steve Mayo are saying). Steve [Edited by Steve2 on 12/2/2014 4:34 PM] | ||
| kmcintyre Elite Posts: 890 Joined: 10/11/2012 Location: Portland, OR User Profile |
Good input. So perhaps I could get more consistency by changing the Account settings to force smaller trades, but more of them. Realistically, my trading equity is constrained. I've previously tried bracketing position size from 2% - 30%, and found 10% seemed to work best in terms of CAR, Calmar, and EE. But those tests were not very exhaustive. Maybe the dates I picked just worked out that way. I was planning on moving to Account settings soon. I can't test every permutation. I think Trade Sort Order and position sizing are worthy of more scrutiny. (Price x Avg Volume Descending was the winner in my previous tests, which were relatively cursory.) I just can't flip a few levers, turn a few knobs, get a big number and have the confidence to trade anymore. But I don't want to suffer from analysis paralysis either. Keith | ||
| Steve Mayo Legend Posts: 414 Joined: 10/11/2012 Location: Austin, TX User Profile |
Conversely, a distribution graph...which is sort of what we are doing with your rolling returns experiment (although that's still a bit flawed due to its repetition of the mid-period results), gives a much better estimate of what to expect in the future. It shows an historic range, and that's really what you want to know, although there still is no guarantee that history will be repeated. Is there variability in results when you change the parameters. Absolutely! But aren't we better off knowing what that variability is rather than falsely thinking one set of metrics (CAR, MDD, Calmar, etc.) is really what we are going to get in the future? As Steve Fox points out, the results in OV are impacted by a lot of different factors, not just start/stop date. That is why I keep stressing that looking at short periods of time is risky. You need lots of trading history to get a good representative sampling of the potential returns. The market does show some covariance from one day to the next and that's why a system like PW/PB or OT that frequently re-optimizes does seem useful for improving short-term returns...but you then have to test that dynamic system over a long(er) period. And, I agree, we really need PW/PB to run weekly as the predictability of a system drops quickly when you go out beyond a few days from the signal. | ||
| Steve2 Elite Posts: 750 Joined: 10/11/2012 Location: Annapolis, MD User Profile |
I don't think trying to force smaller trades will be that helpful. The experiment I did was just to confirm for myself that equity constraints introduce variability and that there is no way with a single simulation to get an accurate prediction of likely returns. You could do this by reducing strategy allocation percentages but that will also really drive down % Invested so returns will be low. While I suppose you could do this to compare portfolio performance, in the end you would need to crank allocations back up to achieve reasonable performance and that would reintroduce the equity constraint problem. As Steve Mayo says, each equity curve is one data point and the only way to really get a feel for probable returns is to do a bunch of simulations and look at the distribution of the returns. Steve |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend(un)/Freeze thread | |
Toggle e-mail notification |
