Current location | Thread information | |
![]() ![]() ![]() ![]() ![]() ![]() |
Last Activity 7/22/2015 8:11 AM 56 replies, 5049 viewings |
|
Printer friendly version |
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
As discussed in the OV-Pro seminar today, Mark Holstius and I (Steve Mayo) are working with Ed to map-out the design for a portfolio switcher. Part of that specification is what "evaluation functions" should we target. Here's the list we are currently considering: CAR MDD mCalmar = CAR/MDD, over the test period (not a fixed 3-years as normally done) mSortino = CAR/downside volatility, sans the adjustment for risk-free treasury rate mSharpe = avg period return / stdev of period returns, sans the adjustment for risk-free treasury rate Beta = COVAR(Port CAR, Market CAR)/VAR(Market CAR) mAlpha = Port CAR - (Market CAR * Port Beta), sans the adjustment for risk-free treasury rate Information Ratio = (Port CAR - Market CAR)/STDev of that Excess Return Expectation of Loss= %Losses (1- hit rate) * average of the period (monthly/weekly) losses Expectation of Gain = %Wins (hit rate) * average of the period gains Volatility-adjusted return{ = avg. period return / stdev of avg period return Risk-adjusted return = avg. period return /% losses * avg period loss [Edited by Steve Mayo on 4/10/2014 6:15 PM] | ||
^ Top | |||
George White![]() Member Posts: 5 Joined: 10/11/2012 ![]() |
HOW ABOUT SLOPE OF ema(8) | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
Thanks for the suggestion, George. Hmm. Wouldn't that be better for the "Scoring" method instead? I'm basically asking what you would like to measure on the OUTPUT, that being the "switched" portolio's equity curve (a/k/a, the backtest). I don't think the recent slope of that equity curve would show if it is a good investment. We are planning to use most of the standard "indicators" like EMA, RSI, periodic return, etc. to score and rank portfolios to determine which one to switch to for the forthcoming month. There are hundred to thousands of possible permuations there. So, the evaluation function that I'm describing applies once you generate all those permutations. Obviously, we can't graph every one, so this evaluation function would be a way to sort the output table into something more reasonable. In that light, I think we would want something that evaluates the overall "goodness" of the resulting equity curves, probably something that balances risk and return like Sharpe or the others I listed, although some might just want the lowest possible volatility/risk or the highest possible return without regard to the other. | ||
^ Top | |||
Jim Dean![]() Elite ![]() ![]() ![]() Posts: 1059 Joined: 10/11/2012 Location: L'ville, GA ![]() |
I know this response will not likely be "popular" with the developers because of the data requirements, but I think it's IMPORTANT, so here goes. We need, in a lot of places in OV and OT, etc, a statistical measure of the RISK EXPOSURE that is taken, as a percentage of Entry price, for the various strategies in the various portfolios. That is, if two ports or strats have roughly similar CAR and MDD, I for one would be drawn to use the one that has me exposed to the least degree of true money-at-risk during the trade. The difficulty is of course that this can only be measured by looking at the price dataseries during the trade ... versus the fixed or trailing stop logic in each strategy. And afaik OVest is not "built" to track that kind of thing. Yet. So ... I don't expect this will be done soon ... but I think it's REALLY important, so I wanted to add it in response to your request for ideas. [Edited by Jim Dean on 4/10/2014 7:06 PM] | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
Does anyone have any experience with the Modigliani ratio? First calculate "Excess Return" of the portfolio over the market's return for each period, say monthly. Excess Return = sum of (port month return - market monthly return) M2 = Sharpe x STDev of (Port Monthly Returns - Market Monthly Returns) M2 is basically the risk-adjusted return of a portfolio relative to the market. It is similar to Sharpe and would rank a series of ports the same way but rather than some abstract number, it is measured in units of percent return so it is more understandable. | ||
^ Top | |||
DBokhart![]() New User Posts: 3 Joined: 10/11/2012 Location: Dallas ![]() |
Steve I prefer Ulcer Index and Ulcer Performance Index, similar to Sharpe ratio but it only measures the top half of the standard deviation ---essentially. Things that go up but not down, obviously everything goes down sometime, but it measures how close it comes to the ideal. The problem with just the UI is that a bank CD is perfect but returns are low, so you put in returns also to get the UPI as a measure of persistent moderate to high growth. Ulcer = \sqrt { R_1^2 + R_2^2 + \cdots R_N^2 \over N } UPI = { Return - RiskFreeReturn \over ulcer \, index } http://en.wikipedia.org/wiki/Ulcer_index After Peter Martin wrote his book in 1988 Fast TRack Software was developed to use it to trade Fidelity select funds as I understand it. It is widely used in ETF and mutual fund trading. Its slow, but a great measure of longer term persistency. http://www.fasttrack.net/newhelp/ulcer.asp http://stockcharts.com/help/doku.php?id=chart_school:technical_indicators:ulcer_index http://www.tangotools.com/ui/ui.htm I have the .pdf of Dr Gary Elsner's article and S & C magazine from about 10 years ago if you would like it, he's probably the current expert on it. This article has a better method of calculating than the original Peter Martin version. Ed has committed to putting it into OT when he gets time, (hint Ed) And Thanks for all the work you have done on OV. email me at dbokhart@aol.com if you would like a copy of the .pdf | ||
^ Top | |||
gbarber![]() Veteran ![]() ![]() ![]() ![]() ![]() Posts: 282 Joined: 12/30/2012 Location: Pearland, TX ![]() |
I would like to see something along the lines of those ideas from Jim Dean and Dbokhart. average expected reward/risk at trade entry and average achieved reward/risk after the trade is closed if you then compute the ratio of achieved versus expected reward/risk numbers and pick the ones that are greater than one, you should have a high likelihood of making money. [Edited by gbarber on 4/11/2014 12:19 AM] | ||
^ Top | |||
Eric Severance![]() Member Posts: 23 Joined: 10/11/2012 Location: Incline Viallge, NV ![]() |
I’ve been working on OV tests as well as rigorous backtesting of strategies (in OT, Tradestation, etc.). Steve, I was happy to hear you brought this (measurement topic) up (again) and created a section in the Forum to get a discussion going. A recent performance measurement article I read (re: Measurement, Alpha Ratio, attached) may be relevant to this discussion IMHO. More modern, advanced measurement metrics like this may help us even better evaluate, compare, and rank, (switch/balance) various strategies and portfolios - to have an even better tool in OV. See what you think. Regards, Eric Severance ![]() | ||
^ Top | |||
Eric Severance![]() Member Posts: 23 Joined: 10/11/2012 Location: Incline Viallge, NV ![]() |
A few years ago, in my backtesting, etc., I had been looking for a good way to summarize performance. Sunny Harris, a real backtesting guru, developed what she called the CPC Index - a good tool IMHO, that I’ve been using ever since. The CPC Index (from Sunny Harris’ book Tradestation Made Easy): I attached the chapter (4th page has formula, etc.) from her book where she talks about this… But here’s the formula: CPC = P * PF * R Where: P = Percent Profitable PF = Profit Factor R = Ratio Avg Win/Avg Loss Profit Factor is calculated by dividing Gross Profit by Gross Loss. By definition, a value greater than 1 means the trades have a positive net profit. A CPC score above 1.2 is considered “safe”. Over 2, better. As an example, I ran the calculation on a test of SPY (long only) on a RTM strategy of mine (below): came out well (CPC score of 2.98), which correlates well with this strat’s equity curve, etc.. *** Multiple ways to use this. For one, imagine if we could have the CPC score as a metric. as a tool to rank Portfolios and strategies on, etc. Maybe it could also be run, say once a week or month, on each symbol in a standard list (i.e. SP500), for XYZ strategy, then take the top CPC scoring ones and use those as a dynamic list! See what you think. Another useful performance measurement and comparison tool is the “Student T” test. Perry Kaufman speaks of it in his book on Trading Systems. Can send if interested. Regards, Eric Severance [Edited by Eric Severance on 4/11/2014 2:59 PM] ![]() ![]() | ||
^ Top | |||
Jim Dean![]() Elite ![]() ![]() ![]() Posts: 1059 Joined: 10/11/2012 Location: L'ville, GA ![]() |
Thanks, Eric - I'd be very pleased to see CPC added to the metrics. Also, I can see some nifty uses for the "dependency" metric mentioned in the article. | ||
^ Top | |||
DBokhart![]() New User Posts: 3 Joined: 10/11/2012 Location: Dallas ![]() |
As a forum newby how do I attach files to a post, cant' find info Thanks Duane Bokhart | ||
^ Top | |||
Jim Dean![]() Elite ![]() ![]() ![]() Posts: 1059 Joined: 10/11/2012 Location: L'ville, GA ![]() |
Down near the bottom of the post-creation window, you'll see a bolded-message checkbox "Attach a file to this thread or posting" Check it, then when you click Submit, follow the instructions browse, submit, then back. | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
Duane, once you have saved a posting with an attachment, you can then get the link for the attachment, then edit the post and use the formatting codes to embed the image in the article text, like this. open-square-braket img=http://www.omnitrader.com/currentclients/omnivestforum/get-attachment.asp?attachmentid=xxxx closed-square-bracket It's a good idea to first downsize your image (max of about 700 pixels wide) before uploading. [Edited by Steve Mayo on 4/12/2014 10:20 AM] | ||
^ Top | |||
Pete Taylor![]() Member ![]() Posts: 28 Joined: 10/11/2012 Location: N Bradley, England ![]() |
I have previously done some research into R multiples when reading books by Van Tharp. He basically says that a trading system or Portfolio in our example should be ranked in terms of R multiples or Risk multiples. By doing this you gain an expectancy of what your system or our portfolios should achieve over some future time. He states that the more time or number of trades that are calculated by the system or portfolio, the more accurate the R multiple expectations will be. I was wondering if we could use this to calculate that the R multiples as a way of ranking them for the switching process. As the R multiple is changing all the time as the newer trades analysed coming into the system, we could rank one portfolio against others using this. This could be done to highlight which portfolios are becoming “good” against others by analysing the change over the last few weeks or a month. Obviously we want to get in & trade portfolios that are becoming good, rather than looking back several months later & wishing that we had. There is a video link below, of him explaining this on YouTube but you will notice that he refers to you initial stop as your 1R or 1 unit of risk. However, we do not have a stop on any of the typical strategies that we use. So in his book he suggests that if you don’t have an initial stop but you have a list of previous trades, then you calculate your 1R by just dividing the total amount of losses incurred in a system by the number of losses, which we could easily do. As you will see in the video you then express all you trades, losses and profits in R multiples, for example -1R, +1.25R etc etc. You would then add up the total trades for say the last month and the ones with the highest R multiples could be the ones to use. I would suggest this could be done on a rolling weekly basis over say the last 4 weeks, rather than waiting to the end of the month, as in Ed’s example in the presentation. I know its a pretty basic and easily calculated (even for me!) but thought it would just highlight changes in performance, which all we want to do. Don’t know if this is of any interest but thought it was a little different approach from the normal ones that you immediately think of. If you think that this could be of interest I can post a real example from a portfolio. Youtube link http://www.youtube.com/watch?v=JFuF6M7z1jg | ||
^ Top | |||
Jim Dean![]() Elite ![]() ![]() ![]() Posts: 1059 Joined: 10/11/2012 Location: L'ville, GA ![]() |
Pete's suggestion is simple, and a good idea ... rather than a moving SMA type window, I'd suggest the averaging of losses and wins should be done using EMA iteration for efficiency and to assure adequate statistical basis. Doing this would mean that the "AvgR" would be a function of the time window used, even for overlapping windows ... but all the other metrics are similarly affected. The equation is very simple: Expectancy = EMA(all prof/loss of all trades) / EMA(all losses) ... of course the EMA's iterate only with each new data item, not with each new bar ========== This would be BETTER, if the risk-denominator was not comprised solely of the average of the Losses (when/if/as they occur ... could be zero or statistically insignificant). The BETTER method would be to track Max Adverse Excursion within any trade, vs Entry price (which quite often will NOT be zero). This is a bar-by-bar calc done during the Strat analysis batch run, which I hope would be reasonable to implement. =============== BEST method: the true "at Risk" value is not solely defined by what happens at Entry, or versus Entry (depending on what stops if any the Strat uses) ... rather, it's the Max Profitable Excursion (ie best price hit during the trade), minus the Max Adverse Excursion that occurs away from that price, SUBSEQUENT to that price. A little messier to calc but a lot better representation of how well the trade is "managed" and how the vagaries of different symbols' volatilities impact how "comfortable" they are to trade. I've always felt that this is the best possible evaluation of the true Risk during the trade, IF no stops are set. Of course, if stops are set, the closest active stop, vs the farthest the price gets from it on any given bar, is the proper distinction to use. I hope that I've not caused anyone's eyes to glaze over. THIS IS A SIMPLE CALCULATION ... I'll provide code for it (just a few lines) if needed. The "barrier" to doing it would really only be whether that bar-by-bar analysis can be shoehorned in to the existing Strategy processing. It would not eat a bunch of CPU time, if bar by bar is currently being done. I'm unclear, these days, if the Strat calc process even HAS a bar-by-bar Mark-to-Market granularity or not, but it certainly will have to if OVest is ever going to allow spec of Fixed Loss stops or other kinds of stops. And, please, even though this does not come with some big well known name or abbreviation or article or book attached to it, I hope that it will be given at least 15 min of careful consideration. [Edited by Jim Dean on 4/12/2014 5:39 PM] | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
Interesting idea, Pete. Thanks for suggesting it. Its a question for the programmers, but I THINK that the only historic data that will be available to the port switcher/balancer is the mark-to-market equity curve (EOD equity), not the individual trade data. If I'm right, then it wouldn't be able to average individual trade R's as Van Tharp suggests. But, I'm certain it could do composite reward:risk measures on the mark-to-market equity. the kind of metrics someone would use to compare two mutual funds, like Calmar, Sharpe, Sortino, Ulcer Index, Info Ratio, Alpha/Beta, etc. I know Jim and others have requested intraday stops and similar risk-controls/metrics be added to the live processing cycle. The difficulty on that, I would think, is the same as going to real-time. Being able to process all the user accounts fast enough. I think Nirvana will get there, it just might take a bit more time and cash flow. :-) [Edited by Steve Mayo on 4/12/2014 5:40 PM] | ||
^ Top | |||
Jim Dean![]() Elite ![]() ![]() ![]() Posts: 1059 Joined: 10/11/2012 Location: L'ville, GA ![]() |
Please note ... as long as we LEAVE OUT any discussion about possible future loss-stops, etc, and just go with what is happening DURING a trade, then I believe that if the teensy extra calc was shoehorned in to the Strat loop calc, and then those results were PROPAGATED as a table, through the portfolio combination process, then all that info would be available as an "Expectancy" for the Portfolio. Thus it would be static data by the time PortWiz/Bal got its hands on it. So, maybe this is doable, without major surgery. It would be VERY powerful, if for no other reason that it's so easy to understand for non-statisticians. | ||
^ Top | |||
kmcintyre![]() Elite ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 890 Joined: 10/11/2012 Location: Portland, OR ![]() |
Steve, Just back from a week RVing so I'm late to this party. My input is not so much a risk/reward ratio metric, but general observations and experience. I used to use TC to rank mutual funds by Linear Regression slope. Then I would look at the volatility associated with the top 10 performers. (I used 63 bars - or about a quarter's time - for the LRSlope.) I would switch to the best performing funds that seemed relatively less volatile. So somewhat scientific, somewhat subjective. But that's not really a risk/reward measure. But it worked pretty well for keeping me in better-than-average performing funds. Lately I've been really thinking about my psychological profile as a trader. Much like planning a hike, it's not only about elevation gain and loss, but also about trip duration, miles per day, etc. Risk/Reward is an instantaneous measure. I am interested in a measure of relative difficulty trading over time. I want to account for depth of drawdowns AND durations of drawdowns. Like Pavlov's dog, I want immediate gratification, and I'm willing to give up a little return to get it. So I would like to rotate between portfolios that are giving me a minimum rate of return (as measured by LR Slope) while not making me endure too much pain for too long. (If I can't make a minimum rate of return, hold cash...) I'm really looking forward to the Excel export feature as I would like to work on a risk indicator that captures both depth and duration. I'm thinking something like - tradability index = 1 / (( MDD * X ) + ( AAD * Y ) + ( MBH * Z )) where X, Y, and Z are coefficients to balance the weighting of each factor and MBH is the Max Bars between new equity Highs. (Or maybe ABH for Average Bars between new equity Highs...) [KCM 4/12/14 - the above could also include a measure of volatility, such as the MAX(Bolinger Band width[nbars]). Also, MDD could be over a long period while AAD[nbars] could be used.] Then I would solve for LRSlope * tradability index [KCM 4/12/14 - changed the formula a bit, inverting tradability index so it reflects a positive inclination (vs an untradability index). Used LRSlope versus CAR...] Again, not risk/reward, but an "ulcer index" that meets my trading profile. Not sure if that helps at all... Cheers Keith [Edited by kmcintyre on 4/13/2014 9:43 AM] | ||
^ Top | |||
Mark Holstius![]() Elite ![]() ![]() ![]() ![]() Posts: 744 Joined: 10/11/2012 Location: Sleepy Hollow, IL ![]() |
Good points Keith... and this market (and year coming up?) will probably test that "pain threshold". I suspect you'd need to use the ADD and ABH so you're not thrown off by outliers, but I look forward to your results when the download capability is available. Please keep us posted. Mark | ||
^ Top | |||
kmcintyre![]() Elite ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 890 Joined: 10/11/2012 Location: Portland, OR ![]() |
It dawned on me that I didn't explain why I prefer Linear Regression Slope to CAR or a % gain measurement... CAR (or % gain) is an instantaneous measure that relies on a single start and end point. The LR slope is a measure of trend that relies on (and reflects) all points within the time range specified. Points seldom lie on the LR line. If the start point is below the line and the end point in above the line, CAR is overstated. If the start point is above the line and the end point is below the line, CAR is understated. CAR will seldom reflect the true trend. The shorter the time frame, the more volatile the market, the more chance there is that CAR (or % change will misrepresent the trend. LR Slope is far better IMO. (Use log scale so the slope represents % change, not absolute change...) Cheers! Keith | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
The "Scoring Function" that determines which port to use for the next period will definitely allow applying a linreg -- it was a separate step that I called score transformation in our Excel mockup but it is a native part of Omnilanguage so much easier to implement. After you compute a bunch of possible stitched ports, using a regression line to "evaluate" the permuted stitched ports (to determine which to use in your "switching system") might be a good idea too. Thanks for suggesting that one, Keith. The issue, however, might be how long do you lookback and do you need to compensate for low R-squared values? Looking a a port at the end of, say, 2007-2009, you might have a "U" shape. Depending on what day you measure, you might get a good slope but the correlation will be close to zero. Wouldn't this get into, therefore, needing to give uses a lookback and R2 threshold setting -- that gets too complicated for the causal user, I fear. [Edited by Steve Mayo on 4/13/2014 5:10 PM] | ||
^ Top | |||
kmcintyre![]() Elite ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 890 Joined: 10/11/2012 Location: Portland, OR ![]() |
You are correct in sighting that LR would not be a great tool for determining the trend between 2007-2009. But I think the issue is not with LR, but the fact that there was no single trend to be determined between 2007 and 2009. LR analysis would be very useful in determining that the trend was changing from a strong downtrend to an strong uptrend. The LR slope changed radically during that period. The problem you have sighted is due to sample rate, which I have pointed out in a previous post (Nyquist theorem...) You can't detect a two year trend with a sample rate of two years. Antialiasing occurs. [KCM - put another way, one can't determine what happened during a cycle with a 2 year period by sampling at a rate of 2 years. If one wants to detect 2 year cycles a minimum sample rate would be 1 year. That would not provide much resolution, but would capture the U trajectory the market took at a very gross level.] Anyway, my preference is to use LR slope verses a point-to-point measure such as CAR or % change. Looking forward to Portfolio Switching. Sounds very powerful! Cheers Keith [Edited by kmcintyre on 4/13/2014 11:39 PM] | ||
^ Top | |||
Jez![]() Member Posts: 7 Joined: 3/3/2014 ![]() |
Guys, I can't help much with risk-return measures but a few things I'd like to know about a strategy is what is the expected outcome, worst outcome, best outcome and the probability of a large string of losses. | ||
^ Top | |||
Steve Mayo![]() Legend ![]() ![]() ![]() ![]() Posts: 414 Joined: 10/11/2012 Location: Austin, TX ![]() |
Thanks Jez, Yes, that's along what I was thinking too. Perhaps rolling 1|3|6|12 month returns with min/max and standard deviation....or a frequency distribution on the monthly returns, or maybe a box plot using quartiles or standard deviations. | ||
^ Top | |||
Ger![]() Regular ![]() ![]() Posts: 62 Joined: 10/11/2012 Location: Austin, TX ![]() |
Question: Has anyone considered an exponential moving average EMA for ranking portfolios or even for strategies. For Portfolios, the input data would be the percent profit (loss) of every completed trade taken be the sum of all strategies. For multiple closed trades on any given day, one may have to calculate an average profit or loss for that day. For strategy ranking, one could use the same type of data, ie percent profit (loss) for calculating an EMA for the individual strategy. Advantage would be the the higher weighting applied to the nearest data. Any thoughts? Gerry |
|
Legend | Action | Notification | |||
Administrator
Forum Moderator |
Registered User
Unregistered User |
![]() |
Toggle e-mail notification |