Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA  User Profile | Y'know what? I'm wrong and you are correct, re the broad market indices. Those don't need special historical function calls since they are actual SYMBOLS. So, by using GetClose("$VIX.N") in a formula that compares it to a threshold, for example, you CAN do filtering or ranking or allocation derived from that index, which is valid historically. You can. And, I'm sorry for creating confusion about this. I've corrected my prior post. ======== however ======== The backtesting issue IS important, for "static Lists" (ie symbols associated by you, with a particular market state) ... or if you're not using ATM, then any static Focus List based on some kind of external-to-OT filtering/selection procedure. For example, Eric Severance gave a short presentation at the Bash regarding the value of using Symbol Ranking based on the a well-known publication's "Growth" list of 50 symbols (name is made up since the source of the info is strictly copyrighted). What wasn't stated (as I recall), is that this also presumed the FL symbols are also *filtered* by whether or not they are on the Growth list. He and MarkH nicely demonstrated that using the Growth-ranked order in lieu of the default "alphabetic" order for selection of symbols that have signals on the same day, provides a definable edge. N was convinced, and they plan to incorporate that ability in OVest ... that is, if you upload a List, it won't auto-alphabetize it and lose the sequence of the rankings. This will be a BIG plus in OVest, I believe. OVest of course supports uploading Historical Lists ... that is, huge text files with lots of strung-together "sets" of symbols, for sequential dates in the past. For example, that "Growth" list comes out once a week. Eric has painstakingly gathered a lot of past weeks to create the file (which btw he cannot "give out" to others due to *very* strict copyright protection - they can and do take people to court about it). Aaaanyways ... this means that OVest has the ability to HISTORICALLY TEST the value of that Growth List's rankings (and the symbols on the list, for that matter), to demo how much going that route improves things. Hurrah, OVest! Unfortunately, N long ago decided not to support Historical Lists for OT/VT ... just too messy for them to deal with, I guess. Fortunately, I'm just about finished development of a comprehensive, robust tool that *will* fully support Historical Symbol Lists in OT/VT. Contact me privately for more info. So, in OT/VT, when your strategy or method is "geared" to the use of certain symbols for a given Market State, or certain symbols in the Focus List, and if that symbol-list is defined by: 1. an external algorithm or entity, such as a guru list or ETF 2. a periodic reshuffling of what symbols are on the list, and/or their order ... then, in that case, since OT (natively) does not provide a way for you to vary the makeup &/or order of the symbols on the list from week to week (or some other frequency) ... that becomes untenable for valid/useful Analysis. With that in mind ... Basic presumption for this all to matter: You are going to use optimization, or formal backtesting via Analysis/PortSim stats or via SW ... to decide what Method-variant works best. I think that virtually everyone does one or more of those things, but if you are the exception, then this "advisory" doesn't apply. If you are doing one of those "pick and choose based on statistics" methods mentioned above, then it's likely that (for daily bars) you're looking back at least a year and possibly more, at some point along the way. Remember ... unless you specify otherwise in the Strategy Test Settings, OT will run the stat's for all the bars you have loaded at the time. So, that's the "time window". Now ... if the List of Symbols you are using is the "most recent" (ie just before the HRE) state ... and if you run a 3-year (or whatever) backtest on that "static list" setup ... then there are two problems that arise: 1. in the recent past, the fact that you have future knowledge of the "successful symbols" builds in a crystal-ball rosy-glow bias. Ungood. 2. in the majority of the past before that, presumably the "HRE at the time" list of symbols has changed frequently. In fact, some of "today's" symbols might not even have existed at some point in your test window. NOTE: this applies even to basic Indexes such as the SP100, DJ-30, Russell-1k, etc ... all of which are updated typically once a year or more. Using the current list BIASES long backtests to show better results, since those outside agencies (SP, DJ, Rus) go through a process of selecting "strong" stocks. CONCLUSION: Static Symbol lists based on outside sources that change over time create ERRORS in the analysis results. The more frequently the outside source changes the Symbols in the list, the greater the error will be. And that's why I've spent a huge amount of time solving the problem, so Historical "sequential" external-list changes can be properly modeled in OT. [Edited by Jim Dean on 6/14/2018 2:03 PM] | |
