Steve Mayo Legend  Posts: 414 Joined: 10/11/2012 Location: Austin, TX  User Profile | I haven't had time to fully explore it yet, but -- in my opinion at least -- changing the account settings could be a great way to "validate" the robustness of a portfolio. We all tend to "jigger the controls" looking for that little bit of extra performance in our strats/ports. In general, its great that we have this wonderful capability in OV to quickly try these changes, but the downside is that we may simply be over-optimizing. A relatively small change (filtering in/out only a few trades) can have a big compounded difference...but it's meaningless if that change is not something that will carry into future trading. How do we know if our systems are predictive (robust)? We don't have good tools for assessing robustness of our "systems" yet, but here's a simple technique that makes for a good stat. Just run your strat/port/account under each of the various sorting methods and average the return/drawdown across them (calculating a standard deviation and confidence interval would be even better). That helps eliminate some of the filtering error. A second technique, thoroughly discussed elsewhere, is to average returns over different time-periods/market modes. | |
