Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA  User Profile | Here's something else to consider, to "balance out" the prior comments. The "prior-history curve-fit boogeyman" WILL BE present, for ANY system or strategy that has been "worked on" at all to improve it. It's almost impossible to avoid some degree of "fit". We need to step back a bit and realize that to some degree, curve-fitting is just part of the game. Sad, but true. That is, even if you create an entirely unique new Strategy, with every component invented from scratch, and base it ONLY on your current knowledge about TA and trading logic, and NEVER do any StrategyWizard runs or CB/NN/GA (NClub tools) optimization ... and you have all OT-parameter optimization turned off ... EVEN IN THAT CASE, there will be a small degree of what could broadly be called "curve-fitting." The reason that this is true is simply due to your extant "common-sense knowledge". For example, "everyone knows" that a MACD(12,26,9) or an RSI(14) or a BolBand(20,2) or an ADX(14) is a "pretty good" setup to work with, when using those indicators. Somewhere, sometime in the past, you or other folks established those "common-knowledge" rules based on observed performance of the indicators on actual charts and/or with actual trading accounts. THEREFORE, any **subsequent** use of those same indicators and param's will be carrying forward that earlier, albeit crude, "curve fit" exercise. Of course this example is pushing it ... but I think it's healthy to recognize that whatever we do, we do for reasons that are based on some form of prior experience and knowledge. That "past history" does not necessarily repeat itself ... but our memory and thinking process does. We can help avoid "FOOLING OURSELVES" (that's the true curve-fit "gremlin") by taking reasonable steps when testing and tweaking, and being very VERY careful to eval those tests and tweaks against truly untouched, truly different data. Prior posts in this thread have provided some broad-brush recommendations as to how to minimize the "fooling myself" pitfalls. [Edited by Jim Dean on 6/28/2010 10:22 AM] | |
