Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA  User Profile | Context is important ... I've been trying to gradually build a SymData.otd file that holds the past year's intraday data for a variety of bar-lengths, so that I can archive it, and then re-use it later for backtesting etc without having to reload hundreds of thousands of bars of data later on, for the same period. I'm doing it for the symbols in the Nas-100. It is comprised of eleven different bar-lengths. Re the numbers on the right of the equals-sign ... the 255 days calculates out as 6640 15m bars for 255 days, but the actual number of bars LOADED in the chart when 6640 is input in DataPeriods turns out to be 6616. I don't know why, but I've recorded those "actual avail" bars as the rightmost number. 390m = 255 / 195m = 510 / 130m = 765 / 78m = 1277 / 1274 65m = 1532 / 1510 60m = 1660 / 1655 39m = 2550 / 30m = 3320 / 3309 26m = 3831 / 3818 15m = 6640 / 6616 13m = 7662 / 7545 10m = 9960 / 9923 6m = 16575 / 5m = 19920 / 19844 3m = 33200 / 33072 2m = 49800 / 49607 I've learned that OT doesn't just save the 1-min bars for the historical period (which is the way that Worden does it) ... OT actually calculates each individual bar-length type, and stores it ... thus making SymData.otd MUCH bigger than I would have expected, for this collection of data. So be it. Anyways, I am ALMOST done ... just THREE SYMBOLS for ONE BAR LENGTH away from reaching my goal ... which took the better part of yesterday to accomplish. My SymData.otd file has reached a whopping 727,484 KB ... and all I'm missing is the 2m bars for AAPL, SPLS, and NIHD. The SymData.otd file gets to that size when I start up OT with the Nas list ... but during the time it is loading the data in the symbols, it throws the error. However, every chart ends up visible with all its bars (except the three symbols I mentioned). THEN, when I close out OT (and get another error and log-submission screen), I notice that the SymData.otd file size DROPS to 467,328 ... presumably it has thrown out a bunch of data ???? So, in helping me with this problem, please also *** check with Cose *** re any internal limits. Please expressly pass on the list I provided above for the 11 timeframes and numbers of bars I want. To build this SymData.otd file, I've created several different profiles and loaded them up one at a time: 5m, 26m, 130m 3m, 15m, 78m 2m, 13m, 65m 10m, 30m, 60m ... all also including 255 days, which is the only voted timeframe. The first three of these are the most important to me, since they represent multiples-of-five between each increment. All listed increments divide evenly into a 390-minute day, btw. If it turns out that I've hit some kind of "hard limit", then I'd be OK with leaving out the last profile-set (10,30,60) ... but I don't want to have to restart this incredibly long and boring process unless I *fully* understand the limiting factors. So, please alert Jon to this, and please get Cose involved with the entire text of this posting. Thanks! Another way of asking this, in a nutshell: What is the structure of the SymData.otd file, and what is the most efficient manner for a user to do a one-time accumulation of past intraday data for a variety of bar-lengths, to help with later backtesting in offline situations? PS ... one reason for wanting this is as I mentioned earlier ... to reduce long download-from-server times, repeatedly, when I want to do extended backtesting. ANOTHER reason is that I often work in areas where the internet speed is either very slow or nonexistent. [Edited by Jim Dean on 1/13/2010 8:31 AM]  Attached file : OT.log (37KB - 520 downloads) Attached file : OT.otd (420KB - 517 downloads) |
