OmniTrader Archives
OmniTrader 2012 Technical Support
Tradescope Bins & Segments
24 replies, 4456 viewings
|
OmniTrader Archives
OmniTrader 2012 Technical Support
Tradescope Bins & Segments
|
Last Activity 3/18/2022 1:29 PM 24 replies, 4456 viewings |
|
|
|
| Printer friendly version |
JamesR Veteran  Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK  User Profile |
Is there any more detailed information on Binning and Segmentation? I have gone through the video that came with Tradescope and have read the user manual but they are both very light in information. This has left me wondering what is the effect of different 'binning' settings. I get the impression that for each variable the result of the calculation will be put into a 'bin' dependent upon the settings you choose, but I don't have any idea why that matters or what the effect is - I have been trying to play around with a small data set to get an idea, but that hasn't helped much. This is further confused for me when the variable is a boolean - if the result can only be yes or no, can there only be 2 bins regardless of the setting I choose for that variable? On segmentation I understand Equal Size and Custom Boundaries but don't understand Equal Sample - I keep reading the user guide definition which says boundaries are selected such that there is an equal probability of a chart moving past each section - no matter how many times I read it and how many times I play with data, I can't get a clear idea in my head what that means!! Is there a link between Binning and Segments? Happy to read about it if there is information available or an explanation would be greatly appreciated otherwise I am going to struggle to trust the Tradescopes when I don't understand what I am asking them to do! Many thanks as always [Edited by JamesR on 2/5/2012 8:55 AM] | ||
| Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
First, a disclaimer ... I'm not privy to any internal documentation that you don't have. So, these comments are "educated guesses" ... "Binning" is a very common way of dealing with disparate datapoints, to try to make sense out of them. From other Nirvana-responses on the forum, it appears that TradeScope uses the most simple form of binning, rather than a true statistical analysis based on things like root mean square and variance and other somewhat-mystifying terms. It might help to think of TS bins as "stretchy buckets". Your settings-inputs, directly or indirectly, determine how many buckets there are, and how big they are. I'll return to that in a moment - for now, just think of a row of buckets, side by side, with numbers marked on them. TS uses these buckets to hold "scores" (grades) of events that have occured in the past, to help you evaluate the likeliness of a similar event producing a positive or negative or neutral score in the future. The nature of the "event" is related to the "boolean variable" that you define, which as you say is either true or false. What that boolean variable does is define the "state" of the price / volume at a point in time, relative to whatever the rule has in it. That state is either True or False. There are two "events" that we can glean from the rule ... WHEN it switches from T to F, and when it switches from F to T. TS (as I understand it) uses those events to trigger the beginning of a measurement - or rather, when it flips from bull to bear. From that point forward, the price rises or falls. TS measures HOW MUCH it rises or falls, for each instance of one of those historical events, and stores that information in a normalized form of some sort ... either a ratio vs price or vs ATR or something else. So, what is "remembered" for each of those T-to-F and F-to-T events is how much price moved on a relative basis after that. Furthermore, TS can adjust its "price-move capture-calibration" to a particular window of time, presumably suited to your trading style. It can look for the net price movement over N-bars since the "boolean event" occured that I just described. So, the "remembered" relative-percent-price-move-since-the-boolean-flip is can be relative to a particular time window. OK - using this method of measurement, TS looks over the past history of each symbol in the focus in the defined backtest period, and collects a huge pile of percent-price-moves. Picture a big pile of index cards, each with a normalized price-change number written on it. NOW we get to the binning thing. That big pile is not useful to us ... so TS sorts thru the pile ahead of time, to let us know how the numbers on the index-cards are "distributed" (from very positive to very negative price change). When you input Equal Size, TS makes the RANGE of price-change represented by each bucket be the same. For example, if the most-positive price change is 200% and the most-negative price change is -100%, and if 30 buckets are used, then each bucket is "assigned" to a price-change range of ten percentage points = [200 - (-100)] / 30 = 300/30 = 10. With that particular collection of buckets (all same size re price-change), TS then starts going thru the big pile of index cards and flipping each of them into whatever bucket defines the range that includes the number that's written on that card. When it gets done with that process, it COUNTS HOW MANY CARDS (events with a given normalized-price-change) are in each bucket. In theory a bucket could have anywhere from 0% of the total number of cards, to 100% of the cards - of course normally there will be a spread of results. That spread is used to paint the TS thermometer on the screen. It's not clear to me (sometimes) why the boundaries marked on the screen are what they are ... but their location and the percentage associated with them are based on the counts of index cards in the various buckets. NOW, let's think about the EQUAL SAMPLE case. In the Equal SIZE case, we looked at the total numeric RANGE of normalized-price-change for the whole pile of cards, to determine the "range-subset-label" of each bucket. But for EQUAL SAMPLES, we start off by COUNTING THE CARDS, regardless of the numbers written on them. Let's say we have 500 instances of boolean-event-change that occured in the historical study. If we use 25 bins, then the Equal Sample method tries its best to "size" the buckets so that there are TWENTY CARDS in EACH bucket ... ie equal samples. How does it do this? It takes the big pile of index-cards and SORTS THEM out ... puts them in a long row with the most-negative-percent-price-change on the left, and the most-positive-percent-price-change on the right of the row. Then, TS "walks along the row", picking up cards as it goes. It puts the first twenty cards in the first of the 25 buckets, and the next twenty cards in the second bucket, and so forth. Each time it gets to a boundary, it takes a look at the VALUE WRITTEN on the last card for one bucket and the first card for the next bucket, and averages that value. That becomes the "range-label" for that bucket. Sometimes, using this method, you get a very predictable set of buckets ... sort of a "bell curve" (or a version of the bell curve) ... where the outermost high/low buckets cover a very wide range of price-changes, and the centermost buckets cover a much tighter range of changes. That occurs when there are fewer cards with hugely-positive or hugely-negative changes ... which is "statistically typical". However, there can be situations where a given boolean-state-change definition defines a trigger that has LOTS of big-positive and/or big-negative price-changes, with relatively few in the middle. Those are the "true-gold" combos ... since they identify excellent trading opportunities. Of course the trick is to figure out the right boolean-rules ;~) Read through the explanation above about how TS examines the buckets when they are full, and paints the thermometer on the screen. Finally, we have CUSTOM BOUNDARIES. I hope by now you can guess how this works. In this situation, TS does not look at the NUMBER of index cards (as it did for Equal Sample), nor at the RANGE of values written on the cards (as it did for Equal Size) ... it just looks at WHAT YOU TOLD IT. You get to arbitrarily define the normalized-price-percent-change boundaries for each bucket. Then TS goes thru the pile of index cards and tosses them into your special buckets ... and paints the thermometer as described before. Whew! Long explanation ... I hope it helped. And, once again, please note that I could be wrong in some parts of this description. I'm simply drawing on an understanding well-established conventional "binning" (ie "bucket-ting" or "segment-ting") methodologies, and trying to connect-the-dots to what TS is doing. If I'm incorrect, I hope that Nirvana will clarify. FINAL POINT ... if you choose Global vs Symbol-by-Symbol, then the most important difference is that you get a LARGER PILE OF INDEX CARDS to work with. That's very important in statistical analysis. The bigger the pile, the more reliable the results. If Nirvana had chosen to use true statistical methods for this, rather than the "arithmetic binning", then they could have output a "degree of confidence" for each of the percentage-up and percentage-down values. I think they made the right choice in keeping it simple. Firstly, it makes it easy to explain (see above). Secondly, it does not produce confusing "confidence" results for SMALL piles of index cards. Some users will chose to use short backtest periods, and/or very complex boolean rules which won't produce a meaningful sampling of events. In those cases, the confidence-values might be misleading. So, should you always use Global? In addition to it being more statistically robust, it also provides (in theory at least) somewhat faster processing time for the "training" of the TS bucket-brigade. ON THE OTHER HAND, if the symbols in the focus list you are using have individually-consistent but WIDELY different personalities from one another, then a that big pile of index cards has apples and oranges and banannas and strawberries all mixed up. The reliability of the predictions would in that case drop off. Nutshell recommendation: 1. if you have a disparate group of symbols, and plenty of historical events to draw from (re backtest period and boolean spec's), use symbol-by-symbol 2. if your symbols are related ... ie part of the same Industry Group or pre-filtered by Group Trader to find ones that are often "in sync", then use Global. 3. if you can't use much history (for some reason), then use Global but ALSO break your focus list up into sub-groups of stocks with similar personalities. JDref-711#3410 [Edited by Jim Dean on 2/5/2012 1:01 PM] | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
First and formost thank you very much for an excellent, clear explanation which I have tucked away for future reference - your efforts are very much appreciated. I do not want to draw you into a long time consuming discussion on this one topic but I do still have one 'gap' in my understanding and this relates to the bins versus segmentation issue. So I am going to try to explain what I think is happening and see if you laugh!!! :-) Taking your binning explanation, I notice that I can set up different binning scenarios for each variable in the TradeScope. So I effectively end up with rows of buckets, one row for each variable and each row having different lables (ranges) on the buckets. I can then seperately set up segments for the Thermometer on my chart. These may or may not match any of the bin setups I have chosen. A quick aside - in the Tradescope presentation it states that '...the more variables you add, the more hits you will get..' from that I am assuming that each variable added creates an OR condition rather than an AND condition. Effectively the Tradescope is looking for a situation where condition 1 is true OR condition 2 is true etc. (if it is an AND condition you would expect to get less hits with each variable added). So going back to my rows of buckets, I am assuming to build the Thermometer, I do similarly to the process of binning, except that I now look for any buckets from any row that fits into the segment in question and build up the segment from these buckets. If I am a million miles off the mark, feel free to publicly shred me !!!! Again - very many thanks for the assistance thus far. | ||
| bvandyke Member  Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
Bill | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
Variable 1: c> sma[50] Variable 2: c< sma[50} i was curious because when i tried this it worked, and i wondered why, so i checked with sales. the upshot was that on each day TS looked to see if the Close waa above or below the 50 day SMA. if the close was above, TS looked at prior history when this was true and determined the probability(likelihood) that the price would go in the direction of the "thermometer" on the screen. if close was below 50 day sma on the day in question, the TS looked back at history and determined the probabilities that the price would move as one sees on the "thermometer" on the screen. also, i was told that if you have 2 variables when each one results in a "number", not a true or false condition, then on a given day TS looks historically to see when Both those numbers had occured in the past. If both those numbers had occured in the past(and occured today) TS again determines based on the history the probabilities of price moving as shown on the "thermometer" on the Screen. if i've misunderstood what i was told or it was not quite right, then OT can correct me. Hope this helps some. Bill | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
TS also offers System entries as triggers, which define "events" and fit naturally into my prior explanation. You can specify a tolerance input for Systems, to make their signal "last longer" ... this is important if you combine several metrics/rules together ... otherwise you won't get an overlap. The manual is silent about the And / Or thing. My ASSUMPTION, for the Boolean and System cases, has been that "AND" is the operative rule when multiple metrics/rules are provided. It would be natural to assume that if "And" is used for two of the three types, then it would also apply to the third type (measurements). However, we must think further about how the measurements are evaluated. Again ... my comments here are educated guesses. I hope that Nirvana will jump in and correct any errors I make, and elaborate on any weak spots. To understand how the Measurements are handled, it's important to understand the "binning" thing that I described before. It would be absurd for TS to use the PRECISE value of a measurement today as a reference-point to search back over history, since there would be very few if any days with the same six-significant-figures-of-precison value as the measurement happens to have, today. So, what TS (presumably) does is to examine the FULL RANGE of the measurement over the historical period, then break that down into "buckets" like I discussed before (ie bins) - to figure out what the labels on the buckets should be (ie what range of measurement values fall into each bucket). Once it knows the bucket/bin-ranges for that measurement, it determines which bucket today's value is in (ie a very very "rounded off" value, not precise at all) ... and looks for past days where the measurement fell into the same bucket. This process is done for each measurement independently, since they all likely have different historical ranges. That is, different bucket-labels for each measurement. If you have several measurements defined in your rule/metric set, then I would PRESUME that an "AND" condition would apply ... although recent posted comments seem to indicate that "OR" is used. If "And" is used, then a given historical day "qualifies" to evaluate price-action potential if ALL of its measurements fell into the same buckets as those measurements are in, today. However, if "Or" is used, then an historical day would qualify if ANY of the measurements fell into comparable buckets as today. Nirvana really needs to qualify this. If some metrics/rules/variables use AND and some use OR, then we need to know this in order to intelligently construct/modify our own TS definitions. I hope these guesses and explanations are helpful. | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
i really appreciate your thoughts for they do make intuitive sense. i'm pasting below the part of the email i got back from Sales in response to the question we're all asking about non-Boolean conditions. Below was the reply from Nirvana. I have no idea where Binning comes in. I sure hope we can get clarification soon. =============================================== "Hi Bill, I spoke to Jeremy Williams who is our mathematician and helped create the Trade Scope. I gave him these 2 scenarios we discussed and he did confirm our assumptions on both. Second senerio: The Three Day outlook uses 2 variables 1. CCI(5) 2. c/EMA(250) In this scenario it is looking for the number of times in the past that both of these values are the same as they are today. Both must be true at the same time. I hope this helps answer your questions." | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Nutshell: the more metrics/rules/variables you specify, the fewer historical hits will be found - but the projections will presumably be more reliable. | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
Just to confirm, I listened to Jeff's seminar again this morning and clearly my memory is what is at fault - he does actually say '..the more variables, the less hits..' which does indicate an AND condition not an OR - which makes sense. I am going to do a little 'Lab work' with Tradescope over the next few days - I have created some symbols of my own with very limited data in them - basically I am going to set up the data so I know how many hits there are for my scope and where they are, I can then look at the TS output and compare that with the data to try to glean a better understanding of what is going on - if that turns up anything of interest, I will report back. Many thanks to both of you for the assistance [Edited by JamesR on 2/8/2012 2:55 AM] | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
Attached are some notes from my experimenting. If anyone can throw any light on these issues I would be very grateful. I am concerned that if I can't make sense of the output against even these simple criteria, I would struggle to do so with a more complex scenario. [Edited by JamesR on 2/9/2012 8:16 AM]  Attached file : Tradescope - Experiments 090212-1.pdf (1048KB - 767 downloads) | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Great documentation. I wish I could help but my guessing-ability doesn't extend that far. I've written heads-up notes to both Barry and Jon about this. They may not have specific answers at hand, but hopefully they can dig them out for us. Another factor that may influence your results is the need for a "warmup" period. The ATR for instance is not known immediately. There also is often a little "slop" re transitional bars in loops or if-statements that could throw things off. So, if you're a glutton for punishment, I'd suggest you retry your test but use 270 bars instead of 27 - that will make the warmup and slop bars have a much smaller influence on the results. | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
Many thanks for taking a look. I did play with different ATR periods and that didn't seem to affect the number of hits - just the way the scope actually displays them. I will do one quick one over multiple bars now just to see if it differs but will have to wait until the weekend to do a more thorough check through. | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
I changed the back test period to run 01 Jan 11 to 07 Feb 12 I set up the TS for 10 day outlook with single boolean looking for close = 1.58 (which is true for all bars). Run the scope and it reports 273 hits ignoring the current bar that is (187 - 10 - 4 = 173 ) which seems to still fit my previous result. I then changed the current bar to NOT be 1.58 and also did the same for several bars back in the data - rubuilt the scope. See second output - I still get 273 hits - no idea why!! Attached file : TS - Experiment (288 bars) - 1.png (50KB - 538 downloads) Attached file : TS - Experiment (288 bars) - 2.png (46KB - 582 downloads) | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
i created a Focus List with just ONE symbol(PCS). Using Omnidata, the stock never went above 40 historically. There are only approx: 4.5yrs. of data so about 1200 bars. i then created a TS with only one simple variable: c > 100 (of course this never occurred historically). i ran the TS and got a result of 95 occurences!! THEN, i went back to the Profile i normally use which has 15 different stocks and includes PCS. i ran the same new TS that i had created and got 1,206 occurences!!! I don't think any of us can figure this out, so as we are using the TS somebody from Nirvana Please help us understand this!!! Bill | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Thanks for your detailed tests. Something sounds pretty wacky here. I have emailed Angela a heads up about this. | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
after thinking about it more, when i ran the TS i created above in the Profile that had the 15 stocks, since it was never true (c >100), i believe TS may look at that condition on the current (last) bar.....and since it's not true in my case of that bar....TS determines a set of probabilities on the graph some way. my getting the over 1,200 times is probably the total #of days that were shown on the graph. whether the Close was above or below 100 (again it was in reality always below 100)....TS calculates the probabilities on the condition today (above or below 100). i reversed the variable and made C < 100, and got the same #of occurences and the same probabilities on the TS graph. what really puzzles me, is why when i created a separate Focus List with only the PCS symbol, i got a dramatically different #of occurences as i mentioned in my prior post. Bill | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
It sounds as though you are seeing similar anomalies to the ones I am seeing. I assume you did have the scope set to Symbol - not global - in your second focus list? I have attached one final document (I won't do much more now until I hear something to explain what is going on) - but this document illustrates my problem - there are 3 symbols in the list - the first has 280 identical bars, the second all bars are identical but the last and the third is the same as the second but with a few bars earlier in the data that match the current one. The situation that gives me the biggest number of hits is again the one when the TS criteria are not met (here I am simply looking for current bar close being greater than previous bar). Let's see what the experts say - maybe I am just doing something wrong! [Edited by JamesR on 2/13/2012 4:15 AM] Attached file : Tradescope - Experiments 120212.pdf (288KB - 577 downloads) | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
here is my understanding of how TS works and if i'm wrong then Nirvana can correct me (and i may be wrong :) ) in my example above i have one variable: c > 100 (which will always be false because the close never went above 40) i got 1206 occurences because i believe that's the #of days TS is looking back and in this case includes all previous days. the result is TS finds it either true or false for each day prior and thus 1206 bars. how it determined the probabilities it comes up with i have no idea. now if i add a second variable like c/sma(10), i have TWO variables, and the 2nd variable is a Measurement which will always result in a number. i did this, ran the TS i created with the 2nd variable added and got only 239 occurences. I believe in this example TS is looking today to see if Variable 1 is true or false, and determining that Variable 1 where C > 100 is FALSE. Then TS looks at the 2nd variable which resulted in some NUMBER, and TS looks back to see how many times the 1st variable was false on a given day AND on that same day the numerical value of my second variable is also equal to what it is Today. assume that today my 1st variable is false which it will be, and my numerical result of todays 2nd variable is 5. then TS looks for how many days in the past both variable 1 was false and the result of my variable 2 equals 5 on the same day. I'm like Jim, that 2nd variable bothers me some in looking for a past match, but if i'm wrong somebody please speak up. I'm still have no answer for why when i create a 2nd Focus List that consists only of 1 symbol for the same security PCS in my example....then when running C > 100 as the only variable i get only 95 occurences. This really puzzles me. Bill | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
"when variable equals five" … probably should say … "When variable's value is in the same bin/bucket as today's" See my prior explanation. Of course this does not address the confusing results. Just a minor correction. | ||
| bvandyke Member Posts: 36 Joined: 4/12/2011 Location: Dallas User Profile |
i hope you're right and maybe Nirvana will provide more clarification on Binning as i'm still not clear on it for sure. Bill | ||
| Angela Duran Elite  Posts: 729 Joined: 11/17/2003 User Profile |
Jeff took a quick look at the examples you posted and thinks this can be easily explained. In your example you used 7 as an ATR value with a 5 day outlook. At the beginning, 6 bars are not used since it needs 7 to calculate the initial ATR value. The other four are thrown out at the end due to the 5 day outlook. | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Please ask Jeff to look at the other examples provided later in the thread, just a few posts ago. There is an Experiments pdf and discussion that shows the explanation you just provided is not sufficient by a long ways. No criticism intended ... it's a messy topic. Thanks | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
Many thanks for your response, it is good to know someone is looking into this because at the moment I feel I cannot use Tradescope as I simply do not understand what the output is telling me. You are correct, I did use a 7 Day ATR and a 5 Day Outlook and discovered, after much trial and error, that the number of hits reported satsifies the relationship NumHits = NumBars - Outlook - 4. So I had already taken account of Outlook when I raised the query. Also, Jeff's initial answer does not add up either. There were 26 bars of data so with a 7 day ATR and a 5 day Outlook you would expect 26 - 6 - 5 = 15 whereas Tradscope actually reports 17 (By my relationship 26 - 5 - 4 = 17). Suspicious that the 4 was somehow related to the Outlook (of 5) I started changing the Outlook and leaving everything else the same. e.g. Change the Outlook to 10 - by Jeff's suggestion I should get 26 - 6 - 10 = 10. Tradescope actually reports 12 (again by my relationship 26 - 10 - 4 = 12). It may suggest a different relationship definition of NumBars - ATR - Outlook + 3, but I don't think the suggested answer works and there are also some other things going on in the examples that are definately not resolved by this answer. I am more than happy to accept and believe that I am doing something wrong or misunderstanding, but others on this forum that know an awful lot more than me seem to be equally confused, so another, more detailed, response would be very much appreciated as I would really like to be able to switch Tradescope back on! [Edited by JamesR on 2/24/2012 2:43 AM] | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
[Edited by JamesR on 6/22/2012 9:34 AM] | ||
| JamesR Veteran Posts: 216 Joined: 10/4/2008 Location: Gloucestershire, England, UK User Profile |
Nirvana Team - is this going to be responded to or has this slipped off of the radar? Many thanks. |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
