For the things you are saying the capabities from this model can come close to the ones from trendingBot. Although the only way to know it is by testing it. Propose any set and we can try both approaches. Or better, tomorrow I can check the raw data for the case study you are referring, feeding in my program and check the result.

Nevertheless, note that a capital difference between both (before confirming that actually the calculation capabilities are equivalent) is that trendingBot does not need to be warned about something getting more or less complex. Its algorithm is capable of dealing with any set (the simplest or the most complex one) an give a SENSIBLE answer; overfitting is just the consequence of a stupid understanding: trendingBot is capable to understand (= is not stupid = seldom shows overfitting). The fact of taking into account all the variables every time does not mean to be incapable of dealing with them in a proper way (= avoiding overfitting at any cost).

You can use the contact form in our webpage and let me know about some way to contact you or propose a public forum to share the results or whatsoever you propose; and we can test both programs undertanding capabilities (from some statement in the link you gave making a reference to a really high number of independent variables, drives me to think that this program’s algorithm is not equivalent to trendingbot’s (around 20 independent variables would virtually saturate any nowadays computer (and would take really long time to be processed)))… Nevertheless, I don’t want to show an attitude equivalent to the one from those I have critised: I want to try it.

And this is the last post I will write here (except for the case of proposing any other public forum)

I have answered the posts including ideas I considered much important to make clear in order to understand the whole picture (and some others just having a so censorable attitude that I wasn’t able to refrain from saying something).
Thanks for all the nice comments, feedback, advices, “i understand your situation” statements…

GAOTD has provided me an excellent opportunity to get noticed and learn a lot and I can not pay them back by continuing writing things here as it would be a standard forum, distracting the attention from their main bussiness.

Anyone can contact us directly and if you are interested in make it public somehow, we are completely open to any suggestion (going to some specialised forum or creatin one for our webpage…).

Have a good day and thanks to everyone.

1. The output is in the form of an equation that is completely deterministic, without any estimate of predictive uncertainty. That is, no statistical prediction intervals for the output (dependent) variable.

2. Independent variables are assumed to be totally uncorrelated. (Correlations among independent variables can result in highly unstable models.)

3. It is unclear as to how TrendingBot “decides” when it has chosen “enough,” but not “too many” independent variables. (One can increase R^2 simply by adding independent variables, but larger values of R^2 only mean better fit to existing data…it does NOT mean better predictive power. Indeed, parsimony in the number of independent variables and number of estimated parameters is a GOOD THING. That is, Occam’s Razor applies: Use as many as needed to achieve good (predictive) results, but use no more.)

In the Statistics.pdf reference [ http://www.trendingit.com/docs/en/Theoretical_overview/1-Statistics.pdf ], it is stated that “no behaviour (no one worthy to be predicted) can be described by attending at a so low number of variables.” Thus it seems that TrendingBot rejects the idea of parsimony.

4. In the same pdf document, it is admitted that TrendingBot cannot deal with stochasticity: ‘trendingBot’s result for any (stochastic) time series = “trend not found”‘

5. As a general rule, if one is building a completely deterministic model (with all probabilities ignored), then one should first determine the model form. For example, in 1964, when DuPont wanted me to build a deterministic, predictive model of the output of “waste nylon” from a new manufacturing process, I went down to the factory floor, WATCHED, and ASKED the workers what they did and how they decided what values to put into their data sheets. [The industrial engineer, mechanical engineer, and foreman all were wrong about how the operation worked.] I then constructed a mathematical model based on how the process operated and how the data were determined. I then used the existing data to determine the parameters and checked to ensure that the model’s input variable historical data generated the actual output variable historical data.

5. The aforementioned pdf document states:

5a. “although there are no essential differences between extrapolation and interpolation methods…” YES THERE IS: OOne interpolates between values, one extrapolates beyond existing values.

5b. In the section, “b-5 probability-related methods [randomness],” it is stated that “these methods do not predict future behaviours on the basis of past ones [effects on the dependent variable from variations in the independent one(s)], but the probability of an event [= invariant phenomenon = not describable as a result of the interaction between independent/dependent variables] to occur.” This is not true. For many years I developed stochastic (probabilistic) models that predicted future values, WITH ESTIMATED PROBABILITIES OF OCCURRENCE or, more accurately, WITH PREDICTION INTERVALS for the predicted future values and associated PROBABILITIES that the predicted values would lie within the intervals.
_________________________________________________________________

You seem to be a quite devote defender of old methods, so I am having the impression that this will be tough (from your side and from mine).
1. You are right -> it intends to rely on the common sense and on the personal experience of anyone (and anyone’s work). As a rough estimation, we say that if the feeding is higher or equal than 30 cases and the resulting mean error yields below 5% (if lower, much better), the introduced behaviour has been quite properly understood and, hence most likely the resulting prediction would be quite accurate; how long? It depends completely upon the situation. That is, we are not intending to teach anyone how to do their jobs, but helping them (higher understanding over data), improving their knowledge. The final decision and the right application of this knowledge can only be given by an experienced person on this specific field.
2. Again, you are right. They are assumed to be uncorrelated because this is the way the would be relevant in order to describe the behaviour from a mathematical point of view. Too similar independent variables = the model will consider in the best scenario just one of them.
3. In this point, your ideas are not clear enough: trendingBot does not decide what variable should be chosen; it considers ALL OF THEM.
Short ideas:
a) Detailed combinatorics (all the possible combinations among all the possible number of variables raised to different exponents) -> millions of combinations, let’s call them: x1(i)
b) Regression part -> (2nd degree) polynomial are performed, taken as independent variable any of this x1(i) and as dependent the corresponding output. Note that this analysis is being performed for every single case (or training point)
c) Decision sub-algorithm -> the differents fits (again various millions) are validated against the given data (all the cases) and only the best one is chosen. Additional note here: trendingBot considers as a trend any fit that, after being applied to all the given training set, shows a (relative) error = it is considered that the behaviour has been more-or-less properly understood and certain forecasting capabilities are assignated to this specific fit.

I don’t know what is the idea of parsimony; with this statement I meant that although the regressive analysis is the most adequate method for data understanding, its application is really restrictive (extending it is precisely the idea behind trendingBot).

4. No. It can (and should) not. After reading above how a trend is determined (low error for the most of the training points), you would come to the same conclusion. Understanding the point properly: certain level of randomness, yes (everything is quasi-random) but essentially-random behaviours, no (random = impossible to be predicted, then what would be the point of using a self-called predictor?).
5. I think this point has been answered above: the program just provides some help, let the final decision to the user.
5a: This statement does not reflect the intention of the writing: I know the difference between interpolation and extrapolations, and consider the latter as the last resource; I meant that the method required to perform a interpolating regression or a extrapolating one shoud be almost identical because the underlying problem is basically the same. Nevertheless this was a completely secondary comment (I don’t know anything about extrapolation methods) having sense within the rest the text but I am not understanding what is its importance to be highlighted here.
5b: random means impredictable, once you can somehow predict it has implicity convert its essence into forecastable and thus not randomb/stochastic. Have you got it though stochastic means? Good for you but them you shouldn’t continue calling this behaviour random: was random when it was impossible for you to understand it, once this problem has been overcome, its characters changes.

I think the differences are quite straighforward (please, if you haven’t test it, don’t say that their performances are equivalent these is really for from true). After a quickly skimming through this page, I know what is about: it is a simple regression analyser. All the examples you can see there involve just 2 variables: 1 dependent vs. 1 independent. For example, time series: whatever value (dependent) vs. time. This really-limited lite version does something on the lines of this program (data->result) but for 5 independent variables, I thought this issue was clear enough.

For example: one of the blind test someone propose me yesterday consisted in different assets (equity bounds), each of them being defined by 4 variables (total, prem, last & vol); trendingbot found a trend easily predicting them during 1 year with a resulting error of 4%. How could deal the program you say with such a problem? Answer: it can not, because traditional regression analysis can not.

You don’t have excel? I will try to sort out this issue ASAP (right now working with OO). Until them why not accepting our challenge? (in our webpage): send us any set of data and get the solution; then, apply the program you say and update this forum with the result. What do you think?

PS: am I the only one experiencing a bit weird behaviour of this forum during teh last hour? Right now it is going really slow (almost not able to see what I am writing).

Suggestions: Provide at least a basic outline in the text file that is with the prgram. Opening a text file to read the information in it and then finding only a link to a blog is annoying to say the least!

For a program whose full version has such a high price, I would advise using more formal terminology tham phrases such as “the whole thing.” Perhaps this is am attempt to be humorous but humor is rather out of place im a high-end business oriented application.

The stated requirements for 1 to 2 Gb of memory and 5/10 Gb of empty hdd space are too large for many people’s systems. Consider ways to reduce these requirements. Remember there are many MANY people running older slower systems with smaller hard drives. Remember too that many business people use laptops and most laptops hard drives are small compared to desktops of equivalent age.

The various exe, msi, and batch files sprinkled throughout the program’s folders are confusing as to what app is supposed to be for what purpose.

The website has very simple sketchy info. On the “about us” page, the “more about us” link merely reloads the same page.

The idea of a yahoo group is good, but when there is no activity at all and only links to “yahoo answers” math pages on the group homepage, the overall impression it gives is negative.

I suggest you spend some time looking at examples of both good and bad site design at these links:
http://www.webpagesthatsuck.com/
http://www.killersites.com/
http://www.worstoftheweb.com/

Top Ten Mistakes in Web Design
http://www.useit.com/alertbox/9605.html

Usability 101: Introduction to Usability
http://www.useit.com/alertbox/20030825.html

Some changes in presentation, good documentation and usage instructions, and a group or forum that has activity of some kind, will do a lot to improve the impression this application and its developer site gives to the public.

Good luck!
________________________________________________________________

Impressive feeback!
We have to work on all the lines you propose, but let me make some precisions:
- “the_whole_thing”, the same as many others details try to give a “casual” image: young and funny, although professional. But perhaps you are right and this casualty should make a bit milder in order to avoid people confusing it with lack of seriourity.
- Regarding the requirements, quite little can be done on this front. Actually, they will follow an increasing progression (in parallel to the complexity of the algorithm). The basic idea is simplifying as less as possible = doing very much = very high (unfortunately unavoidable) computational expense.
- My intention after reading you comment was removing yahoo groups, although yesterday, we’ve got our first post.
- All your suggestions on the webpage front are really interesting, I have already let our webmaster know about them.

Thanks again.

Although the description is sketchy, I’m intrigued. I’m a researcher and a SAS user. I’m disappointed that we are limited to 5 variables. I work with multivariate data, never with as few as 5 variables.
——————————————

I am using Vista and can install the program without problems.

Regarding the multivariate systems I have writen about them in may posts: they are not considering all the variables at the same time or with the exponential variation my algorithm does. Either impose rigid inputs-systems (you have to measure this and this) or rely on lots of user-defined parameters, or give completely indirect answers (new variables giving an indirect idea about what is going on). Any of these multivariate systems will do the proposed problem here: give the data, run the sim and let the program try everything. If you have doubts, you can try with the most powerfull mathematical package: Matlab; its regression/curve fitting sub-package does allow a maximum of 2 independent variables (with no exponential variation and with user intervention (you say what to do)).