Microsoft Neural Networks Question

1) Scaling of Inputs

Is the standarization of the inputs done automatically when running the Microsoft Neural Network algorithm or I should be transforming the variables before running the algorithm?

2) Predicted Probabilities

How do I create a table with the actual predicted probabilities of the model for each observation? In the Mining Model Prediction tab the output would be either 0 or 1, my question is how can I obtain the actual value of the estimated probability?

1) Yes, using z-score ((value -mean)/StdDev), so there is no reason for transformation.

2) Use a prediction function.

PredictProbability will return the probability of the actual prediction. PredictHistogram wil return all the probabilities (for all target states). The actual prediction is, of course, the state from the histogram having the largest probability

|||

Thanks for this. I assume that for categotical variables standarization is also done automatically by some other appropriate transformation, right?

Cheers,

Leo.

|||It sure is. Enjoy!

Microsoft Neural Network, how do I feed it ?

Hello there,

I'm working with Analysis sevices 2005 developer edition. Looking through the documentation i becomes apperent that the NN algorithm takes 255 input attributes by default. This can be changed to any integer value, OK....

My problem is that I want to feed the network with 40000 input variables. In order to do so, I will have to do a select:

SELECT fld1, fld2, ...... fld39999, fld40000

FROM tblSometable

However, this is not possible, as the books online describes it is only possible to return 4096 columns from a select statement.

Question : How do I populate a NN in AS2005, with nmore than 4096 inputs ?!

The limit may actually be smaller. SQL Server supports, I think, only 1024 columns

Analysis Services 2005 supports the concept of nested table which allows multiple related tables to be included in modeling.

With nested tables, Analysis Services can use much larger input sets (even hundreds of thousands) represented in multiple tables. Some details on the nested table concept are available here. http://msdn2.microsoft.com/en-us/library/ms175659.aspx

However, this is a data modeling issue and not related to the NN 255 default limit.

The NN algorithm (and many other algorithms included in SQL Server 2005) will always perform feature selection,using (by default) 255 input attributes. it means that, even when your data contains 40K inputs, NN will still use, by default, only the most significant 255. To increase this limit, change the value of the MAXIMUM_INPUT_ATTRIBUTES algorithm parameter. This parameter has the same name for all the algorithms that perform feature selection. If you cange this parameter value to, say, 40000, then all 40K inputs will be used.

|||

Thanks.

I suspected I could do this using nested tables. Thanks for confirming this. Time might be an issue, however. Mu current test-network consisting of 1000 inputs (MAX_INPUT_ATTRIB = 1000), that are zeros and ones, takes +3 hours on a 3.0Ghz PentiumD with 3 GB om memory. Is the time going to grow exponential or linear with the number of input nodes ?

regs,

Mads

Microsoft Neural Network, how do I feed it ?

Hello there,

I'm working with Analysis sevices 2005 developer edition. Looking through the documentation i becomes apperent that the NN algorithm takes 255 input attributes by default. This can be changed to any integer value, OK....

My problem is that I want to feed the network with 40000 input variables. In order to do so, I will have to do a select:

SELECT fld1, fld2, ...... fld39999, fld40000

FROM tblSometable

However, this is not possible, as the books online describes it is only possible to return 4096 columns from a select statement.

Question : How do I populate a NN in AS2005, with nmore than 4096 inputs ?!

The limit may actually be smaller. SQL Server supports, I think, only 1024 columns

Analysis Services 2005 supports the concept of nested table which allows multiple related tables to be included in modeling.

With nested tables, Analysis Services can use much larger input sets (even hundreds of thousands) represented in multiple tables. Some details on the nested table concept are available here. http://msdn2.microsoft.com/en-us/library/ms175659.aspx

However, this is a data modeling issue and not related to the NN 255 default limit.

The NN algorithm (and many other algorithms included in SQL Server 2005) will always perform feature selection,using (by default) 255 input attributes. it means that, even when your data contains 40K inputs, NN will still use, by default, only the most significant 255. To increase this limit, change the value of the MAXIMUM_INPUT_ATTRIBUTES algorithm parameter. This parameter has the same name for all the algorithms that perform feature selection. If you cange this parameter value to, say, 40000, then all 40K inputs will be used.

|||

Thanks.

I suspected I could do this using nested tables. Thanks for confirming this. Time might be an issue, however. Mu current test-network consisting of 1000 inputs (MAX_INPUT_ATTRIB = 1000), that are zeros and ones, takes +3 hours on a 3.0Ghz PentiumD with 3 GB om memory. Is the time going to grow exponential or linear with the number of input nodes ?

regs,

Mads

Microsoft decision tree algorithm

hai ...............all

well i've read in Claude seidmann book about Data mining with microsoft decision, that the statistical techniques employed to build the decision trees include:

Cart, Chaid and C.45.Could anyone explain to me about cart,chaid and c.45? and how the tree statistical techniques influence the decision tree.

thank you so much

The decision tree algorithm used by MS is not CART, CHAID, or C4.5 - it's an algorithm developed by Microsoft Research.

There's more information available here:

"Scalable Classification over SQL Databases"

ftp://ftp.research.microsoft.com/users/AutoAdmin/icde99.pdf

"Learning Bayesian Networks: The Combination of Knowledge and Statistical Data"

http://research.microsoft.com/research/pubs/view.aspx?tr_id=81

microsoft decision tree algorithm

I have read some sources about microsoft decision tree algorithm like in claude seidman book, paper about scalable classification over sql databases and paper about learning bayesian network. But i still don't understand and i still didn't get the point on how microsoft decision tree algorithm works exactly when splitting an atribut. Because i have read that microsoft decision tree using Bayesian score to split criteria is it true?

Well, anyone could help me to understand about microsoft decision tree algorithm, please give me details explanation with some example(cases).

thanks for anyone help

There are links to the appropriate research papers at http://www.sqlserverdatamining.com/DMCommunity/TipsNTricks/986.aspx

Thanks

-Jamie

|||Hi, there, the first link fails, i am also interested to know how Microsoft Decision Trees splits a continous attribute for derivation of downstream regression models: using information gain or regession approach? thx.|||i'd like to further clarify my previous question for regression tree building: which method is used for splitting a continous variable: information gain or regression approach. The first splitting method (information gain) first bins a continous attribute and turns it into a categorical variable, selects a value which gives highest informtion gain; the second method, according to http://msdn2.microsoft.com/en-us/library/ms175312.aspx, the splitting is performed at the point of non-lineararity(statistical method?). I just wonder which method is used by Microsoft Decision Trees for splitting in the case of a continous variable.|||If the target is a discrete attribute, it uses information gain. For continuous targets, it uses the regression approach|||Thank you for your answer, Jamie. but my question is how to split the continous INPUT attribute (The target variable is assumed automatically to be continuous in the case of regression tree).|||I was describing how the INPUT is handled for various OUTPUTs - if you are only considering the case of regression trees, than it is always the regression approach.|||

Hi, there,

when I use the Microsoft Decision Trees algoirthm to build a regression tree and choose SCORE_METHOD: (1) entropy. :

does the algorithm select the categorical input attribute and split-on by using entropy function? is the continuous input attribute binned into categorical attribute for entropy calculation?

Or this score_method does not act on the regression tree?

Any help is very much appreciated,

Hongqin

|||Continuous variables are split by using a internal binning approach that searches the continuous space for the best split point. This is independent of the score method.|||thx, Jamie.|||

Dear all,

I've tried to change split_method to (2) Complete. It doesn't generate any leaves or trees. It show only top of the tree.

But when I changed it back to (3) Both. The result is ok..but include Binary Split and Complete Split.

By the way, How can i do if I want it to generate only complete split.

Thank you for your answer in advanced.

Nop Vorrasanpisut.

microsoft decision tree algorithm

I have read some sources about microsoft decision tree algorithm like in claude seidman book, paper about scalable classification over sql databases and paper about learning bayesian network. But i still don't understand and i still didn't get the point on how microsoft decision tree algorithm works exactly when splitting an atribut. Because i have read that microsoft decision tree using Bayesian score to split criteria is it true?

Well, anyone could help me to understand about microsoft decision tree algorithm, please give me details explanation with some example(cases).

thanks for anyone help

There are links to the appropriate research papers at http://www.sqlserverdatamining.com/DMCommunity/TipsNTricks/986.aspx

Thanks

-Jamie

|||Hi, there, the first link fails, i am also interested to know how Microsoft Decision Trees splits a continous attribute for derivation of downstream regression models: using information gain or regession approach? thx.|||i'd like to further clarify my previous question for regression tree building: which method is used for splitting a continous variable: information gain or regression approach. The first splitting method (information gain) first bins a continous attribute and turns it into a categorical variable, selects a value which gives highest informtion gain; the second method, according to http://msdn2.microsoft.com/en-us/library/ms175312.aspx, the splitting is performed at the point of non-lineararity(statistical method?). I just wonder which method is used by Microsoft Decision Trees for splitting in the case of a continous variable.|||If the target is a discrete attribute, it uses information gain. For continuous targets, it uses the regression approach|||Thank you for your answer, Jamie. but my question is how to split the continous INPUT attribute (The target variable is assumed automatically to be continuous in the case of regression tree).|||I was describing how the INPUT is handled for various OUTPUTs - if you are only considering the case of regression trees, than it is always the regression approach.|||

Hi, there,

when I use the Microsoft Decision Trees algoirthm to build a regression tree and choose SCORE_METHOD: (1) entropy. :

does the algorithm select the categorical input attribute and split-on by using entropy function? is the continuous input attribute binned into categorical attribute for entropy calculation?

Or this score_method does not act on the regression tree?

Any help is very much appreciated,

Hongqin

|||Continuous variables are split by using a internal binning approach that searches the continuous space for the best split point. This is independent of the score method.|||thx, Jamie.|||

Dear all,

I've tried to change split_method to (2) Complete. It doesn't generate any leaves or trees. It show only top of the tree.

But when I changed it back to (3) Both. The result is ok..but include Binary Split and Complete Split.

By the way, How can i do if I want it to generate only complete split.

Thank you for your answer in advanced.

Nop Vorrasanpisut.

microsoft decision tree algorithm

I have read some sources about microsoft decision tree algorithm like in claude seidman book, paper about scalable classification over sql databases and paper about learning bayesian network. But i still don't understand and i still didn't get the point on how microsoft decision tree algorithm works exactly when splitting an atribut. Because i have read that microsoft decision tree using Bayesian score to split criteria is it true?

Well, anyone could help me to understand about microsoft decision tree algorithm, please give me details explanation with some example(cases).

thanks for anyone help

There are links to the appropriate research papers at http://www.sqlserverdatamining.com/DMCommunity/TipsNTricks/986.aspx

Thanks

-Jamie

|||Hi, there, the first link fails, i am also interested to know how Microsoft Decision Trees splits a continous attribute for derivation of downstream regression models: using information gain or regession approach? thx.|||i'd like to further clarify my previous question for regression tree building: which method is used for splitting a continous variable: information gain or regression approach. The first splitting method (information gain) first bins a continous attribute and turns it into a categorical variable, selects a value which gives highest informtion gain; the second method, according to http://msdn2.microsoft.com/en-us/library/ms175312.aspx, the splitting is performed at the point of non-lineararity(statistical method?). I just wonder which method is used by Microsoft Decision Trees for splitting in the case of a continous variable.|||If the target is a discrete attribute, it uses information gain. For continuous targets, it uses the regression approach|||Thank you for your answer, Jamie. but my question is how to split the continous INPUT attribute (The target variable is assumed automatically to be continuous in the case of regression tree).|||I was describing how the INPUT is handled for various OUTPUTs - if you are only considering the case of regression trees, than it is always the regression approach.|||

Hi, there,

when I use the Microsoft Decision Trees algoirthm to build a regression tree and choose SCORE_METHOD: (1) entropy. :

does the algorithm select the categorical input attribute and split-on by using entropy function? is the continuous input attribute binned into categorical attribute for entropy calculation?

Or this score_method does not act on the regression tree?

Any help is very much appreciated,

Hongqin

|||Continuous variables are split by using a internal binning approach that searches the continuous space for the best split point. This is independent of the score method.|||thx, Jamie.|||

Dear all,

I've tried to change split_method to (2) Complete. It doesn't generate any leaves or trees. It show only top of the tree.

But when I changed it back to (3) Both. The result is ok..but include Binary Split and Complete Split.

By the way, How can i do if I want it to generate only complete split.

Thank you for your answer in advanced.

Nop Vorrasanpisut.

microsoft decision tree algorithm

I have read some sources about microsoft decision tree algorithm like in claude seidman book, paper about scalable classification over sql databases and paper about learning bayesian network. But i still don't understand and i still didn't get the point on how microsoft decision tree algorithm works exactly when splitting an atribut. Because i have read that microsoft decision tree using Bayesian score to split criteria is it true?

Well, anyone could help me to understand about microsoft decision tree algorithm, please give me details explanation with some example(cases).

thanks for anyone help

There are links to the appropriate research papers at http://www.sqlserverdatamining.com/DMCommunity/TipsNTricks/986.aspx

Thanks

-Jamie

|||Hi, there, the first link fails, i am also interested to know how Microsoft Decision Trees splits a continous attribute for derivation of downstream regression models: using information gain or regession approach? thx.|||i'd like to further clarify my previous question for regression tree building: which method is used for splitting a continous variable: information gain or regression approach. The first splitting method (information gain) first bins a continous attribute and turns it into a categorical variable, selects a value which gives highest informtion gain; the second method, according to http://msdn2.microsoft.com/en-us/library/ms175312.aspx, the splitting is performed at the point of non-lineararity(statistical method?). I just wonder which method is used by Microsoft Decision Trees for splitting in the case of a continous variable.|||If the target is a discrete attribute, it uses information gain. For continuous targets, it uses the regression approach|||Thank you for your answer, Jamie. but my question is how to split the continous INPUT attribute (The target variable is assumed automatically to be continuous in the case of regression tree).|||I was describing how the INPUT is handled for various OUTPUTs - if you are only considering the case of regression trees, than it is always the regression approach.|||

Hi, there,

when I use the Microsoft Decision Trees algoirthm to build a regression tree and choose SCORE_METHOD: (1) entropy. :

does the algorithm select the categorical input attribute and split-on by using entropy function? is the continuous input attribute binned into categorical attribute for entropy calculation?

Or this score_method does not act on the regression tree?

Any help is very much appreciated,

Hongqin

|||Continuous variables are split by using a internal binning approach that searches the continuous space for the best split point. This is independent of the score method.|||thx, Jamie.|||

Dear all,

I've tried to change split_method to (2) Complete. It doesn't generate any leaves or trees. It show only top of the tree.

But when I changed it back to (3) Both. The result is ok..but include Binary Split and Complete Split.

By the way, How can i do if I want it to generate only complete split.

Thank you for your answer in advanced.

Nop Vorrasanpisut.

sql

Microsoft decision tree

Hi again ....i'm trying to understand the works of microsoft decision tree algorithm

I have an example case below :

Customer Id

Debt Level

Income Level

Employment Type

Credit Risk

1

High

High

Self-Employed

Bad

2

High

High

Salaried

Bad

3

High

Low

Salaried

Bad

4

Low

Low

Salaried

Good

5

Low

Low

Self-Emplyed

Bad

6

Low

High

Self-Employed

Good

7

Low

High

Salaried

Good

My question is how about the equations used to determine a split?

Please explain me detailed.

Thanks a lot.

This article explains how the Microsoft Decision Tree algorithm works at a high level and also points to the relevant research papers that the implementation is based on:http://msdn2.microsoft.com/en-us/library/ms175312(SQL.90,d=ide).aspx

Microsoft Association rules importing to web form

hi

I am using Microsoft association algorithm to find the association between PATIENT CITY > likely Disease. I like to know how can i import association model after creating from SQL Server BI studio to use in my ASP.NET web form? such a way when the user enters PATIENT CITY, system prompts associated Disease.

I do have Data mining with SQL Server 2005 book, could't find any resource for my objective.

Please suggest best source or tutorial how can i do

your help and insight is highly appreciated.

Raju

Please see Bogdan's suggestions on this thread: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=650506&SiteID=1