Microsoft Neural Network, how do I feed it ?

Hello there,

I'm working with Analysis sevices 2005 developer edition. Looking through the documentation i becomes apperent that the NN algorithm takes 255 input attributes by default. This can be changed to any integer value, OK....

My problem is that I want to feed the network with 40000 input variables. In order to do so, I will have to do a select:

SELECT fld1, fld2, ...... fld39999, fld40000

FROM tblSometable

However, this is not possible, as the books online describes it is only possible to return 4096 columns from a select statement.

Question : How do I populate a NN in AS2005, with nmore than 4096 inputs ?!

The limit may actually be smaller. SQL Server supports, I think, only 1024 columns

Analysis Services 2005 supports the concept of nested table which allows multiple related tables to be included in modeling.

With nested tables, Analysis Services can use much larger input sets (even hundreds of thousands) represented in multiple tables. Some details on the nested table concept are available here. http://msdn2.microsoft.com/en-us/library/ms175659.aspx

However, this is a data modeling issue and not related to the NN 255 default limit.

The NN algorithm (and many other algorithms included in SQL Server 2005) will always perform feature selection,using (by default) 255 input attributes. it means that, even when your data contains 40K inputs, NN will still use, by default, only the most significant 255. To increase this limit, change the value of the MAXIMUM_INPUT_ATTRIBUTES algorithm parameter. This parameter has the same name for all the algorithms that perform feature selection. If you cange this parameter value to, say, 40000, then all 40K inputs will be used.

|||

Thanks.

I suspected I could do this using nested tables. Thanks for confirming this. Time might be an issue, however. Mu current test-network consisting of 1000 inputs (MAX_INPUT_ATTRIB = 1000), that are zeros and ones, takes +3 hours on a 3.0Ghz PentiumD with 3 GB om memory. Is the time going to grow exponential or linear with the number of input nodes ?

regs,

Mads