Gnecco, Giorgio and Kůrková, Věra and Sanguineti, Marcello Some Comparisons of Complexity in DictionaryBased and Linear Computational Models. Neural Networks , 24 (2). pp. 171182. ISSN 08936080 (2011)
This is the latest version of this item.
Abstract
Neural networks provide a more flexible approximation of functions than traditional linear regression. In the latter, one can only adjust the coefficients in linear combinations of fixed sets of functions, such as orthogonal polynomials or Hermite functions, while for neural networks, one may also adjust the parameters of the functions which are being combined. However, some useful properties of linear approximators (such as uniqueness, homogeneity, and continuity of best approximation operators) are not satisfied by neural networks. Moreover, optimization of parameters in neural networks becomes more difficult than in linear regression. Experimental results suggest that these drawbacks of neural networks are offset by substantially lower model complexity, allowing accuracy of approximation even in highdimensional cases. We give some theoretical results comparing requirements on model complexity for two types of approximators, the traditional linear ones and so called variablebasis types, which include neural networks, radial, and kernel models. We compare upper bounds on worstcase errors in variablebasis approximation with lower bounds on such errors for any linear approximator. Using methods from nonlinear approximation and integral representations tailored to computational units, we describe some cases where neural networks outperform any linear approximator.
Item Type:  Article 

Identification Number:  10.1016/j.neunet.2010.10.002 
Uncontrolled Keywords:  Linear approximation schemes; Variablebasis approximation schemes; Model complexity; Worstcase errors; Neural networks; Kernel models 
Subjects:  Q Science > QA Mathematics > QA75 Electronic computers. Computer science 
Research Area:  Computer Science and Applications 
Depositing User:  Giorgio Gnecco 
Date Deposited:  17 Sep 2013 13:06 
Last Modified:  17 Sep 2013 13:06 
URI:  http://eprints.imtlucca.it/id/eprint/1778 
Available Versions of this Item

Some comparisons of complexity in dictionarybased and linear computational models. (deposited 13 Sep 2013 10:46)
 Some Comparisons of Complexity in DictionaryBased and Linear Computational Models. (deposited 17 Sep 2013 13:06) [Currently Displayed]
Actions (login required)
Edit Item 