eprintid: 1778
rev_number: 4
eprint_status: archive
userid: 46
dir: disk0/00/00/17/78
datestamp: 2013-09-17 13:06:11
lastmod: 2013-09-17 13:06:11
status_changed: 2013-09-17 13:06:11
type: article
succeeds: 1717
metadata_visibility: show
creators_name: Gnecco, Giorgio
creators_name: Kůrková, Věra
creators_name: Sanguineti, Marcello
creators_id: giorgio.gnecco@imtlucca.it
creators_id: 
creators_id: 
title: Some Comparisons of Complexity in Dictionary-Based and Linear Computational Models 
ispublished: pub
subjects: QA75
divisions: CSA
full_text_status: none
keywords: Linear approximation schemes; Variable-basis approximation schemes; Model complexity; Worst-case errors; Neural networks; Kernel models
abstract: Neural networks provide a more flexible approximation of functions than traditional linear regression. In the latter, one can only adjust the coefficients in linear combinations of fixed sets of functions, such as orthogonal polynomials or Hermite functions, while for neural networks, one may also adjust the parameters of the functions which are being combined. However, some useful properties of linear approximators (such as uniqueness, homogeneity, and continuity of best approximation operators) are not satisfied by neural networks. Moreover, optimization of parameters in neural networks becomes more difficult than in linear regression. Experimental results suggest that these drawbacks of neural networks are offset by substantially lower model complexity, allowing accuracy of approximation even in high-dimensional cases. We give some theoretical results comparing requirements on model complexity for two types of approximators, the traditional linear ones and so called variable-basis types, which include neural networks, radial, and kernel models. We compare upper bounds on worst-case errors in variable-basis approximation with lower bounds on such errors for any linear approximator. Using methods from nonlinear approximation and integral representations tailored to computational units, we describe some cases where neural networks outperform any linear approximator. 
date: 2011
date_type: published
publication: Neural Networks 
volume: 24
number: 2
publisher: Elseviers
pagerange: 171-182
id_number: 10.1016/j.neunet.2010.10.002
refereed: TRUE
issn: 0893-6080
official_url: http://www.sciencedirect.com/science/article/pii/S0893608010001887
citation:   Gnecco, Giorgio and Kůrková, Věra and Sanguineti, Marcello  Some Comparisons of Complexity in Dictionary-Based and Linear Computational Models.  Neural Networks , 24 (2).  pp. 171-182.  ISSN 0893-6080      (2011)