eprintid: 1728
rev_number: 6
eprint_status: archive
userid: 46
dir: disk0/00/00/17/28
datestamp: 2013-09-16 09:25:24
lastmod: 2013-09-16 12:02:59
status_changed: 2013-09-16 09:25:24
type: article
metadata_visibility: show
creators_name: Gaggero, Mauro
creators_name: Gnecco, Giorgio
creators_name: Sanguineti, Marcello
creators_id: 
creators_id: giorgio.gnecco@imtlucca.it
creators_id: 
title: Dynamic Programming and Value-Function Approximation in Sequential Decision Problems: Error Analysis and Numerical Results
ispublished: pub
subjects: QA75
divisions: CSA
full_text_status: none
keywords: Sequential decision problems; Dynamic programming; Approximation schemes; Curse of dimensionality; Suboptimal solutions; Optimal consumption
abstract: Value-function approximation is investigated for the solution via Dynamic Programming (DP) of continuous-state sequential N-stage decision problems, in which the reward to be maximized has an additive structure over a finite number of stages. Conditions that guarantee smoothness properties of the value function at each stage are derived. These properties are exploited to approximate such functions by means of certain nonlinear approximation schemes, which include splines of suitable order and Gaussian radial-basis networks with variable centers and widths. The accuracies of suboptimal solutions obtained by combining DP with these approximation tools are estimated. The results provide insights into the successful performances appeared in the literature about the use of value-function approximators in DP. The theoretical analysis is applied to a problem of optimal consumption, with simulation results illustrating the use of the proposed solution methodology. Numerical comparisons with classical linear approximators are presented.
date: 2013-02
date_type: published
publication: Journal of Optimization Theory and Applications
volume: 156
number: 2
publisher: Springer-Verlag
pagerange: 380-416
id_number: 10.1007/s10957-012-0118-2
refereed: TRUE
issn: 0022-3239
official_url: http://dx.doi.org/10.1007/s10957-012-0118-2
citation:   Gaggero, Mauro and Gnecco, Giorgio and Sanguineti, Marcello  Dynamic Programming and Value-Function Approximation in Sequential Decision Problems: Error Analysis and Numerical Results.  Journal of Optimization Theory and Applications, 156 (2).  pp. 380-416.  ISSN 0022-3239      (2013)