eprintid: 1780
rev_number: 6
eprint_status: archive
userid: 46
dir: disk0/00/00/17/80
datestamp: 2013-09-17 13:00:19
lastmod: 2015-02-18 11:38:24
status_changed: 2013-09-17 13:00:19
type: article
succeeds: 1738
metadata_visibility: show
creators_name: Gaggero, Mauro
creators_name: Gnecco, Giorgio
creators_name: Sanguineti, Marcello
creators_id: 
creators_id: giorgio.gnecco@imtlucca.it
creators_id: 
title: Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty
ispublished: pub
subjects: QA75
divisions: CSA
full_text_status: none
keywords: Sequential decisions Stochastic optimization Dynamic programming Suboptimal solutions Infinite-dimensional programming Optimal consumption under uncertainty
abstract: Stochastic optimization problems with an objective function that is additive over a finite number of stages are addressed. Although Dynamic Programming allows one to formally solve such problems, closed-form solutions can be derived only in particular cases. The search for suboptimal solutions via two approaches is addressed: approximation of the value functions and approximation of the optimal decision policies. The approximations take on the form of linear combinations of basis functions containing adjustable parameters to be optimized together with the coefficients of the combinations. Two kinds of basis functions are considered: Gaussians with varying centers and widths and sigmoids with varying weights and biases. The accuracies of such suboptimal solutions are investigated via estimates of the error propagation through the stages. Upper bounds are derived on the differences between the optimal value of the objective functional and its suboptimal values corresponding to the use at each stage of approximate value functions and approximate policies. Conditions under which the number of basis functions required for a desired approximation accuracy does not grow “too fast” with respect to the dimensions of the state and random vectors are provided. As an example of application, a multidimensional problem of optimal consumption under uncertainty is investigated, where consumers aim at maximizing a social utility function. Numerical simulations are provided, emphasizing computational pros and cons of the two approaches (i.e., value-function approximation and optimal-policy approximation) using the above-mentioned two kinds of basis functions. To investigate the dependencies of the performances on dimensionality, the numerical analysis is performed for various numbers of consumers. In the simulations, discretization techniques exploiting low-discrepancy sequences are used. Both theoretical and numerical results give insights into the possibility of coping with the curse of dimensionality in stochastic optimization problems whose decision strategies depend on large numbers of variables.
date: 2014-05
date_type: published
publication: Computational Optimization and Applications
volume: 58
number: 1
publisher: Springer
pagerange: 31-85
id_number: 10.1007/s10589-013-9614-z
refereed: TRUE
issn: 0926-6003
official_url: http://dx.doi.org/10.1007/s10589-013-9614-z
citation:   Gaggero, Mauro and Gnecco, Giorgio and Sanguineti, Marcello  Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty.  Computational Optimization and Applications, 58 (1).  pp. 31-85.  ISSN 0926-6003      (2014)