Logo eprints

Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty

Gaggero, Mauro and Gnecco, Giorgio and Sanguineti, Marcello Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty. Computational Optimization and Applications, 58 (1). pp. 31-85. ISSN 0926-6003 (2014)

This is the latest version of this item.

Full text not available from this repository.


Stochastic optimization problems with an objective function that is additive over a finite number of stages are addressed. Although Dynamic Programming allows one to formally solve such problems, closed-form solutions can be derived only in particular cases. The search for suboptimal solutions via two approaches is addressed: approximation of the value functions and approximation of the optimal decision policies. The approximations take on the form of linear combinations of basis functions containing adjustable parameters to be optimized together with the coefficients of the combinations. Two kinds of basis functions are considered: Gaussians with varying centers and widths and sigmoids with varying weights and biases. The accuracies of such suboptimal solutions are investigated via estimates of the error propagation through the stages. Upper bounds are derived on the differences between the optimal value of the objective functional and its suboptimal values corresponding to the use at each stage of approximate value functions and approximate policies. Conditions under which the number of basis functions required for a desired approximation accuracy does not grow “too fast” with respect to the dimensions of the state and random vectors are provided. As an example of application, a multidimensional problem of optimal consumption under uncertainty is investigated, where consumers aim at maximizing a social utility function. Numerical simulations are provided, emphasizing computational pros and cons of the two approaches (i.e., value-function approximation and optimal-policy approximation) using the above-mentioned two kinds of basis functions. To investigate the dependencies of the performances on dimensionality, the numerical analysis is performed for various numbers of consumers. In the simulations, discretization techniques exploiting low-discrepancy sequences are used. Both theoretical and numerical results give insights into the possibility of coping with the curse of dimensionality in stochastic optimization problems whose decision strategies depend on large numbers of variables.

Item Type: Article
Identification Number: 10.1007/s10589-013-9614-z
Uncontrolled Keywords: Sequential decisions Stochastic optimization Dynamic programming Suboptimal solutions Infinite-dimensional programming Optimal consumption under uncertainty
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Research Area: Computer Science and Applications
Depositing User: Giorgio Gnecco
Date Deposited: 17 Sep 2013 13:00
Last Modified: 18 Feb 2015 11:38
URI: http://eprints.imtlucca.it/id/eprint/1780

Available Versions of this Item

Actions (login required)

Edit Item Edit Item