Gaggero, Mauro and Gnecco, Giorgio and Sanguineti, Marcello Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty. Computational Optimization and Applications, 58 (1). pp. 31-85. ISSN 0926-6003 (2014)
This is the latest version of this item.
Abstract
Stochastic optimization problems with an objective function that is additive over a finite number of stages are addressed. Although Dynamic Programming allows one to formally solve such problems, closed-form solutions can be derived only in particular cases. The search for suboptimal solutions via two approaches is addressed: approximation of the value functions and approximation of the optimal decision policies. The approximations take on the form of linear combinations of basis functions containing adjustable parameters to be optimized together with the coefficients of the combinations. Two kinds of basis functions are considered: Gaussians with varying centers and widths and sigmoids with varying weights and biases. The accuracies of such suboptimal solutions are investigated via estimates of the error propagation through the stages. Upper bounds are derived on the differences between the optimal value of the objective functional and its suboptimal values corresponding to the use at each stage of approximate value functions and approximate policies. Conditions under which the number of basis functions required for a desired approximation accuracy does not grow “too fast” with respect to the dimensions of the state and random vectors are provided. As an example of application, a multidimensional problem of optimal consumption under uncertainty is investigated, where consumers aim at maximizing a social utility function. Numerical simulations are provided, emphasizing computational pros and cons of the two approaches (i.e., value-function approximation and optimal-policy approximation) using the above-mentioned two kinds of basis functions. To investigate the dependencies of the performances on dimensionality, the numerical analysis is performed for various numbers of consumers. In the simulations, discretization techniques exploiting low-discrepancy sequences are used. Both theoretical and numerical results give insights into the possibility of coping with the curse of dimensionality in stochastic optimization problems whose decision strategies depend on large numbers of variables.
| Item Type: | Article |
|---|---|
| Identification Number: | 10.1007/s10589-013-9614-z |
| Uncontrolled Keywords: | Sequential decisions Stochastic optimization Dynamic programming Suboptimal solutions Infinite-dimensional programming Optimal consumption under uncertainty |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Research Area: | Computer Science and Applications |
| Depositing User: | Giorgio Gnecco |
| Date Deposited: | 17 Sep 2013 13:00 |
| Last Modified: | 18 Feb 2015 11:38 |
| URI: | http://eprints.imtlucca.it/id/eprint/1780 |
Available Versions of this Item
-
Stochastic N-stage optimization via approximations of value and policy functions. (deposited 16 Sep 2013 10:53)
- Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty. (deposited 17 Sep 2013 13:00) [Currently Displayed]
Actions (login required)
 |
Edit Item |
