Gaggero, Mauro and Gnecco, Giorgio and Sanguineti, Marcello Approximate dynamic programming for stochastic Nstage optimization with application to optimal consumption under uncertainty. Computational Optimization and Applications, 58 (1). pp. 3185. ISSN 09266003 (2014)
This is the latest version of this item.
Abstract
Stochastic optimization problems with an objective function that is additive over a finite number of stages are addressed. Although Dynamic Programming allows one to formally solve such problems, closedform solutions can be derived only in particular cases. The search for suboptimal solutions via two approaches is addressed: approximation of the value functions and approximation of the optimal decision policies. The approximations take on the form of linear combinations of basis functions containing adjustable parameters to be optimized together with the coefficients of the combinations. Two kinds of basis functions are considered: Gaussians with varying centers and widths and sigmoids with varying weights and biases. The accuracies of such suboptimal solutions are investigated via estimates of the error propagation through the stages. Upper bounds are derived on the differences between the optimal value of the objective functional and its suboptimal values corresponding to the use at each stage of approximate value functions and approximate policies. Conditions under which the number of basis functions required for a desired approximation accuracy does not grow “too fast” with respect to the dimensions of the state and random vectors are provided. As an example of application, a multidimensional problem of optimal consumption under uncertainty is investigated, where consumers aim at maximizing a social utility function. Numerical simulations are provided, emphasizing computational pros and cons of the two approaches (i.e., valuefunction approximation and optimalpolicy approximation) using the abovementioned two kinds of basis functions. To investigate the dependencies of the performances on dimensionality, the numerical analysis is performed for various numbers of consumers. In the simulations, discretization techniques exploiting lowdiscrepancy sequences are used. Both theoretical and numerical results give insights into the possibility of coping with the curse of dimensionality in stochastic optimization problems whose decision strategies depend on large numbers of variables.
Item Type:  Article 

Identification Number:  10.1007/s105890139614z 
Uncontrolled Keywords:  Sequential decisions Stochastic optimization Dynamic programming Suboptimal solutions Infinitedimensional programming Optimal consumption under uncertainty 
Subjects:  Q Science > QA Mathematics > QA75 Electronic computers. Computer science 
Research Area:  Computer Science and Applications 
Depositing User:  Giorgio Gnecco 
Date Deposited:  17 Sep 2013 13:00 
Last Modified:  18 Feb 2015 11:38 
URI:  http://eprints.imtlucca.it/id/eprint/1780 
Available Versions of this Item

Stochastic Nstage optimization via approximations of value and policy functions. (deposited 16 Sep 2013 10:53)
 Approximate dynamic programming for stochastic Nstage optimization with application to optimal consumption under uncertainty. (deposited 17 Sep 2013 13:00) [Currently Displayed]
Actions (login required)
Edit Item 