%0 Report
%9 Working Paper
%A Dellino, Gabriella
%A Meloni, Carlo
%D 2012
%F eprints:1446
%I IMT Institute for Advanced Studies Lucca
%K simulation; metamodels; validation; bootstrap
%N  
%T Metamodel variability in robust simulation-optimization: a bootstrap analysis
%U http://eprints.imtlucca.it/1446/
%X Metamodels are often used in simulation-optimization for the design and management of complex systems enabling the integration of discipline-dependent analysis into the  overall decision process. These metamodels yield insight into the relationship between responses and decision variables, providing fast analysis tools in place of the more expensive computer simulations. The combined use of stochastic simulation experiments and metamodels introduces a source of uncertainty in the decision process that we refer to as metamodel variability. To quantify this variability, we combine validation and bootstrapping techniques. The rationale behind the method relies on the fact that, after the validation process, the relative validation errors are small indicating that the metamodels give an adequate approximation, and bootstrapping these errors allows to quantify the metamodels' variability in an acceptable way. The method has the advantage to be general and can be used with different kind of metamodels and validation techniques. The resulting methodology is illustrated through some examples using regression and Kriging metamodels.