Abstract
Simulators of engineering systems often have multiple outputs/responses and two types of inputs, called control factors and noise factors, where control factors are decision variables while noise factors are inputs that vary randomly in the actual system. Such simulators are often used to perform robust parameter design (RPD), i.e., the design of an engineering system so that it is robust to variations in its noise factor inputs via a proper choice of the value of the vector of control factors (also called control-factor setting).In RPD, the quality of a response is often measured by its expected loss (EL), which is the expectation of the quality loss function for the response evaluated at the value of the response with respect to the distribution of the vector of noise factors in the real system. A common approach to RPD is to solve a robust optimization problem with ELs as objective and/or constraint functions to obtain a control-factor setting that makes the system robust to noise-factor variation. However, when the simulator is time-consuming, such robust optimization problems can be too costly to solve with standard optimization algorithms. In this case, Bayesian optimization (BO) methods can be affordable and effective alternatives. A BO method is a computationally efficient method that sequentially adds experiment design points picked by an acquisition function (AF) to improve the estimate of an optimization problem's optimal solution obtained via Gaussian process (GP) models that predict the problem's objective and/or constraint functions. Such a method is more affordable than standard optimization algorithms when the simulator is time-consuming as it typically requires fewer simulator runs than standard algorithms. Hence, BO methods are particularly useful for RPD problems based on time-consuming simulators, which commonly occur in practice. This thesis develops novel BO methods for RPD problems based on simulators with multiple responses in which the quality level of each response is measured by its EL. The thesis is organized as follows:
In Chapter 1, a review of RPD based on time-consuming simulators, a review of BO methods, research motivations of this thesis, and research contributions of this thesis are given.
Chapter 2 considers an important class of RPD problems based on time-consuming simulators of engineering systems with multiple responses, i.e., the problem of minimizing the EL of a response subject to an upper bound constraint on the EL of each of a few other responses called the EL-based constrained RPD problem. This chapter proposes a novel BO method, called the mean-based constrained expected improvement (MCEI) method, to solve the EL-based constrained RPD problem. The proposed method incorporates information from the loss function of each response to predict its EL. This involves modeling each response as an independent Gaussian Process (GP), which makes the quality loss for every response a non-GP given by the composition of the loss function and the GP model for the response. We develop a novel, cheap-to-compute, and intuitively interpretable AF for picking the values of the control and noise factors of the follow-up design point, and we prove the proposed AF possesses some desirable properties, notably that it gives a consistent BO method when the design region is a finite set. Realistic examples show that our proposed BO method is superior to alternative methods.
Chapter 3 considers another important class of RPD problems based on time-consuming simulators of engineering systems with multiple responses, i.e., a multi-objective RPD problem with multiple EL objective functions called the multi-EL-objective RPD problem. The chapter proposes a novel BO method to solve the multi-EL-objective RPD problem. Many effective AFs for multi-objective BO are based on expected improvements to hypervolume indicators, i.e., expected hypervolume improvement (EHI)-type AFs. However, tractable computational methods for such AFs are only available for the case where the objective function values at any input-vector value have independent posterior normal distributions. Unfortunately, when the model for the quality loss of each response incorporates the known loss function for the response via the composition of the loss function and a GP model for the response, each EL value has a nonnormal posterior distribution. In Chapter 3, we propose a novel BO method to solve the multi-EL-objective RPD problem that overcomes the difficulty of incorporating information from the loss functions into an EHI-type AF while maintaining computational tractability of the AF. We model each response as an independent GP, and we develop a normal approximation to the posterior distribution of each response's EL value that incorporates information from the response's loss function via a first order Taylor expansion of the loss function. This enables us to develop an estimator of the Pareto set, an EHI-type AF for selecting the follow-up design point's control-factor setting, and an approximate expected entropy criterion for selecting the follow-up design point's noise-factor setting that are all tractable to compute. Theoretical properties of our proposed BO method such as its consistency are established. Examples based on realistic simulators show that the proposed BO method can significantly outperform alternative methods.
Chapter 4 revisits the EL-based constrained RPD problem considered in Chapter 2, but with the added assumption that the responses and inputs to the simulator are known to be positive-valued. Because the responses and inputs are positive-valued, direct modeling of the functional relationship between each response and the inputs with a standard a priori stationary GP model is inappropriate. This is due to a few reasons, notably the fact that a GP can take negative values whereas each response is positive-valued, and the functional relationship between each positive-valued response and the positive-valued inputs often exhibit nonstationary features. To address these issues, we incorporate parametric transformations into the response and inputs of each GP model used in the MCEI method proposed in Chapter 2, which yields a method that we call the MCEI with transformations (MCEIT) method. Specifically, we propose the use of the dual power transformation to transform each response and the Box-Cox transformation to transform each input. These transformations ensure that each response is positive-valued with prior probability one, and they create transformed responses and transformed inputs that have functional relationships that are well modeled with a priori stationary GP models. We argue that our choices of transformations for the responses and inputs are among the most suitable choices due to various reasons. Notably, for transforming each response, the necessity for a transformation with image equal to the real line and domain equal to the set of positive numbers to ensure a well-defined posterior distribution for each EL makes the dual power transformation the only suitable choice. We show that the MCEIT method is equivalent to a version of the original MCEI method under some conditions, which implies that the MCEIT method inherits all the desirable theoretical properties of the MCEI method proven in Chapter 2. Furthermore, we demonstrate numerically that the MCEIT method yields results that are superior to those given by the original MCEI method.
Chapter 5 concludes the thesis and discusses possible future work.
| Date of Award | 20 Jan 2026 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Matthias Hwai-yong TAN (Supervisor) |
Cite this
- Standard