Abstract
Multi-stage decision-making under uncertainty is a complex process where decisions are made sequentially over multiple stages, and the outcomes of each decision depend on uncertain future events or states of nature. In practice, the underlying true probability distribution that governs the uncertainty is hardly accessible. Instead, we only have the historical data of the uncertainty. Confronted with such a situation, robust decisions could help to avoid extreme adverse outcomes, providing decision-makers with a more reliable solution in the face of uncertainty. In this thesis, we study models that can provide robust decisions in multi-stage decision-making problems.The first model concerns adjustable distributionally robust optimization problems equipped with an infinitely constrained ambiguity set. Building upon the linear decision rule, we introduce an algorithm designed to yield approximate solutions that exhibit monotonic improvement iteratively. Employing the covariance dominance ambiguity set and the fourth moment ambiguity set, the framework is employed in three distinct applications. Numerical results indicate promising advantages of our framework.
The second model is a novel robust satisficing Markov decision processes (RSMDPs). This model provides policies whose expected return is softly constrained to reach a user-specified target under ambiguous transition kernels. We provide a tractable reformulation and further devise a first-order approach for handling large-scale problem instances. Empirical findings indicate that RSMDPs can provide policies to attain their targets, and these targets are remarkably higher than the optimal worst-case returns computed by robust MDPs. Furthermore, our model demonstrates average and percentile performances that are competitive against other benchmark models. Also, our proposed algorithms for RSMDPs are shown to be more gracefully scalable compared to a state-of-the-art commercial solver.
The third model is called risk-aware robust satisficing MDPs---an extension of the RSMDPs that can account for both the ambiguity in transition kernel and reward. To construct our risk-aware RSMDPs, we first propose distributionally robust MDPs (DR) and distributionally robust chance-constrained MDPs (DCC), which optimize the average and percentile performances of policies under reward ambiguity, respectively. The worst-case expected return (in DR) or worst-case percentile performance (in DCC) of the policies are then constrained to reach a user-specified target under transition kernel ambiguity in our risk-aware RSMDPs, which thus are robust against ambiguity from reward and/or transition kernel. Tractable reformulations are provided for all the proposed models, and we also provide scalable tailored first-order methods for the risk-aware RSMDPs. Numerical results showcase that our risk-aware RSMDPs can achieve competitive performance and reach their targets under reward and transition kernel ambiguity. The superior scalability of our first-order methods compared to a state-of-the-art commercial solver is also demonstrated.
| Date of Award | 30 Aug 2024 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Chin Pang HO (Supervisor) & Zhi Chen (External Co-Supervisor) |