Pairwise Estimation for Multivariate Gaussian Processes and Its Applications

Abstract

Gaussian processes (GP) are popular in many contexts, such as geostatistics, computer experiments, machine learning and statistical quality control, and so are multivariate Gaussian processes (MGP). Nevertheless, building a MGP model is a very challenging task, especially for a large-scale one with either a high dimensional covariance matrix or huge number of parameters, i.e., it is of both computational and dimensional challenge to build a large-scale MGP model.

The main contribution of this thesis is that an efficient and scalable pairwise estimation framework is proposed as a remedy to overcome the two challenges in building large-scale MGP models with nonseparable covariance functions. Instead of estimating all parameters together using the MLE method which requires optimizing the time-consuming and nonconvex full likelihood, the proposed method sequentially estimates a subset of parameters based on a series of bivariate GP (joint of two marginal GPs) models, each of which contains a much smaller covariance matrix and fewer parameters than the full MGP model. Accordingly the proposed method has an excellent scalability and computational efficiency in fitting a large-scale MGP model. In fact, this pairwise estimation framework can be also extended to fit large-scale MGP models with other covariance functions that enable the proposed pairwise estimation framework.

In this thesis, the theoretical properties of the proposed pairwise method are also studied. Both asymptotic properties and finite sample properties are investigated here. Although the proposed pairwise method, as a sequential estimation method, inevitably suffers from error accumulation, the finite sample properties show that there is theoretically no error accumulation under some usual regularity conditions. Besides, the error propagation during the process of sequential estimation is partially blocked if these conditions are not satisfied, especially for MGP models with highly correlated marginal GPs. The finite sample properties enable the proposed method to be used in real applications usually with limited data. The asymptotic properties, i.e. the consistency and asymptotic normality of parameters estimated by the proposed method, further show that there is no error accumulation asymptotically and the sequential manner of the proposed pairwise method does not cause extra statistical information loss on parameter estimation. In addition, the asymptotic properties provide a useful asymptotic distribution that can be used in some applications.

In additional to the theoretical study on the proposed method, we also explore its application in multivariate computer experiments where MGP models with nonseparable covariance functions are often advocated to be used. The applicability of the proposed method is illustrated in large-scale multivariate computer emulations. The performance of the proposed method in multivariate emulation is compared with that of other alternative methods in numerical simulations and case studies and desired performance is observed. Furthermore, the proposed pairwise method is applied to multivariate profile monitoring to detect changes in both within-profile correlations and between-profile correlations simultaneously. The asymptotic distribution of parameters estimated by the proposed pairwise method offers a powerful tool to monitor the changes in model parameters and hence the changes in the multivariate profiles. A case study on low-Emittance (low-E) glass data is given as an example to use the asymptotic distribution for monitoring purposes, of which a phase I analysis is proposed.

Date of Award	14 May 2019
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Kwok Leung TSUI (Supervisor) & Qiang ZHOU (Supervisor)