Algorithm development for robust parameter estimation


Student thesis: Doctoral Thesis

View graph of relations


  • Yuan CHEN

Related Research Unit(s)


Awarding Institution
Award date16 Feb 2015


Parameter estimation is a common task required in the fields of science and engineering. It refers to accurately finding the values of interested parameters from observed data, which consist of two components, namely, signal and noise. The numerous estimators developed in the literature can be divided into three categories: parametric, nonparametric, and semiparametric techniques. In parametric approaches, the signal is assumed to satisfy a generating model with a known functional form. In nonparametric methodologies, no assumptions are made about the signal. In semiparametricbased algorithms, a priori knowledge on the signal is available. As parametric-based algorithms allow the derivation of optimal estimators when the assumed model has good approximation to reality, they are widely used to solve parameter estimation problems. Conventionally, we assume that the noise model follows the Gaussian distribution. However, this assumption is not suitable to represent all varieties of noise in the real world, in which noise may exhibit heavy-tailed characteristics. A heavy-tailed distribution, whose tails are longer than those of a normal distribution, is prone to the generation of large values in the time domain. In the literature, the typical models for impulsive noise are single processes and hybrid processes mixed in the probability density function (PDF) domain. The representative single distributions are α-stable, generalized Gaussian (GG), and Student's t distributions while the PDF mixture includes the Gaussian mixture (GM) process. In this thesis, a new noise model mixed in the time domain, namely, a symmetric α-stable Gaussian (SαSG) mixture, is proposed to describe the noise in some applications, such as in astrophysical image processing and indoor communications. Given the existence of outliers in a heavytailed distribution, traditional estimators designed for Gaussian distribution cannot provide a reliable performance. Therefore, robust parameter estimation is considerably important in ensuring that reliable estimates are obtained in impulsive noise. We also focus on developing robust parametric algorithms and conducting relevant performance analysis. We also discuss the robust nonparametric and semiparametric methods. Performance analysis is extremely important in robust signal processing for it evaluates the quality of an estimator and determines whether it is robust. Bias and mean square error (MSE) are the most basic performance measures. Employing the Taylor series expansion (TSE) on cost function, bias and MSE formulas are derived in the thesis. Then, we analyze the bias and MSE expressions of the lp-norm minimizer, which is a robust parametric technique for impulsive noise. Four representative noise models, namely, symmetric α-stable (SαS), GG, Student's t, and GM processes, are investigated, with consideration of both the linear and nonlinear parameter estimation problems at p ≥ 1. The optimal choice of p for different noise environments is also examined. The developed formulas are verified by comparing their outcomes with the simulation results. Although the least lp-norm estimator is nearly optimal for typical noise models, it cannot provide good performance for the SαSG process. Therefore, finding an optimal parametric estimator for the SαSG mixture is crucial and necessary. We start with a special case, namely, the additive Cauchy-Gaussian (ACG), which is a sum of Cauchy and Gaussian random variables in the time domain. The PDF of the ACG, referred to as the Voigt profile, is derived from the convolution of the Cauchy and Gaussian PDFs. To obtain the unknown parameters in both the linear and nonlinear problems under the ACG noise, we devise a classical parametric method called the maximum likelihood estimator (MLE). Given that the Voigt profile suffers from a complicated analytical form, we also derive an M-estimator with the pseudo-Voigt function (MEPV). In our algorithm development, both scenarios involving known and unknown density parameters are considered. For the latter case, the MLE and MEPV cannot be applied to estimate the density parameters directly because of the complication of the Voigt profile. Therefore, density parameters should be obtained by utilizing the characteristic function (CF) and the empirical characteristic function (ECF) prior to applying the proposed methods. Both the MLE and MEPV reach the Cramer-Rao lower bound (CRLB), and the MEPV has less computational cost than the MLE. Aiming for density parameter estimation, we also propose a new suboptimal estimator, referred to as the fractional lower-order moment (FLOM) estimator, with FLOM being calculated by the CF of the ACG. As two-stage estimation is required in both the MLE and MEPV, the performance of both methods depends on the first stage. That is, if the density parameter estimates are inaccurate, then the MLE and MEPV may be analytically intractable. This condition motivates us to pursue a new parametric methodology that avoids density parameter estimation. To satisfy this requirement, we employ a Markov chain Monte Carlo (MCMC) method, namely, the Metropolis-Hastings (M-H) algorithm. We then estimate the linear signal embedded in the ACG mixture noise. Numerical results show the optimality of the proposed algorithm in linear parameter estimation. Nevertheless, the performance of the parametric-based method will degrade if a mismatch appears between the assumed model and the actual signal. In the mismatching condition, nonparametric and semiparametric methodologies play a crucial role. Recently, a nonparametric-based algorithm called the iterative adaptive approach (IAA) has been shown to be an effective high-resolution spectral analysis tool. It mainly involves the reformulation of the nonlinear frequency estimation problem as a linear model whose parameters are updated iteratively according to weighted least squares (WLS). Since the derivation of the IAA is based on the l2-norm, it cannot work properly in a heavy-tailed noise environment. Here, a robust version of the IAA, hereafter referred to as the lp-IAA, is devised to provide an accurate parameter estimation in the impulsive noise environment. In this version, the weighted l2-norm is replaced by the weighted lp-norm with 1 < p < 2. Moreover, the lp-IAA is robust enough to resist outliers in the presence of SαS with 1 < α < 2 and GM distributions. However, this algorithm is only applicable for the case of p > 1. To solve the estimation problem in cases involving Cauchy or GM distributions, we develop an extension called the l1-IAA. On the basis of a semiparametric approach, namely, sparse learning via iterative minimization (SLIM), we also derive the l1-SLIM. Numerical results show that both the l1-IAA and l1-SLIM are superior to the lp-IAA, IAA, and SLIM in the Cauchy and GM noise environments.

    Research areas

  • Parameter estimation, Algorithms