Algorithm development for robust parameter estimation
魯棒參數估計演算法的研究
Student thesis: Doctoral Thesis
Author(s)
Related Research Unit(s)
Detail(s)
Awarding Institution  

Supervisors/Advisors 

Award date  16 Feb 2015 
Link(s)
Permanent Link  https://scholars.cityu.edu.hk/en/theses/theses(e4ee53012eef43369c6881a3555f99e7).html 

Other link(s)  Links 
Abstract
Parameter estimation is a common task required in the fields of science and engineering.
It refers to accurately finding the values of interested parameters from observed
data, which consist of two components, namely, signal and noise. The numerous estimators
developed in the literature can be divided into three categories: parametric,
nonparametric, and semiparametric techniques. In parametric approaches, the signal
is assumed to satisfy a generating model with a known functional form. In nonparametric
methodologies, no assumptions are made about the signal. In semiparametricbased
algorithms, a priori knowledge on the signal is available. As parametricbased
algorithms allow the derivation of optimal estimators when the assumed model has
good approximation to reality, they are widely used to solve parameter estimation
problems.
Conventionally, we assume that the noise model follows the Gaussian distribution.
However, this assumption is not suitable to represent all varieties of noise in the real
world, in which noise may exhibit heavytailed characteristics. A heavytailed distribution,
whose tails are longer than those of a normal distribution, is prone to the
generation of large values in the time domain. In the literature, the typical models
for impulsive noise are single processes and hybrid processes mixed in the probability
density function (PDF) domain. The representative single distributions are αstable,
generalized Gaussian (GG), and Student's t distributions while the PDF mixture
includes the Gaussian mixture (GM) process. In this thesis, a new noise model
mixed in the time domain, namely, a symmetric αstable Gaussian (SαSG) mixture,
is proposed to describe the noise in some applications, such as in astrophysical image
processing and indoor communications. Given the existence of outliers in a heavytailed
distribution, traditional estimators designed for Gaussian distribution cannot
provide a reliable performance.
Therefore, robust parameter estimation is considerably
important in ensuring that reliable estimates are obtained in impulsive noise.
We also focus on developing robust parametric algorithms and conducting relevant
performance analysis. We also discuss the robust nonparametric and semiparametric
methods.
Performance analysis is extremely important in robust signal processing for it
evaluates the quality of an estimator and determines whether it is robust. Bias and
mean square error (MSE) are the most basic performance measures. Employing the
Taylor series expansion (TSE) on cost function, bias and MSE formulas are derived in
the thesis. Then, we analyze the bias and MSE expressions of the lpnorm minimizer,
which is a robust parametric technique for impulsive noise. Four representative noise
models, namely, symmetric αstable (SαS), GG, Student's t, and GM processes, are
investigated, with consideration of both the linear and nonlinear parameter estimation
problems at p ≥ 1. The optimal choice of p for different noise environments is also
examined. The developed formulas are verified by comparing their outcomes with the
simulation results.
Although the least lpnorm estimator is nearly optimal for typical noise models,
it cannot provide good performance for the SαSG process. Therefore, finding an
optimal parametric estimator for the SαSG mixture is crucial and necessary. We
start with a special case, namely, the additive CauchyGaussian (ACG), which is a
sum of Cauchy and Gaussian random variables in the time domain. The PDF of the
ACG, referred to as the Voigt profile, is derived from the convolution of the Cauchy
and Gaussian PDFs.
To obtain the unknown parameters in both the linear and nonlinear problems under
the ACG noise, we devise a classical parametric method called the maximum likelihood
estimator (MLE). Given that the Voigt profile suffers from a complicated analytical
form, we also derive an Mestimator with the pseudoVoigt function (MEPV).
In our algorithm development, both scenarios involving known and unknown density
parameters are considered. For the latter case, the MLE and MEPV cannot be
applied to estimate the density parameters directly because of the complication of
the Voigt profile. Therefore, density parameters should be obtained by utilizing the
characteristic function (CF) and the empirical characteristic function (ECF) prior to
applying the proposed methods. Both the MLE and MEPV reach the CramerRao
lower bound (CRLB), and the MEPV has less computational cost than the MLE.
Aiming for density parameter estimation, we also propose a new suboptimal estimator,
referred to as the fractional lowerorder moment (FLOM) estimator, with FLOM
being calculated by the CF of the ACG.
As twostage estimation is required in both the MLE and MEPV, the performance
of both methods depends on the first stage. That is, if the density parameter estimates
are inaccurate, then the MLE and MEPV may be analytically intractable. This
condition motivates us to pursue a new parametric methodology that avoids density
parameter estimation. To satisfy this requirement, we employ a Markov chain Monte
Carlo (MCMC) method, namely, the MetropolisHastings (MH) algorithm. We then
estimate the linear signal embedded in the ACG mixture noise. Numerical results
show the optimality of the proposed algorithm in linear parameter estimation.
Nevertheless, the performance of the parametricbased method will degrade if a
mismatch appears between the assumed model and the actual signal. In the mismatching
condition, nonparametric and semiparametric methodologies play a crucial
role. Recently, a nonparametricbased algorithm called the iterative adaptive approach
(IAA) has been shown to be an effective highresolution spectral analysis tool.
It mainly involves the reformulation of the nonlinear frequency estimation problem as
a linear model whose parameters are updated iteratively according to weighted least
squares (WLS). Since the derivation of the IAA is based on the l2norm, it cannot
work properly in a heavytailed noise environment.
Here, a robust version of the IAA, hereafter referred to as the lpIAA, is devised to
provide an accurate parameter estimation in the impulsive noise environment. In this
version, the weighted l2norm is replaced by the weighted lpnorm with 1 < p < 2.
Moreover, the lpIAA is robust enough to resist outliers in the presence of SαS with
1 < α < 2 and GM distributions. However, this algorithm is only applicable for
the case of p > 1. To solve the estimation problem in cases involving Cauchy or
GM distributions, we develop an extension called the l1IAA. On the basis of a
semiparametric approach, namely, sparse learning via iterative minimization (SLIM),
we also derive the l1SLIM. Numerical results show that both the l1IAA and l1SLIM
are superior to the lpIAA, IAA, and SLIM in the Cauchy and GM noise environments.
 Parameter estimation, Algorithms