From Least Squares to Signal Processing and Particle Filtering

De facto, signal processing is the interpolation and extrapolation of a sequence of observations viewed as a realization of a stochastic process. Its role in applied statistics ranges from scenarios in forecasting and time series analysis to image reconstruction, machine learning, and the degradation modeling for reliability assessment. This topic, which has an old and honorable history dating back to the times of Gauss and Legendre, should therefore be of interest to readers of Technometrics. A general solution to the problem of filtering and prediction entails some formidable mathematics. Efforts to circumvent the mathematics has resulted in the need for introducing more explicit descriptions of the underlying process. One such example, and a noteworthy one, is the Kalman filter model, which is a special case of state space models or what statisticians refer to as dynamic linear models. Implementing the Kalman filter model in the era of “big and high velocity non-Gaussian data” can pose computational challenges with respect to efficiency and timeliness. Particle filtering is a way to ease such computational burdens. The purpose of this article is to trace the historical evolution of this development from its inception to its current state, with an expository focus on two versions of the particle filter, namely, the propagate first-update next and the update first-propagate next version. By way of going beyond a pure review, this article also makes transparent the importance and the role of a less recognized principle, namely, the principle of conditionalization, in filtering and prediction based on Bayesian methods. Furthermore, the article also articulates the philosophical underpinnings of the filtering and prediction set-up, a matter that needs to be made explicit, and Yule's decomposition of a random variable in terms of a sequence of innovations.