A Regularized Variable Projection Algorithm for Separable Nonlinear Least Squares Problems

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

132 Scopus Citations
View graph of relations



Original languageEnglish
Pages (from-to)526-537
Journal / PublicationIEEE Transactions on Automatic Control
Issue number2
Online published17 May 2018
Publication statusPublished - Feb 2019


Separable nonlinear least squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNLLS problems based on the VP method. Selecting an appropriate regularization parameter is difficult because of the nonlinear optimization procedure. We propose to determine the regularization parameter using the weighted generalized cross validation (WGCV) method at every iteration. This makes the original objective function changing during the optimization procedure. To circumvent this problem, we use an inequation to produce a consistent demand of decreasing at successive iterations. The approximation of the Jacobian of the regularized problem is also discussed. The proposed regularized VP algorithm is tested by the parameter estimation problem of several statistical models. Numerical results demonstrate the effectiveness of the proposed algorithm.

Research Area(s)

  • data fitting, regularization, Separable nonlinear least squares (SNLLS), variable projection (VP), weighted generalized cross validation (WGCV)