An Efficient Sparse Bayesian Learning Algorithm Based on Gaussian-Scale Mixtures

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

3 Scopus Citations
View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Pages (from-to)3065-3078
Journal / PublicationIEEE Transactions on Neural Networks and Learning Systems
Volume33
Issue number7
Online published22 Jan 2021
Publication statusPublished - Jul 2022

Abstract

Sparse Bayesian learning (SBL) is a popular machine learning approach with a superior generalization capability due to the sparsity of its adopted model. However, it entails a matrix inversion at each iteration, hindering its practical applications with large-scale data sets. To overcome this bottleneck, we propose an efficient SBL algorithm with O(n²) computational complexity per iteration based on a Gaussian-scale mixture prior model. By specifying two different hyperpriors, the proposed efficient SBL algorithm can meet two different requirements, such as high efficiency and high sparsity. A surrogate function is introduced herein to approximate the posterior density of model parameters and thereby to avoid matrix inversions. Using a data-dependent term, a joint cost function with separate penalty terms is reformulated in a joint space of model parameters and hyperparameters. The resulting nonconvex optimization problem is solved using a block coordinate descent method in a majorization-minimization framework. Finally, the results of extensive experiments for sparse signal recovery and sparse image reconstruction on benchmark problems are elaborated to substantiate the effectiveness and superiority of the proposed approach in terms of computational time and estimation error.

Research Area(s)

  • Approximation algorithms, Bayes methods, Computational modeling, Covariance matrices, Gamma distribution, Gaussian scale mixture, GSM, iterative algorithms, Kernel, optimization methods, regression, sparse Bayesian learning (SBL)