Fault tolerant learning using kullback-leibler divergence

John Sum; Chi-Sing Leung; Lipin Hsu

doi:10.1109/TENCON.2007.4429073

Fault tolerant learning using kullback-leibler divergence

John Sum
, Chi-Sing Leung
, Lipin Hsu

Department of Electrical Engineering

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

2 Citations (Scopus)

Abstract

In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation results have demonstrated that the RBF network trained in accordance with the new objective function is of better fault tolerance ability, in compared with the one trained by explicit regularization. As KL divergence has relation to Bayesian learning, a discussion on the proposed objective function and the other Bayesian type objective functions is discussed. ©2007 IEEE.

Original language	English
Title of host publication	IEEE Region 10 Annual International Conference, Proceedings/TENCON
DOIs	https://doi.org/10.1109/TENCON.2007.4429073
Publication status	Published - 2007
Event	IEEE Region 10 Conference, TENCON 2007 - Taipei, Taiwan, China Duration: 30 Oct 2007 → 2 Nov 2007

Conference

Conference	IEEE Region 10 Conference, TENCON 2007
Place	Taiwan, China
City	Taipei
Period	30/10/07 → 2/11/07

Access Output

10.1109/TENCON.2007.4429073

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{0eb8ef6fa6eb4a13957a2c44206aee25,

title = "Fault tolerant learning using kullback-leibler divergence",

abstract = "In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation results have demonstrated that the RBF network trained in accordance with the new objective function is of better fault tolerance ability, in compared with the one trained by explicit regularization. As KL divergence has relation to Bayesian learning, a discussion on the proposed objective function and the other Bayesian type objective functions is discussed. {\textcopyright}2007 IEEE.",

author = "John Sum and Chi-Sing Leung and Lipin Hsu",

year = "2007",

doi = "10.1109/TENCON.2007.4429073",

language = "English",

isbn = "1424412722",

booktitle = "IEEE Region 10 Annual International Conference, Proceedings/TENCON",

note = "IEEE Region 10 Conference, TENCON 2007 ; Conference date: 30-10-2007 Through 02-11-2007",

}

TY - GEN

T1 - Fault tolerant learning using kullback-leibler divergence

AU - Sum, John

AU - Leung, Chi-Sing

AU - Hsu, Lipin

PY - 2007

Y1 - 2007

N2 - In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation results have demonstrated that the RBF network trained in accordance with the new objective function is of better fault tolerance ability, in compared with the one trained by explicit regularization. As KL divergence has relation to Bayesian learning, a discussion on the proposed objective function and the other Bayesian type objective functions is discussed. ©2007 IEEE.

AB - In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation results have demonstrated that the RBF network trained in accordance with the new objective function is of better fault tolerance ability, in compared with the one trained by explicit regularization. As KL divergence has relation to Bayesian learning, a discussion on the proposed objective function and the other Bayesian type objective functions is discussed. ©2007 IEEE.

UR - https://www.scopus.com/pages/publications/48749093753

UR - https://www.scopus.com/pages/publications/48749093753#tab=impact

U2 - 10.1109/TENCON.2007.4429073

DO - 10.1109/TENCON.2007.4429073

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 1424412722

SN - 9781424412723

BT - IEEE Region 10 Annual International Conference, Proceedings/TENCON

T2 - IEEE Region 10 Conference, TENCON 2007

Y2 - 30 October 2007 through 2 November 2007

ER -

Fault tolerant learning using kullback-leibler divergence

Abstract

Conference

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this