Convergence of the generalized back-propagation algorithm with constant learning rates

S. C. Ng; S. H. Leung; A. Luk

doi:10.1109/IJCNN.1998.685924

Convergence of the generalized back-propagation algorithm with constant learning rates

S. C. Ng, S. H. Leung, A. Luk

Department of Electrical Engineering

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

5 Citations (Scopus)

Abstract

A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima has been recently introduced in [1]. In this paper, we will analyze the convergence of the generalized back-propagation algorithm. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.

Original language	English
Title of host publication	The 11998 IEEE International Joint Conference on Neural Networks Proceedings
Subtitle of host publication	IEEE World Congress con Computational Intelligence
Publisher	IEEE
Pages	1090-1094
Volume	2
ISBN (Print)	0-7803-4859-1
DOIs	https://doi.org/10.1109/IJCNN.1998.685924
Publication status	Published - May 1998
Event	1998 IEEE International Joint Conference on Neural Networks (IJCNN 1998) - Anchorage, AK, USA Duration: 4 May 1998 → 9 May 1998

Publication series

Name
ISSN (Print)	1098-7576

Conference

Conference	1998 IEEE International Joint Conference on Neural Networks (IJCNN 1998)
City	Anchorage, AK, USA
Period	4/05/98 → 9/05/98

Access Output

10.1109/IJCNN.1998.685924

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{eee2892ecff149a1a381f4e850aad344,

title = "Convergence of the generalized back-propagation algorithm with constant learning rates",

abstract = "A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima has been recently introduced in [1]. In this paper, we will analyze the convergence of the generalized back-propagation algorithm. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.",

author = "Ng, {S. C.} and Leung, {S. H.} and A. Luk",

year = "1998",

month = may,

doi = "10.1109/IJCNN.1998.685924",

language = "English",

isbn = "0-7803-4859-1",

volume = "2",

publisher = "IEEE",

pages = "1090--1094",

booktitle = "The 11998 IEEE International Joint Conference on Neural Networks Proceedings",

address = "United States",

note = "1998 IEEE International Joint Conference on Neural Networks (IJCNN 1998) ; Conference date: 04-05-1998 Through 09-05-1998",

}

Ng, SC, Leung, SH & Luk, A 1998, Convergence of the generalized back-propagation algorithm with constant learning rates. in The 11998 IEEE International Joint Conference on Neural Networks Proceedings : IEEE World Congress con Computational Intelligence. vol. 2, IEEE, pp. 1090-1094, 1998 IEEE International Joint Conference on Neural Networks (IJCNN 1998), Anchorage, AK, USA, 4/05/98. https://doi.org/10.1109/IJCNN.1998.685924

Convergence of the generalized back-propagation algorithm with constant learning rates. / Ng, S. C.; Leung, S. H.; Luk, A.
The 11998 IEEE International Joint Conference on Neural Networks Proceedings : IEEE World Congress con Computational Intelligence. Vol. 2 IEEE, 1998. p. 1090-1094.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Convergence of the generalized back-propagation algorithm with constant learning rates

AU - Ng, S. C.

AU - Leung, S. H.

AU - Luk, A.

PY - 1998/5

Y1 - 1998/5

N2 - A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima has been recently introduced in [1]. In this paper, we will analyze the convergence of the generalized back-propagation algorithm. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.

AB - A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima has been recently introduced in [1]. In this paper, we will analyze the convergence of the generalized back-propagation algorithm. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.

UR - http://www.scopus.com/inward/record.url?scp=0031619046&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-0031619046&origin=recordpage

U2 - 10.1109/IJCNN.1998.685924

DO - 10.1109/IJCNN.1998.685924

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 0-7803-4859-1

VL - 2

SP - 1090

EP - 1094

BT - The 11998 IEEE International Joint Conference on Neural Networks Proceedings

PB - IEEE

T2 - 1998 IEEE International Joint Conference on Neural Networks (IJCNN 1998)

Y2 - 4 May 1998 through 9 May 1998

ER -

Convergence of the generalized back-propagation algorithm with constant learning rates

Abstract

Publication series

Conference

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this