A Layer-by-Layer Least Squares based Recurrent Networks Training Algorithm : Stalling and Escape

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

6 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)15-25
Journal / PublicationNeural Processing Letters
Volume7
Issue number1
Publication statusPublished - 1998

Abstract

The limitations of the least squares based training algorithm is dominated by stalling problem and evaluation error by transformation matrix to obtain an unacceptable solution. This paper presents a new approach for the recurrent networks training algorithm based upon the Layer-by-Layer Least Squares based algorithm to overcome the aforementioned problems. In accordance with our proposed algorithm, all the weights are evaluated by the least squares method without the evaluation of transformation matrix to speed up the rate of convergence. A probabilistic mechanism, based upon the modified weights updated equations, is introduced to eliminate the stalling problem experienced by the pure least squares type computation. As a result, the merits of the proposed algorithm are capable of providing an ability of escaping from local minima to obtain a good optimal solution and still maintaining the characteristic of fast convergence.

Research Area(s)

  • Convergence stalling, Fast convergence speed, Layer-by-Layer Least Squares algorithm, Recurrent networks