TY - JOUR
T1 - A Layer-by-Layer Least Squares based Recurrent Networks Training Algorithm
T2 - Stalling and Escape
AU - Cho, Siu-Yeung
AU - Chow, Tommy W.S.
PY - 1998
Y1 - 1998
N2 - The limitations of the least squares based training algorithm is dominated by stalling problem and evaluation error by transformation matrix to obtain an unacceptable solution. This paper presents a new approach for the recurrent networks training algorithm based upon the Layer-by-Layer Least Squares based algorithm to overcome the aforementioned problems. In accordance with our proposed algorithm, all the weights are evaluated by the least squares method without the evaluation of transformation matrix to speed up the rate of convergence. A probabilistic mechanism, based upon the modified weights updated equations, is introduced to eliminate the stalling problem experienced by the pure least squares type computation. As a result, the merits of the proposed algorithm are capable of providing an ability of escaping from local minima to obtain a good optimal solution and still maintaining the characteristic of fast convergence.
AB - The limitations of the least squares based training algorithm is dominated by stalling problem and evaluation error by transformation matrix to obtain an unacceptable solution. This paper presents a new approach for the recurrent networks training algorithm based upon the Layer-by-Layer Least Squares based algorithm to overcome the aforementioned problems. In accordance with our proposed algorithm, all the weights are evaluated by the least squares method without the evaluation of transformation matrix to speed up the rate of convergence. A probabilistic mechanism, based upon the modified weights updated equations, is introduced to eliminate the stalling problem experienced by the pure least squares type computation. As a result, the merits of the proposed algorithm are capable of providing an ability of escaping from local minima to obtain a good optimal solution and still maintaining the characteristic of fast convergence.
KW - Convergence stalling
KW - Fast convergence speed
KW - Layer-by-Layer Least Squares algorithm
KW - Recurrent networks
UR - http://www.scopus.com/inward/record.url?scp=0031989111&partnerID=8YFLogxK
UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-0031989111&origin=recordpage
M3 - 21_Publication in refereed journal
VL - 7
SP - 15
EP - 25
JO - Neural Processing Letters
JF - Neural Processing Letters
SN - 1370-4621
IS - 1
ER -