Training recurrent neural networks using optimization layer-by-layer recursive least squares algorithm for vibration signals system identification and fault diagnostic analysis

Depending on the representation ability and the dynamic capability of recurrent neural networks for nonlinear models, a neural-based approach for general practical nonlinear systems identification is demonstrated in this paper. The paper presents a new method for the training of recurrent neural networks, namely the Optimization Layer-by-Layer Recursive Least Squares (OLL-RLS) algorithm. The method is based upon the weights to be optimized in layer-by-layer fashion, using the standard RLS method to speed up the convergence rate. The derivations of the OLL-RLS algorithm for the recurrent networks and the difficulties by the numerical instability and convergence stalling are also included and addressed in this paper. These difficulties can be tackled by introducing noise to the OLL-RLS algorithm, which enhances the convergence property significantly. Further, the convergence analysis and the computational complexity of the OLL-RLS algorithm are discussed. The validation results show that when applying to the systems identification, the novel OLL-RLS algorithm significantly outperforms other training algorithms, in both convergence speed and generalization capability. The proposed neural-based approach is also applied to the railway carriage system identification by identifying the train vibration signal. The identification results are very promising and show that the OLL-RLS algorithm can provide a significant contribution to fault diagnostic analysis by monitoring the identification error characteristics.