Characterization of training errors in supervised learning using gradient-based rules

In the majority of the existing supervised paradigms, a neural network is trained by minimizing an error function using a learning rule. The commonly used learning rules are gradient-based learning rules such as the popular backpropagation algorithm. This paper addresses an important issue on error minimization in supervised learning of neural networks using gradient-based learning rules. This paper characterizes asymptotic properties of training errors for various forms of neural networks in supervised learning and discusses their practical implications for designing neural networks via remarks and examples.