Ebook: Second-Order Methods for Neural Networks: Fast and Reliable Training Methods for Multi-Layer Perceptrons

About This Book This book is about training methods - in particular, fast second-order training methods - for multi-layer perceptrons (MLPs). MLPs (also known as feed-forward neural networks) are the most widely-used class of neural network. Over the past decade MLPs have achieved increasing popularity among scientists, engineers and other professionals as tools for tackling a wide variety of information processing tasks. In common with all neural networks, MLPsare trained (rather than programmed) to carryout the chosen information processing function. Unfortunately, the (traditional' method for trainingMLPs- the well-knownbackpropagation method - is notoriously slow and unreliable when applied to many prac­ tical tasks. The development of fast and reliable training algorithms for MLPsis one of the most important areas ofresearch within the entire field of neural computing. The main purpose of this book is to bring to a wider audience a range of alternative methods for training MLPs, methods which have proved orders of magnitude faster than backpropagation when applied to many training tasks. The book also addresses the well-known (local minima' problem, and explains ways in which fast training methods can be com­ bined with strategies for avoiding (or escaping from) local minima. All the methods described in this book have a strong theoretical foundation, drawing on such diverse mathematical fields as classical optimisation theory, homotopic theory and stochastic approximation theory.

---

This volume aims to develop the reader's understanding of the theoretical and practical issues involved in the development of efficient MLP training strategies, and to describe and evaluate the performance of a wide range of specific training algorithm. Particular emphasis is given to the development of methods which a strong theoretical foundation, rather than heuristic, "rule of thumb" training strategies. Second-Order Methods for Neural Networks will be of interest to academic researchers and postgraduate students working with neural networks (especially supervised learning with multi-layer perceptrons), industrial researchers and programmers developing neural network software, and professionals using neural networks as optimisation tools.

---

This volume aims to develop the reader's understanding of the theoretical and practical issues involved in the development of efficient MLP training strategies, and to describe and evaluate the performance of a wide range of specific training algorithm. Particular emphasis is given to the development of methods which a strong theoretical foundation, rather than heuristic, "rule of thumb" training strategies. Second-Order Methods for Neural Networks will be of interest to academic researchers and postgraduate students working with neural networks (especially supervised learning with multi-layer perceptrons), industrial researchers and programmers developing neural network software, and professionals using neural networks as optimisation tools.
Content:
Front Matter....Pages i-xiv
Multi-Layer Perceptron Training....Pages 1-22
Classical Optimisation....Pages 23-42
Second-Order Optimisation Methods....Pages 43-71
Second-Order Training Methods for MLPs....Pages 73-86
An Experimental Comparison of MLP Training Methods....Pages 87-117
Global Optimisation....Pages 119-131
Back Matter....Pages 133-145

---

This volume aims to develop the reader's understanding of the theoretical and practical issues involved in the development of efficient MLP training strategies, and to describe and evaluate the performance of a wide range of specific training algorithm. Particular emphasis is given to the development of methods which a strong theoretical foundation, rather than heuristic, "rule of thumb" training strategies. Second-Order Methods for Neural Networks will be of interest to academic researchers and postgraduate students working with neural networks (especially supervised learning with multi-layer perceptrons), industrial researchers and programmers developing neural network software, and professionals using neural networks as optimisation tools.
Content:
Front Matter....Pages i-xiv
Multi-Layer Perceptron Training....Pages 1-22
Classical Optimisation....Pages 23-42
Second-Order Optimisation Methods....Pages 43-71
Second-Order Training Methods for MLPs....Pages 73-86
An Experimental Comparison of MLP Training Methods....Pages 87-117
Global Optimisation....Pages 119-131
Back Matter....Pages 133-145
....