Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm

2016 ◽  
Vol 25 (06) ◽  
pp. 1650033 ◽  
Author(s):  
Hossam Faris ◽  
Ibrahim Aljarah ◽  
Nailah Al-Madi ◽  
Seyedali Mirjalili
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Optimization Technique ◽  
Back Propagation ◽  
Feedforward Neural Networks ◽  
Training Algorithms ◽  
Local Optima ◽  
Local Solutions

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

ANALYSIS OF MULTIDIMENSIONAL XOR CLASSIFICATION PROBLEM WITH EVOLUTIONARY FEEDFORWARD NEURAL NETWORKS

2007 ◽  
Vol 16 (01) ◽  
pp. 111-120 ◽  
Author(s):  
MANISH MANGAL ◽  
MANU PRATAP SINGH
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Search Algorithm ◽  
Random Search ◽  
Feedforward Neural Networks ◽  
Feedforward Neural Network ◽  
Data Sets ◽  
Classification Problems ◽  
Random Search Algorithm

This paper describes the application of two evolutionary algorithms to the feedforward neural networks used in classification problems. Besides of a simple backpropagation feedforward algorithm, the paper considers the genetic algorithm and random search algorithm. The objective is to analyze the performance of GAs over the simple backpropagation feedforward in terms of accuracy or speed in this problem. The experiments considered a feedforward neural network trained with genetic algorithm/random search algorithm and 39 types of network structures and artificial data sets. In most cases, the evolutionary feedforward neural networks seemed to have better of equal accuracy than the original backpropagation feedforward neural network. We found few differences in the accuracy of the networks solved by applying the EAs, but found ample differences in the execution time. The results suggest that the evolutionary feedforward neural network with random search algorithm might be the best algorithm on the data sets we tested.

scholarly journals Using Feed Forward Neural Network to Solve Eigenvalue Problems

2014 ◽  
Vol 2014 ◽  
pp. 1-8
Author(s):  
Luma N. M. Tawfiq ◽  
Othman M. Salih
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Eigenvalue Problems ◽  
Back Propagation ◽  
Training Algorithms ◽  
Parallel Processor ◽  
Feed Forward Neural Network ◽  
Levenberg Marquardt ◽  
Quasi Newton

The aim of this paper is to presents a parallel processor technique for solving eigenvalue problem for ordinary differential equations using artificial neural networks. The proposed network is trained by back propagation with different training algorithms quasi-Newton, Levenberg-Marquardt, and Bayesian Regulation. The next objective of this paper was to compare the performance of aforementioned algorithms with regard to predicting ability.

scholarly journals Artificial neural network for the recognition of human emotions under a backpropagation algorithm

Athenea ◽  
2021 ◽  
Vol 2 (5) ◽  
pp. 29-34
Author(s):  
Alexander Caicedo ◽  
Anthony Caicedo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Information Processing ◽  
Emotion Recognition ◽  
Buenos Aires ◽  
Back Propagation ◽  
Feedforward Neural Networks ◽  
Backpropagation Algorithm ◽  
Neural Information ◽  
Neural Information Processing

The era of the technological revolution increasingly encourages the development of technologies that facilitate in one way or another people's daily activities, thus generating a great advance in information processing. The purpose of this work is to implement a neural network that allows classifying the emotional states of a person based on the different human gestures. A database is used with information on students from the PUCE-E School of Computer Science and Engineering. Said information are images that express the gestures of the students and with which the comparative analysis with the input data is carried out. The environment in which this work converges proposes that the implementation of this project be carried out under the programming of a multilayer neuralnetwork. Multilayer feeding neural networks possess a number of properties that make them particularly suitable for complex pattern classification problems [8]. Back-Propagation [4], which is a backpropagation algorithm used in the Feedforward neural network, was taken into consideration to solve the classification of emotions. Keywords: Image processing, neural networks, gestures, back-propagation, feedforward, classification, emotions. References [1]S. Gangwar, S. Shukla, D. Arora. “Human Emotion Recognition by Using Pattern Recognition Network”, Journal of Engineering Research and Applications, Vol. 3, Issue 5, pp.535-539, 2013. [2]K. Rohit. “Back Propagation Neural Network based Emotion Recognition System”. International Journal of Engineering Trends and Technology (IJETT), Vol. 22, Nº 4, 2015. [3]S. Eishu, K. Ranju, S. Malika, “Speech Emotion Recognition using BFO and BPNN”, International Journal of Advances in Science and Technology (IJAST), ISSN2348-5426, Vol. 2 Issue 3, 2014. [4]A. Fiszelew, R. García-Martínez and T. de Buenos Aires. “Generación automática de redes neuronales con ajuste de parámetros basado en algoritmos genéticos”. Revista del Instituto Tecnológico de Buenos Aires, 26, 76-101, 2002. [5]Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel. “Handwritten digit recognition with a back-propagation network”. In Advances in neural information processing systems. pp. 396-404, 1990. [6]G. Bebis and M. Georgiopoulos. “Feed-forward neural networks”. IEEE Potentials, 13(4), 27-31, 1994. [7]G. Huang, Q. Zhu and C. Siew. “Extreme learning machine: a new learning scheme of feedforward neural networks”. In Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference. Vol. 2, pp. 985-990. IEEE, 2004. [8]D. Montana and L. Davis. “Training Feedforward Neural Networks Using Genetic Algorithms”. In IJCAI, Vol. 89, pp. 762-767, 1989. [9]I. Sutskever, O. Vinyals and Q. Le. “Sequence to sequence learning with neural networks”. In Advances in neural information processing systems. pp. 3104-3112, 2014. [10]J. Schmidhuber. “Deep learning in neural networks: An overview”. Neural networks, 61, 85-117, 2015. [11]R. Santos, M. Ruppb, S. Bonzi and A. Filetia, “Comparación entre redes neuronales feedforward de múltiples capas y una red de función radial para detectar y localizar fugas en tuberías que transportan gas”. Chem. Ing.Trans , 32 (1375), e1380, 2013.

scholarly journals A Review of Heuristic Global Optimization Based Artificial Neural Network Training Approahes

2017 ◽  
Vol 6 (1) ◽  
pp. 26
Author(s):  
D. Geraldine Bessie Amali ◽  
Dinakaran M.
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Global Optimization ◽  
Optimization Algorithms ◽  
Feedforward Neural Networks ◽  
Training Algorithms ◽  
Neural Network Training ◽  
Network Training ◽  
Multimodal Function ◽  
Artificial Neural

Artificial Neural Networks have earned popularity in recent years because of their ability to approximate nonlinear functions. Training a neural network involves minimizing the mean square error between the target and network output. The error surface is nonconvex and highly multimodal. Finding the minimum of a multimodal function is a NP complete problem and cannot be solved completely. Thus application of heuristic global optimization algorithms that computes a good global minimum to neural network training is of interest. This paper reviews the various heuristic global optimization algorithms used for training feedforward neural networks and recurrent neural networks. The training algorithms are compared in terms of the learning rate, convergence speed and accuracy of the output produced by the neural network. The paper concludes by suggesting directions for novel ANN training algorithms based on recent advances in global optimization.

scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

scholarly journals Optical Recognition of Handwritten Logic Formulas Using Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2761
Author(s):  
Vaios Ampelakiotis ◽  
Isidoros Perikos ◽  
Ioannis Hatzilygeroudis ◽  
George Tsihrintzis
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Gradient Descent ◽  
Feedforward Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Gradient Descent Algorithm ◽  
Two Stages ◽  
And Training

In this paper, we present a handwritten character recognition (HCR) system that aims to recognize first-order logic handwritten formulas and create editable text files of the recognized formulas. Dense feedforward neural networks (NNs) are utilized, and their performance is examined under various training conditions and methods. More specifically, after three training algorithms (backpropagation, resilient propagation and stochastic gradient descent) had been tested, we created and trained an NN with the stochastic gradient descent algorithm, optimized by the Adam update rule, which was proved to be the best, using a trainset of 16,750 handwritten image samples of 28 × 28 each and a testset of 7947 samples. The final accuracy achieved is 90.13%. The general methodology followed consists of two stages: the image processing and the NN design and training. Finally, an application has been created that implements the methodology and automatically recognizes handwritten logic formulas. An interesting feature of the application is that it allows for creating new, user-oriented training sets and parameter settings, and thus new NN models.

scholarly journals Rain–Induced Flood Prediction for Niger Delta Sub-Region of Nigeria Using Neural Networks

2020 ◽  
Vol 5 (9) ◽  
pp. 1124-1130
Author(s):  
Ledisi Giok Kabari ◽  
Young Claudius Mazi
Keyword(s):  
Neural Network ◽  
Climate Change ◽  
Neural Networks ◽  
Niger Delta ◽  
Feedforward Neural Networks ◽  
Flood Prediction ◽  
The Neural Network ◽  
Significant Damage ◽  
Weather Underground ◽  
Short Term Prediction

Climate change generates so many direct and indirect effects on the environment.  Some of those effects have serious consequences. Rain-induced flooding is one of the direct effects of climate change and its impact on the environment is usually devastating and worrisome. Floods are one of the most commonly occurring disasters and have caused significant damage to life, including agriculture and economy. They are usually caused in areas where there is excessive downpour and poor drainage systems. The study uses Feedforward Multilayer Neural Network to perform short-term prediction of the amount of rainfall flood for the Niger Delta   sub region of Nigeria given previous rainfall data for a specified period of time. The data for training and testing of the Neural Network was sourced from Weather Underground official web site https://www.wunderground.com.  An iterative Methodology was used and implemented in MATLAB. We adopted multi-layer Feedforward Neural Networks. The study accurately predicts the rain-induced flood for the Niger Delta   sub region of Nigeria.

Sign in / Sign up

Export Citation Format

Share Document