A VECTOR MATRIX REAL TIME RECURSIVE BACKPROPAGATION ALGORITHM FOR RECURRENT NEURAL NETWORKS THAT APPROXIMATE MULTI-VALUED PERIODIC FUNCTIONS

2009 ◽  
Vol 08 (04) ◽  
pp. 395-411 ◽  
Author(s):  
PETER STUBBERUD
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Recurrent Neural Networks ◽  
Feedforward Neural Networks ◽  
Universal Function ◽  
Multiple Delays ◽  
Feedback Path ◽  
Vector Matrix ◽  
Universal Approximators

Unlike feedforward neural networks (FFNN) which can act as universal function approximators, recursive, or recurrent, neural networks can act as universal approximators for multi-valued functions. In this paper, a real time recursive backpropagation (RTRBP) algorithm in a vector matrix form is developed for a two-layer globally recursive neural network that has multiple delays in its feedback path. This algorithm has been evaluated on two GRNNs that approximate both an analytic and nonanalytic periodic multi-valued function that a feedforward neural network is not capable of approximating.

Recurrent Neural Networks for Fault Diagnosis and Severity Assessment of a Screw Compressor

1999 ◽  
Vol 121 (4) ◽  
pp. 724-729 ◽  
Author(s):  
C. James Li ◽  
Yimin Fan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Experimental Validation ◽  
Feedforward Neural Networks ◽  
Previous Method ◽  
Feedforward Neural Network ◽  
Screw Compressor ◽  
Severity Assessment

This paper describes a method to diagnose the most frequent faults of a screw compressor and assess magnitude of these faults by tracking changes in compressor’s dynamics. To determine the condition of the compressor, a feedforward neural network model is first employed to identify the dynamics of the compressor. A recurrent neural network is then used to classify the model into one of the three conditions including baseline, gaterotor wear and excessive friction. Finally, another recurrent neural network estimates the magnitude of a fault from the model. The method’s ability to generalize was evaluated. Experimental validation of the method was also performed. The results show significant improvement over the previous method which used only feedforward neural networks.

A Low-Sensitivity Recurrent Neural Network

1998 ◽  
Vol 10 (1) ◽  
pp. 165-188 ◽  
Author(s):  
Andrew D. Back ◽  
Ah Chung Tsoi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
High Sensitivity ◽  
Feedforward Neural Networks ◽  
Parameter Sensitivity ◽  
Model Parameters ◽  
Low Sensitivity

The problem of high sensitivity in modeling is well known. Small perturbations in the model parameters may result in large, undesired changes in the model behavior. A number of authors have considered the issue of sensitivity in feedforward neural networks from a probabilistic perspective. Less attention has been given to such issues in recurrent neural networks. In this article, we present a new recurrent neural network architecture, that is capable of significantly improved parameter sensitivity properties compared to existing recurrent neural networks. The new recurrent neural network generalizes previous architectures by employing alternative discrete-time operators in place of the shift operator normally used. An analysis of the model demonstrates the existence of parameter sensitivity in recurrent neural networks and supports the proposed architecture. The new architecture performs significantly better than previous recurrent neural networks, as shown by a series of simple numerical experiments.

scholarly journals ECG Identification For Personal Authentication Using LSTM-Based Deep Recurrent Neural Networks

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3069 ◽  
Author(s):  
Beom-Hun Kim ◽  
Jae-Young Pyun
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Support Vector ◽  
Time System ◽  
Real Time System ◽  
Deep Recurrent Neural Network ◽  
Personal Authentication

Securing personal authentication is an important study in the field of security. Particularly, fingerprinting and face recognition have been used for personal authentication. However, these systems suffer from certain issues, such as fingerprinting forgery, or environmental obstacles. To address forgery or spoofing identification problems, various approaches have been considered, including electrocardiogram (ECG). For ECG identification, linear discriminant analysis (LDA), support vector machine (SVM), principal component analysis (PCA), deep recurrent neural network (DRNN), and recurrent neural network (RNN) have been conventionally used. Certain studies have shown that the RNN model yields the best performance in ECG identification as compared with the other models. However, these methods require a lengthy input signal for high accuracy. Thus, these methods may not be applied to a real-time system. In this study, we propose using bidirectional long short-term memory (LSTM)-based deep recurrent neural networks (DRNN) through late-fusion to develop a real-time system for ECG-based biometrics identification and classification. We suggest a preprocessing procedure for the quick identification and noise reduction, such as a derivative filter, moving average filter, and normalization. We experimentally evaluated the proposed method using two public datasets: MIT-BIH Normal Sinus Rhythm (NSRDB) and MIT-BIH Arrhythmia (MITDB). The proposed LSTM-based DRNN model shows that in NSRDB, the overall precision was 100%, recall was 100%, accuracy was 100%, and F1-score was 1. For MITDB, the overall precision was 99.8%, recall was 99.8%, accuracy was 99.8%, and F1-score was 0.99. Our experiments demonstrate that the proposed model achieves an overall higher classification accuracy and efficiency compared with the conventional LSTM approach.

scholarly journals The Performance Differences between Using Recurrent Neural Networks and Feedforward Neural Network in Sentiment Analysis Problem

2020 ◽  
pp. 1512-1524
Author(s):  
Samar K. Abaas ◽  
Loay E. George
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Social Media ◽  
Sentiment Analysis ◽  
Recurrent Neural Networks ◽  
Keyword Search ◽  
Performance Enhancement ◽  
Short Term Memory ◽  
Feedforward Neural Networks ◽  
Related Data

 With the spread use of internet, especially the web of social media, an unusual quantity of information is found that includes a number of study fields such as psychology, entertainment, sociology, business, news, politics, and other cultural fields of nations. Data mining methodologies that deal with social media allows producing enjoyable scene on the human behaviour and interaction. This paper demonstrates the application and precision of sentiment analysis using traditional feedforward and two of recurrent neural networks (gated recurrent unit (GRU) and long short term memory (LSTM)) to find the differences between them. In order to test the system’s performance, a set of tests is applied on two public datasets. The first dataset is collected data from IMDB that contains movie reviews expressed through long sentences of English, whereas the second dataset is a collection of keyword search results of tweets using the Twitter Search API; these tweets are written in English words with short sentences. In this work, a certain pre-processing operation is added to the system and a set of tests is conducted to evaluate the performance enhancement on the whole system due to the addition of these operations. The results of the usage of the traditional feedforward neural networks are poor and do not perform the desired purpose in analysis, because of their inability to save information at a long term and, therefore, their loss of efficiency. While the results of using GRU and LSTM are relatively good and do perform the desired purpose in analysis. A recurrent neural network has been built so that any type of text-related data can be pushed to get the polarity of sentiment by multi deep operations that are dependent on the extracted information.

Real-time Detection of Aortic Valve in Echocardiography using Convolutional Neural Networks

2020 ◽  
Vol 16 (5) ◽  
pp. 584-591 ◽  
Author(s):  
Muhammad Hanif Ahmad Nizar ◽  
Chow Khuen Chan ◽  
Azira Khalil ◽  
Ahmad Khairuddin Mohamed Yusof ◽  
Khin Wee Lai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Heart Disease ◽  
Aortic Valve ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Valvular Heart Disease ◽  
Detection System ◽  
Processing Unit ◽  
Real Time Detection

Background: Valvular heart disease is a serious disease leading to mortality and increasing medical care cost. The aortic valve is the most common valve affected by this disease. Doctors rely on echocardiogram for diagnosing and evaluating valvular heart disease. However, the images from echocardiogram are poor in comparison to Computerized Tomography and Magnetic Resonance Imaging scan. This study proposes the development of Convolutional Neural Networks (CNN) that can function optimally during a live echocardiographic examination for detection of the aortic valve. An automated detection system in an echocardiogram will improve the accuracy of medical diagnosis and can provide further medical analysis from the resulting detection. Methods: Two detection architectures, Single Shot Multibox Detector (SSD) and Faster Regional based Convolutional Neural Network (R-CNN) with various feature extractors were trained on echocardiography images from 33 patients. Thereafter, the models were tested on 10 echocardiography videos. Results: Faster R-CNN Inception v2 had shown the highest accuracy (98.6%) followed closely by SSD Mobilenet v2. In terms of speed, SSD Mobilenet v2 resulted in a loss of 46.81% in framesper- second (fps) during real-time detection but managed to perform better than the other neural network models. Additionally, SSD Mobilenet v2 used the least amount of Graphic Processing Unit (GPU) but the Central Processing Unit (CPU) usage was relatively similar throughout all models. Conclusion: Our findings provide a foundation for implementing a convolutional detection system to echocardiography for medical purposes.

scholarly journals Identification of Granule Growth Regimes in High Shear Wet Granulation Processes Using a Physics-Constrained Neural Network

Processes ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 737
Author(s):  
Chaitanya Sampat ◽  
Rohit Ramachandran
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Data Driven ◽  
Wet Granulation ◽  
Manufacturing Processes ◽  
High Shear ◽  
Process Data ◽  
Physical Constraints ◽  
Granule Growth

The digitization of manufacturing processes has led to an increase in the availability of process data, which has enabled the use of data-driven models to predict the outcomes of these manufacturing processes. Data-driven models are instantaneous in simulate and can provide real-time predictions but lack any governing physics within their framework. When process data deviates from original conditions, the predictions from these models may not agree with physical boundaries. In such cases, the use of first-principle-based models to predict process outcomes have proven to be effective but computationally inefficient and cannot be solved in real time. Thus, there remains a need to develop efficient data-driven models with a physical understanding about the process. In this work, we have demonstrate the addition of physics-based boundary conditions constraints to a neural network to improve its predictability for granule density and granule size distribution (GSD) for a high shear granulation process. The physics-constrained neural network (PCNN) was better at predicting granule growth regimes when compared to other neural networks with no physical constraints. When input data that violated physics-based boundaries was provided, the PCNN identified these points more accurately compared to other non-physics constrained neural networks, with an error of <1%. A sensitivity analysis of the PCNN to the input variables was also performed to understand individual effects on the final outputs.

Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm

2016 ◽  
Vol 25 (06) ◽  
pp. 1650033 ◽  
Author(s):  
Hossam Faris ◽  
Ibrahim Aljarah ◽  
Nailah Al-Madi ◽  
Seyedali Mirjalili
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Optimization Technique ◽  
Back Propagation ◽  
Feedforward Neural Networks ◽  
Training Algorithms ◽  
Local Optima ◽  
Local Solutions

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

scholarly journals Real-time classification of hand movements as a basis for intuitive control of grasp neuroprostheses

2020 ◽  
Vol 6 (2) ◽  
Author(s):  
Dmitry Amelin ◽  
Ivan Potapov ◽  
Josep Cardona Audí ◽  
Andreas Kogut ◽  
Rüdiger Rupp ◽  
...  
Keyword(s):  
Neural Networks ◽  
Standard Deviation ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Recurrent Neural Networks ◽  
Healthy Subjects ◽  
Hand Movements ◽  
Cord Injury ◽  
Field Programmable

AbstractThis paper reports on the evaluation of recurrent and convolutional neural networks as real-time grasp phase classifiers for future control of neuroprostheses for people with high spinal cord injury. A field-programmable gate array has been chosen as an implementation platform due to its form factor and ability to perform parallel computations, which are specific for the selected neural networks. Three different phases of two grasp patterns and the additional open hand pattern were predicted by means of surface Electromyography (EMG) signals (i.e. Seven classes in total). Across seven healthy subjects, CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) had a mean accuracy of 85.23% with a standard deviation of 4.77% and 112 µs per prediction and 83.30% with a standard deviation of 4.36% and 40 µs per prediction, respectively.

scholarly journals Recurrent Neural Networks for Narrowband Signal Detection in the Time-Frequency Domain

2004 ◽  
Vol 213 ◽  
pp. 483-486
Author(s):  
David Brodrick ◽  
Douglas Taylor ◽  
Joachim Diederich
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Signal Detection ◽  
Frequency Domain ◽  
Recurrent Neural Networks ◽  
Radio Frequency Interference ◽  
Recurrent Networks ◽  
Time Frequency ◽  
Narrowband Signal ◽  
Radio Signals

A recurrent neural network was trained to detect the time-frequency domain signature of narrowband radio signals against a background of astronomical noise. The objective was to investigate the use of recurrent networks for signal detection in the Search for Extra-Terrestrial Intelligence, though the problem is closely analogous to the detection of some classes of Radio Frequency Interference in radio astronomy.

Sign in / Sign up

Export Citation Format

Share Document