Experimental studies of memory surfaces and learning surfaces in recurrent neural networks

1994 ◽  
Vol 25 (8) ◽  
pp. 27-39
Author(s):  
Tatsumi Watanabe ◽  
Yoshiki Uchikawa ◽  
Kazutoshi Gouhara
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies

MONITORING AND CLASSIFYING THE STATE OF HARD DISKS USING RECURRENT NEURAL NETWORKS

2021 ◽  
pp. 36-43
Author(s):  
L. A. Demidova ◽  
A. V. Filatov
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies ◽  
Machine Learning Algorithms ◽  
Classification Model ◽  
Dense Layer ◽  
Hard Disks ◽  
Destructive Testing ◽  
Data Set ◽  
State Classification

The article considers an approach to solving the problem of monitoring and classifying the states of hard disks, which is solved on a regular basis, within the framework of the concept of non-destructive testing. It is proposed to solve this problem by developing a classification model using machine learning algorithms, in particular, using recurrent neural networks with Simple RNN, LSTM and GRU architectures. To develop a classification model, a data set based on the values of SMART sensors installed on hard disks it used. It represents a group of multidimensional time series. At the same time, the structure of the classification model contains two layers of a neural network with one of the recurrent architectures, as well as a Dropout layer and a Dense layer. The results of experimental studies confirming the advantages of LSTM and GRU architectures as part of hard disk state classification models are presented.

Recurrent Neural Networks with Small Weights Implement Definite Memory Machines

2003 ◽  
Vol 15 (8) ◽  
pp. 1897-1929 ◽  
Author(s):  
Barbara Hammer ◽  
Peter Tiňo
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies ◽  
Activation Function ◽  
Transition Function ◽  
Recurrent Network ◽  
Point Of View ◽  
Recurrent Networks ◽  
Contraction Parameter ◽  
Arbitrary Precision

Recent experimental studies indicate that recurrent neural networks initialized with “small” weights are inherently biased toward definite memory machines (Tiňno, Čerňanský, & Beňušková, 2002a, 2002b). This article establishes a theoretical counterpart: transition function of recurrent network with small weights and squashing activation function is a contraction. We prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine. Conversely, every definite memory machine can be simulated by a recurrent network with contractive transition function. Hence, initialization with small weights induces an architectural bias into learning with recurrent neural networks. This bias might have benefits from the point of view of statistical learning theory: it emphasizes one possible region of the weight space where generalization ability can be formally proved. It is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct (PAC) sense if arbitrary precision and inputs are considered. We prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so-called distribution independent uniform convergence of empirical distances property and hence, unlike general recurrent networks, are distribution independent PAC learnable.

scholarly journals Artificial neural network technology for lips reading

2021 ◽  
Vol 333 ◽  
pp. 01009
Author(s):  
Anna Pyataeva ◽  
Anton Dzyuba
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Video Sequence ◽  
Experimental Studies ◽  
Network Technology ◽  
Haar Features ◽  
Two Stages ◽  
Face Search ◽  
Neural Network Technology

The paper presents the use of neural networks for the task of automated speech reading by lips articulation. Speech recognition is performed in two stages. First, a face search is performed and the lips area is selected in a separate frame of the video sequence using Haar features. Then the sequence of frames goes to the input of deep learning convolutional and recurrent neural networks for speech viseme recognition. Experimental studies were carried out using independently obtained videos with Russian-speaking speakers.

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

LatticeRnn: Recurrent Neural Networks Over Lattices

2016 ◽  
Author(s):  
Faisal Ladhak ◽  
Ankur Gandhe ◽  
Markus Dreyer ◽  
Lambert Mathias ◽  
Ariya Rastrow ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks
Sign in / Sign up

Export Citation Format

Share Document