An Energy Efficient Accelerator for Bidirectional Recurrent Neural Networks (BiRNNs) Using Hybrid-Iterative Compression With Error Sensitivity

2021 ◽  
pp. 1-12
Author(s):  
Guocai Nan ◽  
Zhengkuan Wang ◽  
Chenghua Wang ◽  
Bi Wu ◽  
Zhican Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Energy Efficient ◽  
Error Sensitivity ◽  
Iterative Compression

scholarly journals Spatially Arranged Sparse Recurrent Neural Networks for Energy Efficient Associative Memory

2020 ◽  
Vol 31 (1) ◽  
pp. 24-38 ◽  
Author(s):  
Gouhei Tanaka ◽  
Ryosho Nakane ◽  
Tomoya Takeuchi ◽  
Toshiyuki Yamane ◽  
Daiju Nakano ◽  
...  
Keyword(s):  
Neural Networks ◽  
Associative Memory ◽  
Recurrent Neural Networks ◽  
Energy Efficient

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

Sign in / Sign up

Export Citation Format

Share Document