scholarly journals Effective Training of Deep Convolutional Neural Networks for Hyperspectral Image Classification through Artificial Labeling

2020 ◽  
Vol 12 (16) ◽  
pp. 2653 ◽  
Author(s):  
Wojciech Masarczyk ◽  
Przemysław Głomb ◽  
Bartosz Grabowski ◽  
Mateusz Ostaszewski
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Network Architecture ◽  
Hyperspectral Image ◽  
Large Data ◽  
Learning Approaches ◽  
Classification Problems ◽  
Deep Convolutional Neural Networks ◽  
Additional Advantage ◽  
Hyperspectral Classification

Hyperspectral imaging is a rich source of data, allowing for a multitude of effective applications. However, such imaging remains challenging because of large data dimension and, typically, a small pool of available training examples. While deep learning approaches have been shown to be successful in providing effective classification solutions, especially for high dimensional problems, unfortunately they work best with a lot of labelled examples available. The transfer learning approach can be used to alleviate the second requirement for a particular dataset: first the network is pre-trained on some dataset with large amount of training labels available, then the actual dataset is used to fine-tune the network. This strategy is not straightforward to apply with hyperspectral images, as it is often the case that only one particular image of some type or characteristic is available. In this paper, we propose and investigate a simple and effective strategy of transfer learning that uses unsupervised pre-training step without label information. This approach can be applied to many of the hyperspectral classification problems. The performed experiments show that it is very effective at improving the classification accuracy without being restricted to a particular image type or neural network architecture. The experiments were carried out on several deep neural network architectures and various sizes of labeled training sets. The greatest improvement in overall accuracy on the Indian Pines and Pavia University datasets is over 21 and 13 percentage points, respectively. An additional advantage of the proposed approach is the unsupervised nature of the pre-training step, which can be done immediately after image acquisition, without the need of the potentially costly expert’s time.

scholarly journals Multispectral Face Recognition Using Transfer Learning with Adaptation of Domain Specific Units

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4520
Author(s):  
Luis Lopes Chambino ◽  
José Silvestre Silva ◽  
Alexandre Bernardino
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Facial Recognition ◽  
Spectral Band ◽  
Support Vector ◽  
Multispectral Images ◽  
Forgery Detection ◽  
K Nearest Neighbor ◽  
Deep Convolutional Neural Networks ◽  
Domain Specific

Facial recognition is a method of identifying or authenticating the identity of people through their faces. Nowadays, facial recognition systems that use multispectral images achieve better results than those that use only visible spectral band images. In this work, a novel architecture for facial recognition that uses multiple deep convolutional neural networks and multispectral images is proposed. A domain-specific transfer-learning methodology applied to a deep neural network pre-trained in RGB images is shown to generalize well to the multispectral domain. We also propose a skin detector module for forgery detection. Several experiments were planned to assess the performance of our methods. First, we evaluate the performance of the forgery detection module using face masks and coverings of different materials. A second study was carried out with the objective of tuning the parameters of our domain-specific transfer-learning methodology, in particular which layers of the pre-trained network should be retrained to obtain good adaptation to multispectral images. A third study was conducted to evaluate the performance of support vector machines (SVM) and k-nearest neighbor classifiers using the embeddings obtained from the trained neural network. Finally, we compare the proposed method with other state-of-the-art approaches. The experimental results show performance improvements in the Tufts and CASIA NIR-VIS 2.0 multispectral databases, with a rank-1 score of 99.7% and 99.8%, respectively.

Stable and accurate a posteriori LES of 2D turbulence with convolutional neural networks: Backscatter analysis and generalization via transfer learning

2021 ◽  
Author(s):  
Yifei Guan ◽  
Ashesh Chattopadhyay ◽  
Adam Subel ◽  
Pedram Hassanzadeh
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Network Architecture ◽  
Turbulence Modeling ◽  
Scale Effects ◽  
A Priori ◽  
A Posteriori ◽  
Subgrid Scale ◽  
Artificial Neural Network Ann ◽  
Short Term Prediction

<p>In large eddy simulations (LES), the subgrid-scale effects are modeled by physics-based or data-driven methods. This work develops a convolutional neural network (CNN) to model the subgrid-scale effects of a two-dimensional turbulent flow. The model is able to capture both the inter-scale forward energy transfer and backscatter in both a priori and a posteriori analyses. The LES-CNN model outperforms the physics-based eddy-viscosity models and the previous proposed local artificial neural network (ANN) models in both short-term prediction and long-term statistics. Transfer learning is implemented to generalize the method for turbulence modeling at higher Reynolds numbers. Encoder-decoder network architecture is proposed to generalize the model to a higher computational grid resolution.</p>

Enhancing Vibration-Based Structural Health Monitoring via Edge Computing: A Tiny Machine Learning Perspective

2021 ◽  
Author(s):  
Federica Zonzini ◽  
Francesca Romano ◽  
Antonio Carbone ◽  
Matteo Zauli ◽  
Luca De Marchi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Structural Health Monitoring ◽  
Health Monitoring ◽  
Network Architecture ◽  
Binary Classification ◽  
Low Cost ◽  
Classification Problems ◽  
Computationally Efficient ◽  
Structural Health

Abstract Despite the outstanding improvements achieved by artificial intelligence in the Structural Health Monitoring (SHM) field, some challenges need to be coped with. Among them, the necessity to reduce the complexity of the models and the data-to-user latency time which are still affecting state-of-the-art solutions. This is due to the continuous forwarding of a huge amount of data to centralized servers, where the inference process is usually executed in a bulky manner. Conversely, the emerging field of Tiny Machine Learning (TinyML), promoted by the recent advancements by the electronic and information engineering community, made sensor-near data inference a tangible, low-cost and computationally efficient alternative. In line with this observation, this work explored the embodiment of the One Class Classifier Neural Network, i.e., a neural network architecture solving binary classification problems for vibration-based SHM scenarios, into a resource-constrained device. To this end, OCCNN has been ported on the Arduino Nano 33 BLE Sense platform and validated with experimental data from the Z24 bridge use case, reaching an average accuracy and precision of 95% and 94%, respectively.

scholarly journals A Meta-Learning Approach for Custom Model Training

2019 ◽  
Vol 33 ◽  
pp. 9937-9938
Author(s):  
Amir Erfan Eshratifar ◽  
Mohammad Saeed Abrishami ◽  
David Eigen ◽  
Massoud Pedram
Keyword(s):  
Transfer Learning ◽  
Large Data ◽  
Learning Approaches ◽  
Target Task ◽  
Training Approach ◽  
Starting Point ◽  
Meta Learning ◽  
Future Task ◽  
Model Training ◽  
Training Examples

Transfer-learning and meta-learning are two effective methods to apply knowledge learned from large data sources to new tasks. In few-class, few-shot target task settings (i.e. when there are only a few classes and training examples available in the target task), meta-learning approaches that optimize for future task learning have outperformed the typical transfer approach of initializing model weights from a pretrained starting point. But as we experimentally show, metalearning algorithms that work well in the few-class setting do not generalize well in many-shot and many-class cases. In this paper, we propose a joint training approach that combines both transfer-learning and meta-learning. Benefiting from the advantages of each, our method obtains improved generalization performance on unseen target tasks in both few- and many-class and few- and many-shot scenarios.

scholarly journals PARROT is a flexible recurrent neural network framework for analysis of large protein datasets

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Daniel Griffith ◽  
Alex S Holehouse
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
High Throughput ◽  
Recurrent Neural Network ◽  
Transcriptional Activation ◽  
Network Architecture ◽  
Learning Approaches ◽  
Large Protein ◽  
Protein Datasets

The rise of high-throughput experiments has transformed how scientists approach biological questions. The ubiquity of large-scale assays that can test thousands of samples in a day has necessitated the development of new computational approaches to interpret this data. Among these tools, machine learning approaches are increasingly being utilized due to their ability to infer complex nonlinear patterns from high-dimensional data. Despite their effectiveness, machine learning (and in particular deep learning) approaches are not always accessible or easy to implement for those with limited computational expertise. Here we present PARROT, a general framework for training and applying deep learning-based predictors on large protein datasets. Using an internal recurrent neural network architecture, PARROT is capable of tackling both classification and regression tasks while only requiring raw protein sequences as input. We showcase the potential uses of PARROT on three diverse machine learning tasks: predicting phosphorylation sites, predicting transcriptional activation function of peptides generated by high-throughput reporter assays, and predicting the fibrillization propensity of amyloid beta with data generated by deep mutational scanning. Through these examples, we demonstrate that PARROT is easy to use, performs comparably to state-of-the-art computational tools, and is applicable for a wide array of biological problems.

scholarly journals Design and implementation of convolutional neural network architectures for low-dose CT image noise reduction

2021 ◽  
Author(s):  
Seyyedomid Badretale
Keyword(s):  
Neural Network ◽  
Image Quality ◽  
Convolutional Neural Network ◽  
Network Architecture ◽  
Low Dose ◽  
Ct Images ◽  
Learning Approaches ◽  
Low Dose Ct ◽  
Image Noise Reduction ◽  
Low Dose Computed Tomography

An essential objective in low-dose Computed Tomography (CT) imaging is how best to preserve the image quality. While the image quality lowers with reducing the X-ray dosage, improving the quality is crucial. Therefore, a novel method to denoise low-dose CT images has been presented in this thesis. Different from the traditional algorithms which utilize similar shared features of CT images in the spatial domain, the deep learning approaches are suggested for low-dose CT denoising. The proposed algorithm learns an end-to-end mapping from the low-dose CT images for denoising the low-dose CT images. The first method is based on a fully convolutional neural network. The second approach is a deep convolutional neural network architecture consisting of five major sections. The results of two frameworks are compared with the state-of-the-art methods. Several metrics for assessing image quality are applied in this thesis in order to highlight the supremacy of the performed method.

Application of convolutional neural networks for monitoring of marine objects

2020 ◽  
pp. 53-61
Author(s):  
Н.А. Полковникова ◽  
Е.В. Тузинкевич ◽  
А.Н. Попов
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Research Process ◽  
Probability Of Detection ◽  
Neural Network Architecture ◽  
Deep Convolutional Neural Networks

В статье рассмотрены технологии компьютерного зрения на основе глубоких свёрточных нейронных сетей. Применение нейронных сетей особенно эффективно для решения трудно формализуемых задач. Разработана архитектура свёрточной нейронной сети применительно к задаче распознавания и классификации морских объектов на изображениях. В ходе исследования выполнен ретроспективный анализ технологий компьютерного зрения и выявлен ряд проблем, связанных с применением нейронных сетей: «исчезающий» градиент, переобучение и вычислительная сложность. При разработке архитектуры нейросети предложено использовать функцию активации RELU, обучение некоторых случайно выбранных нейронов и нормализацию с целью упрощения архитектуры нейросети. Сравнение используемых в нейросети функций активации ReLU, LeakyReLU, Exponential ReLU и SOFTMAX выполнено в среде Matlab R2020a. На основе свёрточной нейронной сети разработана программа на языке программирования Visual C# в среде MS Visual Studio для распознавания морских объектов. Программапредназначена для автоматизированной идентификации морских объектов, производит детектирование (нахождение объектов на изображении) и распознавание объектов с высокой вероятностью обнаружения. The article considers computer vision technologies based on deep convolutional neural networks. Application of neural networks is particularly effective for solving difficult formalized problems. As a result convolutional neural network architecture to the problem of recognition and classification of marine objects on images is implemented. In the research process a retrospective analysis of computer vision technologies was performed and a number of problems associated with the use of neural networks were identified: vanishing gradient, overfitting and computational complexity. To solve these problems in neural network architecture development, it was proposed to use RELU activation function, training some randomly selected neurons and normalization for simplification of neural network architecture. Comparison of ReLU, LeakyReLU, Exponential ReLU, and SOFTMAX activation functions used in the neural network implemented in Matlab R2020a.The computer program based on convolutional neural network for marine objects recognition implemented in Visual C# programming language in MS Visual Studio integrated development environment. The program is designed for automated identification of marine objects, produces detection (i.e., presence of objects on image), and objects recognition with high probability of detection.

Sign in / Sign up

Export Citation Format

Share Document