scholarly journals When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on a Multitask Learning Framework

2020 ◽  
Vol 12 (20) ◽  
pp. 3276 ◽  
Author(s):  
Zhicheng Zhao ◽  
Ze Luo ◽  
Jian Li ◽  
Can Chen ◽  
Yingchao Piao
Keyword(s):  
Remote Sensing ◽  
Supervised Learning ◽  
Remote Sensing Image ◽  
Multitask Learning ◽  
Natural Image ◽  
Scene Classification ◽  
Remote Sensing Images ◽  
Learning Framework ◽  
Continuous Progress ◽  
Image Datasets

In recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of remote sensing image datasets is generally small. In addition, many problems related to small objects and complex backgrounds arise in remote sensing image scenes, presenting great challenges for CNN-based recognition methods. In this article, to improve the feature extraction ability and generalization ability of such models and to enable better use of the information contained in the original remote sensing images, we introduce a multitask learning framework which combines the tasks of self-supervised learning and scene classification. Unlike previous multitask methods, we adopt a new mixup loss strategy to combine the two tasks with dynamic weight. The proposed multitask learning framework empowers a deep neural network to learn more discriminative features without increasing the amounts of parameters. Comprehensive experiments were conducted on four representative remote sensing scene classification datasets. We achieved state-of-the-art performance, with average accuracies of 94.21%, 96.89%, 99.11%, and 98.98% on the NWPU, AID, UC Merced, and WHU-RS19 datasets, respectively. The experimental results and visualizations show that our proposed method can learn more discriminative features and simultaneously encode orientation information while effectively improving the accuracy of remote sensing scene classification.

scholarly journals Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning Analysis

2019 ◽  
Vol 12 (1) ◽  
pp. 86 ◽  
Author(s):  
Rafael Pires de Lima ◽  
Kurt Marfurt
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Network Models ◽  
Remote Sensing Image ◽  
Natural Images ◽  
Natural Image ◽  
Scene Classification ◽  
Neural Network Models

Remote-sensing image scene classification can provide significant value, ranging from forest fire monitoring to land-use and land-cover classification. Beginning with the first aerial photographs of the early 20th century to the satellite imagery of today, the amount of remote-sensing data has increased geometrically with a higher resolution. The need to analyze these modern digital data motivated research to accelerate remote-sensing image classification. Fortunately, great advances have been made by the computer vision community to classify natural images or photographs taken with an ordinary camera. Natural image datasets can range up to millions of samples and are, therefore, amenable to deep-learning techniques. Many fields of science, remote sensing included, were able to exploit the success of natural image classification by convolutional neural network models using a technique commonly called transfer learning. We provide a systematic review of transfer learning application for scene classification using different datasets and different deep-learning models. We evaluate how the specialization of convolutional neural network models affects the transfer learning process by splitting original models in different points. As expected, we find the choice of hyperparameters used to train the model has a significant influence on the final performance of the models. Curiously, we find transfer learning from models trained on larger, more generic natural images datasets outperformed transfer learning from models trained directly on smaller remotely sensed datasets. Nonetheless, results show that transfer learning provides a powerful tool for remote-sensing scene classification.

scholarly journals Semi-Supervised Remote Sensing Image Semantic Segmentation via Consistency Regularization and Average Update of Pseudo-Label

2020 ◽  
Vol 12 (21) ◽  
pp. 3603 ◽  
Author(s):  
Jiaxin Wang ◽  
Chris H. Q. Ding ◽  
Sibao Chen ◽  
Chenggang He ◽  
Bin Luo
Keyword(s):  
Remote Sensing ◽  
Image Segmentation ◽  
Supervised Learning ◽  
Semantic Segmentation ◽  
Remote Sensing Image ◽  
Unlabeled Data ◽  
Training Method ◽  
Remote Sensing Images ◽  
Supervised Training ◽  
Great Progress

Image segmentation has made great progress in recent years, but the annotation required for image segmentation is usually expensive, especially for remote sensing images. To solve this problem, we explore semi-supervised learning methods and appropriately utilize a large amount of unlabeled data to improve the performance of remote sensing image segmentation. This paper proposes a method for remote sensing image segmentation based on semi-supervised learning. We first design a Consistency Regularization (CR) training method for semi-supervised training, then employ the new learned model for Average Update of Pseudo-label (AUP), and finally combine pseudo labels and strong labels to train semantic segmentation network. We demonstrate the effectiveness of the proposed method on three remote sensing datasets, achieving better performance without more labeled data. Extensive experiments show that our semi-supervised method can learn the latent information from the unlabeled data to improve the segmentation performance.

scholarly journals Hierarchical Multi-View Semi-Supervised Learning for Very High-Resolution Remote Sensing Image Classification

2020 ◽  
Vol 12 (6) ◽  
pp. 1012 ◽  
Author(s):  
Cheng Shi ◽  
Zhiyong Lv ◽  
Xiuhong Yang ◽  
Pengfei Xu ◽  
Irfana Bibi
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Image Classification ◽  
Supervised Learning ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Remote Sensing Images ◽  
Training Set ◽  
Remote Sensing Image Classification ◽  
Very High

Traditional classification methods used for very high-resolution (VHR) remote sensing images require a large number of labeled samples to obtain higher classification accuracy. Labeled samples are difficult to obtain and costly. Therefore, semi-supervised learning becomes an effective paradigm that combines the labeled and unlabeled samples for classification. In semi-supervised learning, the key issue is to enlarge the training set by selecting highly-reliable unlabeled samples. Observing the samples from multiple views is helpful to improving the accuracy of label prediction for unlabeled samples. Hence, the reasonable view partition is very important for improving the classification performance. In this paper, a hierarchical multi-view semi-supervised learning framework with CNNs (HMVSSL) is proposed for VHR remote sensing image classification. Firstly, a superpixel-based sample enlargement method is proposed to increase the number of training samples in each view. Secondly, a view partition method is designed to partition the training set into two independent views, and the partitioned subsets are characterized by being inter-distinctive and intra-compact. Finally, a collaborative classification strategy is proposed for the final classification. Experiments are conducted on three VHR remote sensing images, and the results show that the proposed method performs better than several state-of-the-art methods.

scholarly journals RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

2020 ◽  
Vol 13 (1) ◽  
pp. 62
Author(s):  
Linshu Hu ◽  
Mengjiao Qin ◽  
Feng Zhang ◽  
Zhenhong Du ◽  
Renyi Liu
Keyword(s):  
Remote Sensing ◽  
Image Enhancement ◽  
Similarity Index ◽  
Super Resolution ◽  
Remote Sensing Image ◽  
Fine Tuning ◽  
Natural Image ◽  
Great Success ◽  
Remote Sensing Images ◽  
Low Light

Image enhancement (IE) technology can help enhance the brightness of remote-sensing images to obtain better interpretation and visualization effects. Convolutional neural networks (CNN), such as the Low-light CNN (LLCNN) and Super-resolution CNN (SRCNN), have achieved great success in image enhancement, image super resolution, and other image-processing applications. Therefore, we adopt CNN to propose a new neural network architecture with end-to-end strategy for low-light remote-sensing IE, named remote-sensing CNN (RSCNN). In RSCNN, an upsampling operator is adopted to help learn more multi-scaled features. With respect to the lack of labeled training data in remote-sensing image datasets for IE, we use real natural image patches to train firstly and then perform fine-tuning operations with simulated remote-sensing image pairs. Reasonably designed experiments are carried out, and the results quantitatively show the superiority of RSCNN in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) over conventional techniques for low-light remote-sensing IE. Furthermore, the results of our method have obvious qualitative advantages in denoising and maintaining the authenticity of colors and textures.

scholarly journals Prototype Calibration with Feature Generation for Few-Shot Remote Sensing Image Scene Classification

2021 ◽  
Vol 13 (14) ◽  
pp. 2728
Author(s):  
Qingjie Zeng ◽  
Jie Geng ◽  
Kai Huang ◽  
Wen Jiang ◽  
Jun Guo
Keyword(s):  
Remote Sensing ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Scene Classification ◽  
Remote Sensing Images ◽  
Training Set ◽  
Shot Classification ◽  
Support Set ◽  
Feature Expression

Few-shot classification of remote sensing images has attracted attention due to its important applications in various fields. The major challenge in few-shot remote sensing image scene classification is that limited labeled samples can be utilized for training. This may lead to the deviation of prototype feature expression, and thus the classification performance will be impacted. To solve these issues, a prototype calibration with a feature-generating model is proposed for few-shot remote sensing image scene classification. In the proposed framework, a feature encoder with self-attention is developed to reduce the influence of irrelevant information. Then, the feature-generating module is utilized to expand the support set of the testing set based on prototypes of the training set, and prototype calibration is proposed to optimize features of support images that can enhance the representativeness of each category features. Experiments on NWPU-RESISC45 and WHU-RS19 datasets demonstrate that the proposed method can yield superior classification accuracies for few-shot remote sensing image scene classification.

scholarly journals A NOVEL FRAMEWORK FOR REMOTE SENSING IMAGE SCENE CLASSIFICATION

2018 ◽  
Vol XLII-3 ◽  
pp. 657-663 ◽  
Author(s):  
S. Jiang ◽  
H. Zhao ◽  
W. Wu ◽  
Q. Tan
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Semantic Category ◽  
Distribution Patterns ◽  
Remote Sensing Image ◽  
Scene Classification ◽  
Remote Sensing Images ◽  
Training Stage ◽  
Feature Extractor ◽  
High Level

High resolution remote sensing (HRRS) images scene classification aims to label an image with a specific semantic category. HRRS images contain more details of the ground objects and their spatial distribution patterns than low spatial resolution images. Scene classification can bridge the gap between low-level features and high-level semantics. It can be applied in urban planning, target detection and other fields. This paper proposes a novel framework for HRRS images scene classification. This framework combines the convolutional neural network (CNN) and XGBoost, which utilizes CNN as feature extractor and XGBoost as a classifier. Then, this framework is evaluated on two different HRRS images datasets: UC-Merced dataset and NWPU-RESISC45 dataset. Our framework achieved satisfying accuracies on two datasets, which is 95.57 % and 83.35 % respectively. From the experiments result, our framework has been proven to be effective for remote sensing images classification. Furthermore, we believe this framework will be more practical for further HRRS scene classification, since it costs less time on training stage.

scholarly journals Early Labeled and Small Loss Selection Semi-Supervised Learning Method for Remote Sensing Image Scene Classification

2021 ◽  
Vol 13 (20) ◽  
pp. 4039
Author(s):  
Ye Tian ◽  
Yuxin Dong ◽  
Guisheng Yin
Keyword(s):  
Remote Sensing ◽  
Supervised Learning ◽  
Early Stage ◽  
Remote Sensing Image ◽  
Learning Method ◽  
Scene Classification ◽  
Small Loss ◽  
Basic Work ◽  
Spatial Differences ◽  
Remote Sensing Image Processing

The classification of aerial scenes has been extensively studied as the basic work of remote sensing image processing and interpretation. However, the performance of remote sensing image scene classification based on deep neural networks is limited by the number of labeled samples. In order to alleviate the demand for massive labeled samples, various methods have been proposed to apply semi-supervised learning to train the classifier using labeled and unlabeled samples. However, considering the complex contextual relationship and huge spatial differences, the existing semi-supervised learning methods bring different degrees of incorrectly labeled samples when pseudo-labeling unlabeled data. In particular, when the number of labeled samples is small, it affects the generalization performance of the model. In this article, we propose a novel semi-supervised learning method with early labeled and small loss selection. First, the model learns the characteristics of simple samples in the early stage and uses multiple early models to screen out a small number of unlabeled samples for pseudo-labeling based on this characteristic. Then, the model is trained in a semi-supervised manner by combining labeled samples, pseudo-labeled samples, and unlabeled samples. In the training process of the model, small loss selection is used to further eliminate some of the noisy labeled samples to improve the recognition accuracy of the model. Finally, in order to verify the effectiveness of the proposed method, it is compared with several state-of-the-art semi-supervised classification methods. The results show that when there are only a few labeled samples in remote sensing image scene classification, our method is always better than previous methods.

scholarly journals Distribution Consistency Loss for Large-Scale Remote Sensing Image Retrieval

2020 ◽  
Vol 12 (1) ◽  
pp. 175 ◽  
Author(s):  
Lili Fan ◽  
Hongwei Zhao ◽  
Haoyu Zhao
Keyword(s):  
Remote Sensing ◽  
Image Retrieval ◽  
Large Scale ◽  
Metric Learning ◽  
Remote Sensing Image ◽  
Positive Sample ◽  
Fine Tuning ◽  
Remote Sensing Images ◽  
Hard Samples ◽  
Image Datasets

Remote sensing images are featured by massiveness, diversity and complexity. These features put forward higher requirements for the speed and accuracy of remote sensing image retrieval. The extraction method plays a key role in retrieving remote sensing images. Deep metric learning (DML) captures the semantic similarity information between data points by learning embedding in vector space. However, due to the uneven distribution of sample data in remote sensing image datasets, the pair-based loss currently used in DML is not suitable. To improve this, we propose a novel distribution consistency loss to solve this problem. First, we define a new way to mine samples by selecting five in-class hard samples and five inter-class hard samples to form an informative set. This method can make the network extract more useful information in a short time. Secondly, in order to avoid inaccurate feature extraction due to sample imbalance, we assign dynamic weight to the positive samples according to the ratio of the number of hard samples and easy samples in the class, and name the loss caused by the positive sample as the sample balance loss. We combine the sample balance of the positive samples with the ranking consistency of the negative samples to form our distribution consistency loss. Finally, we built an end-to-end fine-tuning network suitable for remote sensing image retrieval. We display comprehensive experimental results drawing on three remote sensing image datasets that are publicly available and show that our method achieves the state-of-the-art performance.

scholarly journals A Survey on Deep Learning-Driven Remote Sensing Image Scene Understanding: Scene Classification, Scene Retrieval and Scene-Guided Object Detection

2019 ◽  
Vol 9 (10) ◽  
pp. 2110 ◽  
Author(s):  
Yating Gu ◽  
Yantian Wang ◽  
Yansheng Li
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Object Detection ◽  
Scene Understanding ◽  
Remote Sensing Image ◽  
Future Research ◽  
Natural Image ◽  
Scene Classification ◽  
Potential Applications ◽  
Future Research Directions

As a fundamental and important task in remote sensing, remote sensing image scene understanding (RSISU) has attracted tremendous research interest in recent years. RSISU includes the following sub-tasks: remote sensing image scene classification, remote sensing image scene retrieval, and scene-driven remote sensing image object detection. Although these sub-tasks have different goals, they share some communal hints. Hence, this paper tries to discuss them as a whole. Similar to other domains (e.g., speech recognition and natural image recognition), deep learning has also become the state-of-the-art technique in RSISU. To facilitate the sustainable progress of RSISU, this paper presents a comprehensive review of deep-learning-based RSISU methods, and points out some future research directions and potential applications of RSISU.

Sign in / Sign up

Export Citation Format

Share Document