scholarly journals Defect Texts Mining of Secondary Device in Smart Substation with GloVe and Attention-Based Bidirectional LSTM

Energies ◽  
2020 ◽  
Vol 13 (17) ◽  
pp. 4522
Author(s):  
Kai Chen ◽  
Rabea Jamil Mahfoud ◽  
Yonghui Sun ◽  
Dongliang Nan ◽  
Kaike Wang ◽  
...  
Keyword(s):  
Text Classification ◽  
Short Term Memory ◽  
Feature Space ◽  
Classification Model ◽  
Attention Model ◽  
Operation And Maintenance ◽  
Word Representation ◽  
Low Efficiency ◽  
Smart Substation ◽  
Secondary Device

In the process of the operation and maintenance of secondary devices in smart substation, a wealth of defect texts containing the state information of the equipment is generated. Aiming to overcome the low efficiency and low accuracy problems of artificial power text classification and mining, combined with the characteristics of power equipment defect texts, a defect texts mining method for a secondary device in a smart substation is proposed, which integrates global vectors for word representation (GloVe) method and attention-based bidirectional long short-term memory (BiLSTM-Attention) method in one model. First, the characteristics of the defect texts are analyzed and preprocessed to improve the quality of the defect texts. Then, defect texts are segmented into words, and the words are mapped to the high-dimensional feature space based on the global vectors for word representation (GloVe) model to form distributed word vectors. Finally, a text classification model based on BiLSTM-Attention was proposed to classify the defect texts of a secondary device. Precision, Recall and F1-score are selected as evaluation indicators, and compared with traditional machine learning and deep learning models. The analysis of a case study shows that the BiLSTM-Attention model has better performance and can achieve the intelligent, accurate and efficient classification of secondary device defect texts. It can assist the operation and maintenance personnel to make scientific maintenance decisions on a secondary device and improve the level of intelligent management of equipment.

scholarly journals Chinese Text Classification Model Based on Deep Learning

2018 ◽  
Vol 10 (11) ◽  
pp. 113 ◽  
Author(s):  
Yue Li ◽  
Xutao Wang ◽  
Pengjian Xu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Language Processing ◽  
Chinese Text ◽  
Text Classification ◽  
Short Term Memory ◽  
Classification Model ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers’ models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts.

scholarly journals Reducing the Deterioration of Sentiment Analysis Results Due to the Time Impact

Information ◽  
2018 ◽  
Vol 9 (8) ◽  
pp. 184 ◽  
Author(s):  
Yuliya Rubtsova
Keyword(s):  
Computational Complexity ◽  
Text Classification ◽  
Feature Space ◽  
Sentiment Classification ◽  
Text Collections ◽  
Word Representation ◽  
F Measure ◽  
Over Time

The research identifies and substantiates the problem of quality deterioration in the sentiment classification of text collections identical in composition and characteristics, but staggered over time. It is shown that the quality of sentiment classification can drop up to 15% in terms of the F-measure over a year and a half. This paper presents three different approaches to improving text classification by sentiment in continuously-updated text collections in Russian: using a weighing scheme with linear computational complexity, adding lexicons of emotional vocabulary to the feature space and distributed word representation. All methods are compared, and it is shown which method is most applicable in certain cases. Experiments comparing the methods on sufficiently representative text collections are described. It is shown that suggested approaches could reduce the deterioration of sentiment classification results for collections staggered over time.

scholarly journals A Sequential Graph Neural Network for Short Text Classification

Algorithms ◽  
2021 ◽  
Vol 14 (12) ◽  
pp. 352
Author(s):  
Ke Zhao ◽  
Lan Huang ◽  
Rui Song ◽  
Qiang Shen ◽  
Hao Xu
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Short Term Memory ◽  
Contextual Information ◽  
Extended Model ◽  
Short Text ◽  
Convolutional Networks ◽  
Word Representation ◽  
Benchmark Datasets ◽  
Sequential Information

Short text classification is an important problem of natural language processing (NLP), and graph neural networks (GNNs) have been successfully used to solve different NLP problems. However, few studies employ GNN for short text classification, and most of the existing graph-based models ignore sequential information (e.g., word orders) in each document. In this work, we propose an improved sequence-based feature propagation scheme, which fully uses word representation and document-level word interaction and overcomes the limitations of textual features in short texts. On this basis, we utilize this propagation scheme to construct a lightweight model, sequential GNN (SGNN), and its extended model, ESGNN. Specifically, we build individual graphs for each document in the short text corpus based on word co-occurrence and use a bidirectional long short-term memory network (Bi-LSTM) to extract the sequential features of each document; therefore, word nodes in the document graph retain contextual information. Furthermore, two different simplified graph convolutional networks (GCNs) are used to learn word representations based on their local structures. Finally, word nodes combined with sequential information and local information are incorporated as the document representation. Extensive experiments on seven benchmark datasets demonstrate the effectiveness of our method.

scholarly journals Convolutional neural network-based model for web-based text classification

2019 ◽  
Vol 9 (6) ◽  
pp. 5785
Author(s):  
Satyabrata Aich ◽  
Sabyasachi Chakraborty ◽  
Hee-Cheol Kim
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Text Classification ◽  
Short Term Memory ◽  
Classification Model ◽  
Learning Models ◽  
Real World Problem ◽  
Traditional Approaches ◽  
The Web

<table width="593" border="1" cellspacing="0" cellpadding="0"><tbody><tr><td valign="top" width="387"><p>There is an increasing amount of text data available on the web with multiple topical granularities; this necessitates proper categorization/classification of text to facilitate obtaining useful information as per the needs of users. Some traditional approaches such as bag-of-words and bag-of-ngrams models provide good results for text classification. However, texts available on the web in the current state contain high event-related granularity on different topics at different levels, which may adversely affect the accuracy of traditional approaches. With the invention of deep learning models, which already have the capability of providing good accuracy in the field of image processing and speech recognition, the problems inherent in the traditional text classification model can be overcome. Currently, there are several deep learning models such as a convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long-short term memory that are widely used for various text-related tasks; however, among them, the CNN model is popular because it is simple to use and has high accuracy for text classification. In this study, classification of random texts on the web into categories is attempted using a CNN-based model by changing the hyperparameters and sequence of text vectors. We attempt to tune every hyperparameter that is unique for the classification task along with the sequences of word vectors to obtain the desired accuracy; the accuracy is found to be in the range of 85–92%. This model can be considered as a reliable model and applied to solve real-world problem or extract useful information for various text mining applications.</p></td></tr></tbody></table>

scholarly journals Extraction of Arecanut Planting Distribution Based on the Feature Space Optimization of PlanetScope Imagery

Agriculture ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. 371
Author(s):  
Yu Jin ◽  
Jiawei Guo ◽  
Huichun Ye ◽  
Jinling Zhao ◽  
Wenjiang Huang ◽  
...  
Keyword(s):  
Random Forest ◽  
Satellite Imagery ◽  
Feature Space ◽  
Kappa Coefficient ◽  
Classification Model ◽  
Support Vector ◽  
Textural Feature ◽  
Monitoring Accuracy ◽  
Areca Catechu ◽  
High Level

The remote sensing extraction of large areas of arecanut (Areca catechu L.) planting plays an important role in investigating the distribution of arecanut planting area and the subsequent adjustment and optimization of regional planting structures. Satellite imagery has previously been used to investigate and monitor the agricultural and forestry vegetation in Hainan. However, the monitoring accuracy is affected by the cloudy and rainy climate of this region, as well as the high level of land fragmentation. In this paper, we used PlanetScope imagery at a 3 m spatial resolution over the Hainan arecanut planting area to investigate the high-precision extraction of the arecanut planting distribution based on feature space optimization. First, spectral and textural feature variables were selected to form the initial feature space, followed by the implementation of the random forest algorithm to optimize the feature space. Arecanut planting area extraction models based on the support vector machine (SVM), BP neural network (BPNN), and random forest (RF) classification algorithms were then constructed. The overall classification accuracies of the SVM, BPNN, and RF models optimized by the RF features were determined as 74.82%, 83.67%, and 88.30%, with Kappa coefficients of 0.680, 0.795, and 0.853, respectively. The RF model with optimized features exhibited the highest overall classification accuracy and kappa coefficient. The overall accuracy of the SVM, BPNN, and RF models following feature optimization was improved by 3.90%, 7.77%, and 7.45%, respectively, compared with the corresponding unoptimized classification model. The kappa coefficient also improved. The results demonstrate the ability of PlanetScope satellite imagery to extract the planting distribution of arecanut. Furthermore, the RF is proven to effectively optimize the initial feature space, composed of spectral and textural feature variables, further improving the extraction accuracy of the arecanut planting distribution. This work can act as a theoretical and technical reference for the agricultural and forestry industries.

scholarly journals Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1579 ◽  
Author(s):  
Kyoung Ju Noh ◽  
Chi Yoon Jeong ◽  
Jiyoun Lim ◽  
Seungeun Chung ◽  
Gague Kim ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Short Term Memory ◽  
Domain Adaptation ◽  
Classification Model ◽  
Speech Emotion Recognition ◽  
Target Domain ◽  
Model Generalization ◽  
Speech Database ◽  
Emotion Labels ◽  
Temporal Feature

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.

Additive DenseNet: Dense connections based on simple addition operations

2020 ◽  
pp. 1-11
Author(s):  
Dawei Yu ◽  
Jie Yang ◽  
Yun Zhang ◽  
Shujuan Yu
Keyword(s):  
Text Classification ◽  
Deep Neural Networks ◽  
Model Complexity ◽  
Residual Network ◽  
Large Margin ◽  
Simple Addition ◽  
High Resource ◽  
Model Training ◽  
Low Efficiency ◽  
Resource Economy

The Densely Connected Network (DenseNet) has been widely recognized as a highly competitive architecture in Deep Neural Networks. And its most outstanding property is called Dense Connections, which represent each layer’s input by concatenating all the preceding layers’ outputs and thus improve the performance by encouraging feature reuse to the extreme. However, it is Dense Connections that cause the challenge of dimension-enlarging, making DenseNet very resource-intensive and low efficiency. In the light of this, inspired by the Residual Network (ResNet), we propose an improved DenseNet named Additive DenseNet, which features replacing concatenation operations (used in Dense Connections) with addition operations (used in ResNet), and in terms of feature reuse, it upgrades addition operations to accumulating operations (namely ∑ (·)), thus enables each layer’s input to be the summation of all the preceding layers’ outputs. Consequently, Additive DenseNet can not only preserve the dimension of input from enlarging, but also retain the effect of Dense Connections. In this paper, Additive DenseNet is applied to text classification task. The experimental results reveal that compared to DenseNet, our Additive DenseNet can reduce the model complexity by a large margin, such as GPU memory usage and quantity of parameters. And despite its high resource economy, Additive DenseNet can still outperform DenseNet on 6 text classification datasets in terms of accuracy and show competitive performance for model training.

scholarly journals Prediction of Head Movement in 360-Degree Videos Using Attention Model

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3678
Author(s):  
Dongwon Lee ◽  
Minji Choi ◽  
Joohyun Lee
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
The Other ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Attention Model ◽  
Long Short Term Memory ◽  
Machine Learning Models

In this paper, we propose a prediction algorithm, the combination of Long Short-Term Memory (LSTM) and attention model, based on machine learning models to predict the vision coordinates when watching 360-degree videos in a Virtual Reality (VR) or Augmented Reality (AR) system. Predicting the vision coordinates while video streaming is important when the network condition is degraded. However, the traditional prediction models such as Moving Average (MA) and Autoregression Moving Average (ARMA) are linear so they cannot consider the nonlinear relationship. Therefore, machine learning models based on deep learning are recently used for nonlinear predictions. We use the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural network methods, originated in Recurrent Neural Networks (RNN), and predict the head position in the 360-degree videos. Therefore, we adopt the attention model to LSTM to make more accurate results. We also compare the performance of the proposed model with the other machine learning models such as Multi-Layer Perceptron (MLP) and RNN using the root mean squared error (RMSE) of predicted and real coordinates. We demonstrate that our model can predict the vision coordinates more accurately than the other models in various videos.

Sign in / Sign up

Export Citation Format

Share Document