scholarly journals Emotion-Semantic-Enhanced Bidirectional LSTM with Multi-Head Attention Mechanism for Microblog Sentiment Analysis

Information ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 280
Author(s):  
Shaoxiu Wang ◽  
Yonghua Zhu ◽  
Wenjing Gao ◽  
Meng Cao ◽  
Mengyao Li
Keyword(s):  
Sentiment Analysis ◽  
Short Term Memory ◽  
Syntactic Structure ◽  
Contextual Information ◽  
Research Field ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Mechanism Model ◽  
Hidden Layer ◽  
The Impact

The sentiment analysis of microblog text has always been a challenging research field due to the limited and complex contextual information. However, most of the existing sentiment analysis methods for microblogs focus on classifying the polarity of emotional keywords while ignoring the transition or progressive impact of words in different positions in the Chinese syntactic structure on global sentiment, as well as the utilization of emojis. To this end, we propose the emotion-semantic-enhanced bidirectional long short-term memory (BiLSTM) network with the multi-head attention mechanism model (EBILSTM-MH) for sentiment analysis. This model uses BiLSTM to learn feature representation of input texts, given the word embedding. Subsequently, the attention mechanism is used to assign the attentive weights of each words to the sentiment analysis based on the impact of emojis. The attentive weights can be combined with the output of the hidden layer to obtain the feature representation of posts. Finally, the sentiment polarity of microblog can be obtained through the dense connection layer. The experimental results show the feasibility of our proposed model on microblog sentiment analysis when compared with other baseline models.

scholarly journals Hybrid LSTM Self-Attention Mechanism Model for Forecasting the Reform of Scientific Research in Morocco

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Asmaa Fahim ◽  
Qingmei Tan ◽  
Mouna Mazzi ◽  
Md Sahabuddin ◽  
Bushra Naz ◽  
...  
Keyword(s):  
Time Series ◽  
Short Term Memory ◽  
Research Area ◽  
Vital Role ◽  
Attention Mechanism ◽  
Percentage Increase ◽  
Mechanism Model ◽  
Research Outcomes ◽  
Future Outcomes ◽  
The Impact

Education is the cultivation of people to promote and guarantee the development of society. Education reforms can play a vital role in the development of a country. However, it is crucial to continually monitor the educational model’s performance by forecasting the outcome’s progress. Machine learning-based models are currently a hot topic in improving the forecasting research area. Forecasting models can help to analyse the impact of future outcomes by showing yearly trends. For this study, we developed a hybrid, forecasting time-series model by long short-term memory (LSTM) network and self-attention mechanism (SAM) to monitor Morocco’s educational reform. We analysed six universities’ performance and provided a prediction model to evaluate the best-performing university’s performance after implementing the latest reform, i.e., from 2015–2030. We forecasted the six universities’ research outcomes and tested our proposed methodology’s accuracy against other time-series models. Results show that our model performs better for predicting research outcomes. The percentage increase in university performance after nine years is discussed to help predict the best-performing university. Our proposed algorithm accuracy and performance are better than other algorithms like LSTM and RNN.

Adaptive particle swarm optimization algorithm based long short-term memory networks for sentiment analysis

2021 ◽  
pp. 1-17
Author(s):  
J. Shobana ◽  
M. Murali
Keyword(s):  
Particle Swarm Optimization ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Contextual Information ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Swarm Optimization ◽  
Adaptive Particle Swarm Optimization ◽  
Proposed Model

Text Sentiment analysis is the process of predicting whether a segment of text has opinionated or objective content and analyzing the polarity of the text’s sentiment. Understanding the needs and behavior of the target customer plays a vital role in the success of the business so the sentiment analysis process would help the marketer to improve the quality of the product as well as a shopper to buy the correct product. Due to its automatic learning capability, deep learning is the current research interest in Natural language processing. Skip-gram architecture is used in the proposed model for better extraction of the semantic relationships as well as contextual information of words. However, the main contribution of this work is Adaptive Particle Swarm Optimization (APSO) algorithm based LSTM for sentiment analysis. LSTM is used in the proposed model for understanding complex patterns in textual data. To improve the performance of the LSTM, weight parameters are enhanced by presenting the Adaptive PSO algorithm. Opposition based learning (OBL) method combined with PSO algorithm becomes the Adaptive Particle Swarm Optimization (APSO) classifier which assists LSTM in selecting optimal weight for the environment in less number of iterations. So APSO - LSTM ‘s ability in adjusting the attributes such as optimal weights and learning rates combined with the good hyper parameter choices leads to improved accuracy and reduces losses. Extensive experiments were conducted on four datasets proved that our proposed APSO-LSTM model secured higher accuracy over the classical methods such as traditional LSTM, ANN, and SVM. According to simulation results, the proposed model is outperforming other existing models.

scholarly journals Exploring Impact of Age and Gender on Sentiment Analysis Using Machine Learning

Electronics ◽  
2020 ◽  
Vol 9 (2) ◽  
pp. 374 ◽  
Author(s):  
Sudhanshu Kumar ◽  
Monika Gahalawat ◽  
Partha Pratim Roy ◽  
Debi Prosad Dogra ◽  
Byung-Gyu Kim
Keyword(s):  
Machine Learning ◽  
Sentiment Analysis ◽  
Short Term Memory ◽  
Age Groups ◽  
Modern World ◽  
Support Vector ◽  
Digital Information ◽  
Age And Gender ◽  
And Gender ◽  
The Impact

Sentiment analysis is a rapidly growing field of research due to the explosive growth in digital information. In the modern world of artificial intelligence, sentiment analysis is one of the essential tools to extract emotion information from massive data. Sentiment analysis is applied to a variety of user data from customer reviews to social network posts. To the best of our knowledge, there is less work on sentiment analysis based on the categorization of users by demographics. Demographics play an important role in deciding the marketing strategies for different products. In this study, we explore the impact of age and gender in sentiment analysis, as this can help e-commerce retailers to market their products based on specific demographics. The dataset is created by collecting reviews on books from Facebook users by asking them to answer a questionnaire containing questions about their preferences in books, along with their age groups and gender information. Next, the paper analyzes the segmented data for sentiments based on each age group and gender. Finally, sentiment analysis is done using different Machine Learning (ML) approaches including maximum entropy, support vector machine, convolutional neural network, and long short term memory to study the impact of age and gender on user reviews. Experiments have been conducted to identify new insights into the effect of age and gender for sentiment analysis.

Predicting Outcomes of Business Process Executions Based on LSTM Neural Networks and Attention Mechanism

2021 ◽  
Author(s):  
Jiaojiao Wang ◽  
Dongjin Yu ◽  
Chengfei Liu ◽  
Xiaoxiao Sun
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Sequential Data ◽  
Time Prediction ◽  
Prediction Time ◽  
Highly Sensitive ◽  
Long Short Term Memory ◽  
Early Decision ◽  
Lstm Network ◽  
Hidden Layer

Abstract To effectively predict the outcome of an on-going process instance helps make an early decision, which plays an important role in so-called predictive process monitoring. Existing methods in this field are tailor-made for some empirical operations such as the prefix extraction, clustering, and encoding, leading that their relative accuracy is highly sensitive to the dataset. Moreover, they have limitations in real-time prediction applications due to the lengthy prediction time. Since Long Short-term Memory (LSTM) neural network provides a high precision in the prediction of sequential data in several areas, this paper investigates LSTM and its enhancements and proposes three different approaches to build more effective and efficient models for outcome prediction. The first move on enhancement is that we combine the original LSTM network from two directions, forward and backward, to capture more features from the completed cases. The second move on enhancement is that we add attention mechanism after extracting features in the hidden layer of LSTM network to distinct them from their attention weight. A series of extensive experiments are evaluated on twelve real datasets when comparing with other approaches. The results show that our approaches outperform the state-of-the-art ones in terms of prediction effectiveness and time performance.

scholarly journals Combined Self-Attention Mechanism for Chinese Named Entity Recognition in Military

2019 ◽  
Vol 11 (8) ◽  
pp. 180
Author(s):  
Fei Liao ◽  
Liangli Ma ◽  
Jingjing Pei ◽  
Linshan Tan
Keyword(s):  
Domain Knowledge ◽  
Short Term Memory ◽  
Contextual Information ◽  
Named Entity Recognition ◽  
Attention Mechanism ◽  
The Self ◽  
Entity Recognition ◽  
Named Entity ◽  
Vector Representations ◽  
The Military

Military named entity recognition (MNER) is one of the key technologies in military information extraction. Traditional methods for the MNER task rely on cumbersome feature engineering and specialized domain knowledge. In order to solve this problem, we propose a method employing a bidirectional long short-term memory (BiLSTM) neural network with a self-attention mechanism to identify the military entities automatically. We obtain distributed vector representations of the military corpus by unsupervised learning and the BiLSTM model combined with the self-attention mechanism is adopted to capture contextual information fully carried by the character vector sequence. The experimental results show that the self-attention mechanism can improve effectively the performance of MNER task. The F-score of the military documents and network military texts identification was 90.15% and 89.34%, respectively, which was better than other models.

scholarly journals Research on Short-Term Load Prediction Based on Seq2seq Model

Energies ◽  
2019 ◽  
Vol 12 (16) ◽  
pp. 3199 ◽  
Author(s):  
Gangjun Gong ◽  
Xiaonan An ◽  
Nawaraj Kumar Mahato ◽  
Shuyan Sun ◽  
Si Chen ◽  
...  
Keyword(s):  
Power System ◽  
Prediction Model ◽  
Short Term Memory ◽  
Predictive Performance ◽  
Weather Conditions ◽  
Attention Mechanism ◽  
Load Prediction ◽  
Short Term ◽  
System Load ◽  
The Impact

Electricity load prediction is the primary basis on which power-related departments to make logical and effective generation plans and scientific scheduling plans for the most effective power utilization. The perpetual evolution of deep learning has recommended advanced and innovative concepts for short-term load prediction. Taking into consideration the time and nonlinear characteristics of power system load data and further considering the impact of historical and future information on the current state, this paper proposes a Seq2seq short-term load prediction model based on a long short-term memory network (LSTM). Firstly, the periodic fluctuation characteristics of users’ load data are analyzed, establishing a correlation of the load data so as to determine the model’s order in the time series. Secondly, the specifications of the Seq2seq model are given preference and a coalescence of the Residual mechanism (Residual) and the two Attention mechanisms (Attention) is developed. Then, comparing the predictive performance of the model under different types of Attention mechanism, this paper finally adopts the Seq2seq short-term load prediction model of Residual LSTM and the Bahdanau Attention mechanism. Eventually, the prediction model obtains better results when merging the actual power system load data of a certain place. In order to validate the developed model, the Seq2seq was compared with recurrent neural network (RNN), LSTM, and gated recurrent unit (GRU) algorithms. Last but not least, the performance indices were calculated. when training and testing the model with power system load data, it was noted that the root mean square error (RMSE) of Seq2seq was decreased by 6.61%, 16.95%, and 7.80% compared with RNN, LSTM, and GRU, respectively. In addition, a supplementary case study was carried out using data for a small power system considering different weather conditions and user behaviors in order to confirm the applicability and stability of the proposed model. The Seq2seq model for short-term load prediction can be reported to demonstrate superiority in all areas, exhibiting better prediction and stable performance.

scholarly journals A Hybrid GCN and LSTM Structure Based on Attention Mechanism for Crime Prediction

CONVERTER ◽  
2021 ◽  
pp. 328-338
Author(s):  
Jinming Hu
Keyword(s):  
Short Term Memory ◽  
The United States ◽  
Typical Feature ◽  
Research Field ◽  
Attention Mechanism ◽  
Experimental Result ◽  
Good Prediction ◽  
Convolutional Network ◽  
Crime Prediction ◽  
New Research

Globalization has been the major contributor to economy boom. While at the same time, it has stimulated the development of crime method as frequent cross-border communication allowed. With the improvement in big data and prediction system of policing work, it has become a new research field to establish an efficient crime prediction model, by which police departments could clamp down on criminal activities more accurately. Besides, this model will be quite beneficial for commanding and dispatching police force thus to improve work efficiency. This paper proposes a combination model, which uses Long Short-Term Memory Network (LSTM) and Graph Convolutional Network (GCN) to predict crime rate and takes advantage of the Attention mechanism to improve the experimental result. By extracting the spatio-temporal characteristics of crimes and increasing the proportion of typical feature, it can not only predict crime quantity, but also detect the degree of crime risk in each region. A rolling forecast of crime data for about three years in the Boston of the United States shows that our model has good prediction performance.

scholarly journals Arabic Sentiment Analysis (ASA) Using Deep Learning Approach

2020 ◽  
Vol 26 (6) ◽  
pp. 85-93
Author(s):  
Abdulhakeem Qusay Al-Bayati ◽  
Ahmed S. Al-Araji ◽  
Saman Hameed Ameen
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Web Sites ◽  
Short Term Memory ◽  
Morphological Structure ◽  
Arabic Language ◽  
Feature Representation ◽  
Main Task ◽  
Arabic Sentiment Analysis

Sentiment analysis is one of the major fields in natural language processing whose main task is to extract sentiments, opinions, attitudes, and emotions from a subjective text. And for its importance in decision making and in people's trust with reviews on web sites, there are many academic researches to address sentiment analysis problems. Deep Learning (DL) is a powerful Machine Learning (ML) technique that has emerged with its ability of feature representation and differentiating data, leading to state-of-the-art prediction results. In recent years, DL has been widely used in sentiment analysis, however, there is scarce in its implementation in the Arabic language field. Most of the previous researches address other languages like English. The proposed model tackles Arabic Sentiment Analysis (ASA) by using a DL approach. ASA is a challenging field where Arabic language has a rich morphological structure more than other languages. In this work, Long Short-Term Memory (LSTM) as a deep neural network has been used for training the model combined with word embedding as a first hidden layer for features extracting. The results show an accuracy of about 82% is achievable using DL method.

scholarly journals Assessing Regression-Based Sentiment Analysis Techniques in Financial Texts

2019 ◽  
Author(s):  
Taynan Ferreira ◽  
Francisco Paiva ◽  
Roberto Silva ◽  
Angel Paula ◽  
Anna Costa ◽  
...  
Keyword(s):  
Sentiment Analysis ◽  
Feature Representation ◽  
Support Vector ◽  
Data Set ◽  
Feature Representations ◽  
Textual Data ◽  
Enormous Amount ◽  
Financial Domain ◽  
Classification Tasks ◽  
The Impact

Sentiment analysis (SA) is increasing its importance due to the enormous amount of opinionated textual data available today. Most of the researches have investigated different models, feature representation and hyperparameters in SA classification tasks. However, few studies were conducted to evaluate the impact of these features on regression SA tasks. In this paper, we conduct such assessment on a financial domain data set by investigating different feature representations and hyperparameters in two important models -- Support Vector Regression (SVR) and Convolution Neural Networks (CNN). We conclude presenting the most relevant feature representations and hyperparameters and how they impact outcomes on a regression SA task.

scholarly journals An efficient sentiment analysis methodology based on long short-term memory networks

2021 ◽  
Author(s):  
J. Shobana ◽  
M. Murali
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Contextual Information ◽  
Short Term ◽  
Good Decision ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Current Research Interest

AbstractSentiment analysis is the process of determining the sentiment polarity (positivity, neutrality or negativity) of the text. As online markets have become more popular over the past decades, online retailers and merchants are asking their buyers to share their opinions about the products they have purchased. As a result, millions of reviews are generated daily, making it difficult to make a good decision about whether a consumer should buy a product. Analyzing these enormous concepts is difficult and time-consuming for product manufacturers. Deep learning is the current research interest in Natural language processing. In the proposed model, Skip-gram architecture is used for better feature extraction of semantic and contextual information of words. LSTM (long short-term memory) is used in the proposed model for understanding complex patterns in textual data. To improve the performance of the LSTM, weight parameters are optimized by the adaptive particle Swarm Optimization algorithm. Extensive experiments were conducted on four datasets proved that our proposed APSO-LSTM model secured higher accuracy over the classical methods such as traditional LSTM, ANN, and SVM. According to simulation results, the proposed model is outperforming other existing models in different metrics.

Sign in / Sign up

Export Citation Format

Share Document