scholarly journals Antonym-Synonym Classification Based on New Sub-Space Embeddings

2019 ◽  
Vol 33 ◽  
pp. 6204-6211
Author(s):  
Muhammad Asif Ali ◽  
Yifang Sun ◽  
Xiaoling Zhou ◽  
Wei Wang ◽  
Xiang Zhao
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
Relation Extraction ◽  
Specific Information ◽  
Lexical Semantic ◽  
Word Level ◽  
Novel Approach ◽  
Proposed Model ◽  
Low Performance ◽  
And Performance

Distinguishing antonyms from synonyms is a key challenge for many NLP applications focused on the lexical-semantic relation extraction. Existing solutions relying on large-scale corpora yield low performance because of huge contextual overlap of antonym and synonym pairs. We propose a novel approach entirely based on pre-trained embeddings. We hypothesize that the pre-trained embeddings comprehend a blend of lexical-semantic information and we may distill the task-specific information using Distiller, a model proposed in this paper. Later, a classifier is trained based on features constructed from the distilled sub-spaces along with some word level features to distinguish antonyms from synonyms. Experimental results show that the proposed model outperforms existing research on antonym synonym distinction in both speed and performance.

scholarly journals Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Tiantian Chen ◽  
Nianbin Wang ◽  
Hongbin Wang ◽  
Haomin Zhan
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Features ◽  
Distant Supervision ◽  
Word Level ◽  
Proposed Model ◽  
Relation Prediction ◽  
Better Than

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large-scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word-level attention mechanism-based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1-score on a widely used dataset, which proves that our model performs significantly better than the state-of-the-art RE systems.

scholarly journals An Attention-Based Model Using Character Composition of Entities in Chinese Relation Extraction

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 79 ◽  
Author(s):  
Xiaoyu Han ◽  
Yue Zhang ◽  
Wenkai Zhang ◽  
Tinglei Huang
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Named Entity Recognition ◽  
Relation Extraction ◽  
Entity Recognition ◽  
Additional Information ◽  
Named Entity ◽  
Proposed Model ◽  
The Relationship ◽  
Crucial Part

Relation extraction is a vital task in natural language processing. It aims to identify the relationship between two specified entities in a sentence. Besides information contained in the sentence, additional information about the entities is verified to be helpful in relation extraction. Additional information such as entity type getting by NER (Named Entity Recognition) and description provided by knowledge base both have their limitations. Nevertheless, there exists another way to provide additional information which can overcome these limitations in Chinese relation extraction. As Chinese characters usually have explicit meanings and can carry more information than English letters. We suggest that characters that constitute the entities can provide additional information which is helpful for the relation extraction task, especially in large scale datasets. This assumption has never been verified before. The main obstacle is the lack of large-scale Chinese relation datasets. In this paper, first, we generate a large scale Chinese relation extraction dataset based on a Chinese encyclopedia. Second, we propose an attention-based model using the characters that compose the entities. The result on the generated dataset shows that these characters can provide useful information for the Chinese relation extraction task. By using this information, the attention mechanism we used can recognize the crucial part of the sentence that can express the relation. The proposed model outperforms other baseline models on our Chinese relation extraction dataset.

Enhancing the Security and Performance of Cloud for E-Governance Infrastructure

2022 ◽  
Vol 12 (1) ◽  
pp. 0-0
Keyword(s):  
Large Scale ◽  
Governance System ◽  
Readiness Assessment ◽  
Cloud Data ◽  
Governance Model ◽  
Proposed Model ◽  
And Performance ◽  
Governance Infrastructure ◽  
Hashing Function ◽  
Distributed Infrastructure

E-Governance is getting momentous in India. Over the years, e-Governance has played a major part in every sphere of the economy. In this paper, we have proposed E-MODI (E-governance model for open distributed infrastructure) a centralized e-Governance system for government of India, the implementation of this system is technically based on open distributed infrastructure which comprises of various government bodies in one single centralized unit. Our proposed model identifies three different patterns of cloud computing which are DGC, SGC and CGC. In addition, readiness assessment of the services needs to migrate into cloud. In this paper, we propose energy efficient VM allocation algorithm to achieve higher energy efficiency in large scale cloud data centers when system on optimum mode. Our objectives have been explained in details and experiments were designed to demonstrate the robustness of the multi-layered security which is an integration of High secure lightweight block cipher CSL along with Ultra powerful BLAKE3 hashing function in order to maintain information security triad.

scholarly journals Commonsense Knowledge Aware Conversation Generation with Graph Attention

2018 ◽  
Author(s):  
Hao Zhou ◽  
Tom Young ◽  
Minlie Huang ◽  
Haizhou Zhao ◽  
Jingfang Xu ◽  
...  
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Semantic Information ◽  
Attention Mechanism ◽  
Generation Model ◽  
Dynamic Graph ◽  
Commonsense Knowledge ◽  
Word Generation ◽  
Proposed Model ◽  
Knowledge Graphs

Commonsense knowledge is vital to many natural language processing tasks. In this paper, we present a novel open-domain conversation generation model to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. Given a user post, the model retrieves relevant knowledge graphs from a knowledge base and then encodes the graphs with a static graph attention mechanism, which augments the semantic information of the post and thus supports better understanding of the post. Then, during word generation, the model attentively reads the retrieved knowledge graphs and the knowledge triples within each graph to facilitate better generation through a dynamic graph attention mechanism. This is the first attempt that uses large-scale commonsense knowledge in conversation generation. Furthermore, unlike existing models that use knowledge triples (entities) separately and independently, our model treats each knowledge graph as a whole, which encodes more structured, connected semantic information in the graphs. Experiments show that the proposed model can generate more appropriate and informative responses than state-of-the-art baselines. 

scholarly journals Deep Learning Based on Hierarchical Self-Attention for Finance Distress Prediction Incorporating Text

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Sumei Ruan ◽  
Xusheng Sun ◽  
Ruanxingchen Yao ◽  
Wei Li
Keyword(s):  
Financial Distress ◽  
Financial Risk ◽  
Large Scale ◽  
Early Stage ◽  
Fine Tuning ◽  
Original Text ◽  
Word Level ◽  
Sentence Level ◽  
Proposed Model ◽  
Red Flag

To detect comprehensive clues and provide more accurate forecasting in the early stage of financial distress, in addition to financial indicators, digitalization of lengthy but indispensable textual disclosure, such as Management Discussion and Analysis (MD&A), has been emphasized by researchers. However, most studies divide the long text into words and count words to treat the text as word count vectors, bringing massive invalid information but ignoring meaningful contexts. Aiming to efficiently represent the text of large size, an end-to-end neural networks model based on hierarchical self-attention is proposed in this study after the state-of-the-art pretrained model is introduced for text embedding including contexts. The proposed model has two notable characteristics. First, the hierarchical self-attention only affords the essential content with high weights in word-level and sentence-level and automatically neglects lots of information that has no business with risk prediction, which is suitable for extracting effective parts of the large-scale text. Second, after fine-tuning, the word embedding adapts the specific contexts of samples and conveys the original text expression more accurately without excessive manual operations. Experiments confirm that the addition of text improves the accuracy of financial distress forecasting and the proposed model outperforms benchmark models better at AUC and F2-score. For visualization, the elements in the weight matrix of hierarchical self-attention act as scalers to estimate the importance of each word and sentence. In this way, the “red-flag” statement that implies financial risk is figured out and highlighted in the original text, providing effective references for decision-makers.

scholarly journals A Novel Approach for Malicious URL Detection Based on the Joint Model

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
JianTing Yuan ◽  
YiPeng Liu ◽  
Long Yu
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Uniform Resource Locator ◽  
Word Level ◽  
Network Algorithm ◽  
Model Combining ◽  
Novel Approach ◽  
Neural Network Algorithm

The number of malicious websites is increasing yearly, and many companies and individuals worldwide have suffered losses. Therefore, the detection of malicious websites is a task that needs continuous development. In this study, a joint neural network algorithm model combining the attention mechanism, bidirectional independent recurrent neural network (Bi-IndRNN), and capsule network (CapsNet) is proposed. The word vector tool word2vec trains the character- and word-level uniform resource locator (URL) static embedding vector features. At the same time, the algorithm will also extract texture fingerprint features that can compare the content differences of different malicious web URL binary files. Then, the extracted features are fused and input into the joint neural network algorithm model. First, the multihead attention mechanism is used to extract contextual semantic features by adjusting weights and Bi-IndRNN. Second, CapsNet with dynamic routing is used to extract deep semantic information. Finally, the sigmoid classifier is used for classification. This study uses different methods from different angles to extract more comprehensive features. From the experimental results, the method proposed in this study improves the classification accuracy of malicious web page detection compared with other researchers.

scholarly journals Relation classification via BERT with piecewise convolution and focal loss

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0257092
Author(s):  
Jianyi Liu ◽  
Xi Duan ◽  
Ru Zhang ◽  
Youqiang Sun ◽  
Lei Guan ◽  
...  
Keyword(s):  
Neural Networks ◽  
Semantic Information ◽  
Semantic Representation ◽  
Language Model ◽  
Relation Extraction ◽  
Fine Tuning ◽  
Superior Result ◽  
Useful Knowledge ◽  
Proposed Model ◽  
Relation Classification

Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.

scholarly journals Applying Word Co-Occurrence Graph in Enhancing LDA Model for Topic Discovering in Large-Scaled Text Corpus

2019 ◽  
Vol 8 (2S8) ◽  
pp. 1366-1371
Keyword(s):  
Statistical Analysis ◽  
Topic Modeling ◽  
Topic Model ◽  
Bag Of Words ◽  
Text Corpus ◽  
Document Collections ◽  
Text Document ◽  
Novel Approach ◽  
Proposed Model ◽  
Low Performance

Topic modeling, such as LDA is considered as a useful tool for the statistical analysis of text document collections and other text-based data. Recently, topic modeling becomes an attractive researching field due to its wide applications. However, there are remained disadvantages of traditional topic modeling like as LDA due the shortcoming of bag-of-words (BOW) model as well as low-performance in handle large text corpus. Therefore, in this paper, we present a novel approach of topic model, called LDA-GOW, which is the combination of word co-occurrence, also called: graph-of-words (GOW) model and traditional LDA topic discovering model. The LDA-GOW topic model not only enable to extract more informative topics from text but also be able to leverage the topic discovering process from large-scaled text corpus. We test our proposed model in comparing with the traditional LDA topic model, within several standardized datasets, include: WebKB, Reuters-R8 and annotated scientific documents which are collected from ACM digital library to demonstrate the effectiveness of our proposed model. For overall experiments, our proposed LDA-GOW model gains approximately 70.86% in accuracy.

scholarly journals Distributed Cooperative Jamming with Neighborhood Selection Strategy for Unmanned Aerial Vehicle Swarms

Electronics ◽  
2022 ◽  
Vol 11 (2) ◽  
pp. 184
Author(s):  
Yongkun Zhou ◽  
Dan Song ◽  
Bowen Ding ◽  
Bin Rao ◽  
Man Su ◽  
...  
Keyword(s):  
Large Scale ◽  
State Transition ◽  
Selection Strategy ◽  
Cooperative Jamming ◽  
Proposed Model ◽  
Neighborhood Selection ◽  
Aerial Vehicle ◽  
Ca Model ◽  
And Performance ◽  
Better Than

In system science, a swarm possesses certain characteristics which the isolated parts and the sum do not have. In order to explore emergence mechanism of a large–scale electromagnetic agents (EAs), a neighborhood selection (NS) strategy–based electromagnetic agent cellular automata (EA–CA) model is proposed in this paper. The model describes the process of agent state transition, in which a neighbor with the smallest state difference in each sector area is selected for state transition. Meanwhile, the evolution rules of the traditional CA are improved, and performance of different evolution strategies are compared. An application scenario in which the emergence of multi–jammers suppresses the radar radiation source is designed to demonstrate the effect of the EA–CA model. Experimental results show that the convergence speed of NS strategy is better than those of the traditional CA evolution rules, and the system achieves effective jamming with the target after emergence. It verifies the effectiveness and prospects of the proposed model in the application of electronic countermeasures (ECM).

Sign in / Sign up

Export Citation Format

Share Document