scholarly journals Network representation learning method embedding linear and nonlinear network structures

Semantic Web ◽  
2022 ◽  
pp. 1-16
Author(s):  
Hu Zhang ◽  
Jingjing Zhou ◽  
Ru Li ◽  
Yue Fan
Keyword(s):  
Rapid Development ◽  
Linear Structure ◽  
Representation Learning ◽  
Nonlinear Structure ◽  
Structure Information ◽  
Nonlinear Network ◽  
Network Representation ◽  
Proposed Model ◽  
Representation Method ◽  
Low Dimensional

With the rapid development of neural networks, much attention has been focused on network embedding for complex network data, which aims to learn low-dimensional embedding of nodes in the network and how to effectively apply learned network representations to various graph-based analytical tasks. Two typical models exist namely the shallow random walk network representation method and deep learning models such as graph convolution networks (GCNs). The former one can be used to capture the linear structure of the network using depth-first search (DFS) and width-first search (BFS), whereas Hierarchical GCN (HGCN) is an unsupervised graph embedding that can be used to describe the global nonlinear structure of the network via aggregating node information. However, the two existing kinds of models cannot simultaneously capture the nonlinear and linear structure information of nodes. Thus, the nodal characteristics of nonlinear and linear structures are explored in this paper, and an unsupervised representation method based on HGCN that joins learning of shallow and deep models is proposed. Experiments on node classification and dimension reduction visualization are carried out on citation, language, and traffic networks. The results show that, compared with the existing shallow network representation model and deep network model, the proposed model achieves better performances in terms of micro-F1, macro-F1 and accuracy scores.

scholarly journals Deep Multiple Auto-Encoder-Based Multi-view Clustering

2021 ◽  
Author(s):  
Guowang Du ◽  
Lihua Zhou ◽  
Yudi Yang ◽  
Kevin Lü ◽  
Lizhen Wang
Keyword(s):  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Representation Learning ◽  
Underlying Structure ◽  
Unified Framework ◽  
Nonlinear Structure ◽  
Heterogeneous Information ◽  
Structure Information ◽  
Real World Datasets ◽  
Low Dimensional

AbstractMulti-view clustering (MVC), which aims to explore the underlying structure of data by leveraging heterogeneous information of different views, has brought along a growth of attention. Multi-view clustering algorithms based on different theories have been proposed and extended in various applications. However, most existing MVC algorithms are shallow models, which learn structure information of multi-view data by mapping multi-view data to low-dimensional representation space directly, ignoring the nonlinear structure information hidden in each view, and thus, the performance of multi-view clustering is weakened to a certain extent. In this paper, we propose a deep multi-view clustering algorithm based on multiple auto-encoder, termed MVC-MAE, to cluster multi-view data. MVC-MAE adopts auto-encoder to capture the nonlinear structure information of each view in a layer-wise manner and incorporate the local invariance within each view and consistent as well as complementary information between any two views together. Besides, we integrate the representation learning and clustering into a unified framework, such that two tasks can be jointly optimized. Extensive experiments on six real-world datasets demonstrate the promising performance of our algorithm compared with 15 baseline algorithms in terms of two evaluation metrics.

scholarly journals A TWO-CHANNEL MODEL FOR REPRESENTATION LEARNING IN VIETNAMESE SENTIMENT CLASSIFICATION PROBLEM

2020 ◽  
Vol 36 (4) ◽  
pp. 305-323
Author(s):  
Quan Hoang Nguyen ◽  
Ly Vu ◽  
Quang Uy Nguyen
Keyword(s):  
Channel Model ◽  
Rapid Development ◽  
Representation Learning ◽  
Classification Problem ◽  
Sentiment Classification ◽  
Complex Sentences ◽  
Digital World ◽  
Part Of Speech ◽  
Proposed Model ◽  
Important Research Topic

Sentiment classification (SC) aims to determine whether a document conveys a positive or negative opinion. Due to the rapid development of the digital world, SC has become an important research topic that affects many aspects of our life. In SC based on machine learning, the representation of the document strongly influences on its accuracy. Word Embedding (WE)-based techniques, i.e., Word2vec techniques, are proved to be beneficial techniques to the SC problem. However, Word2vec is often not enough to represent the semantic of documents with complex sentences of Vietnamese. In this paper, we propose a new representation learning model called a \textbf{two-channel vector} to learn a higher-level feature of a document in SC. Our model uses two neural networks to learn the semantic feature, i.e., Word2vec and the syntactic feature, i.e., Part of Speech tag (POS). Two features are then combined and input to a \textit{Softmax} function to make the final classification. We carry out intensive experiments on $4$ recent Vietnamese sentiment datasets to evaluate the performance of the proposed architecture. The experimental results demonstrate that the proposed model can significantly enhance the accuracy of SC problems compared to two single models and a state-of-the-art ensemble method.

scholarly journals Exponential Family Graph Embeddings

2020 ◽  
Vol 34 (04) ◽  
pp. 3357-3364
Author(s):  
Abdulkadir Celikkanat ◽  
Fragkiskos D. Malliaros
Keyword(s):  
Random Walk ◽  
Exponential Family ◽  
Representation Learning ◽  
Learning Problems ◽  
Interaction Patterns ◽  
Network Representation ◽  
Learning Tasks ◽  
Learning Techniques ◽  
Real World Datasets ◽  
Low Dimensional

Representing networks in a low dimensional latent space is a crucial task with many interesting applications in graph learning problems, such as link prediction and node classification. A widely applied network representation learning paradigm is based on the combination of random walks for sampling context nodes and the traditional Skip-Gram model to capture center-context node relationships. In this paper, we emphasize on exponential family distributions to capture rich interaction patterns between nodes in random walk sequences. We introduce the generic exponential family graph embedding model, that generalizes random walk-based network representation learning techniques to exponential family conditional distributions. We study three particular instances of this model, analyzing their properties and showing their relationship to existing unsupervised learning models. Our experimental evaluation on real-world datasets demonstrates that the proposed techniques outperform well-known baseline methods in two downstream machine learning tasks.

scholarly journals An Attention-Based Graph Neural Network for Heterogeneous Structural Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 4132-4139
Author(s):  
Huiting Hong ◽  
Hantao Guo ◽  
Yucheng Lin ◽  
Xiaoqing Yang ◽  
Zang Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Structural Information ◽  
Representation Learning ◽  
Graph Representation ◽  
Heterogeneous Information ◽  
Domain Experts ◽  
Proposed Model ◽  
Meta Path ◽  
Low Dimensional ◽  
Public Datasets

In this paper, we focus on graph representation learning of heterogeneous information network (HIN), in which various types of vertices are connected by various types of relations. Most of the existing methods conducted on HIN revise homogeneous graph embedding models via meta-paths to learn low-dimensional vector space of HIN. In this paper, we propose a novel Heterogeneous Graph Structural Attention Neural Network (HetSANN) to directly encode structural information of HIN without meta-path and achieve more informative representations. With this method, domain experts will not be needed to design meta-path schemes and the heterogeneous information can be processed automatically by our proposed model. Specifically, we implicitly represent heterogeneous information using the following two methods: 1) we model the transformation between heterogeneous vertices through a projection in low-dimensional entity spaces; 2) afterwards, we apply the graph neural network to aggregate multi-relational information of projected neighborhood by means of attention mechanism. We also present three extensions of HetSANN, i.e., voices-sharing product attention for the pairwise relationships in HIN, cycle-consistency loss to retain the transformation between heterogeneous entity spaces, and multi-task learning with full use of information. The experiments conducted on three public datasets demonstrate that our proposed models achieve significant and consistent improvements compared to state-of-the-art solutions.

scholarly journals Identifying Disease Related Genes by Network Representation and Convolutional Neural Network

2021 ◽  
Vol 9 ◽  
Author(s):  
Bolin Chen ◽  
Yourui Han ◽  
Xuequn Shang ◽  
Shenggui Zhang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Biological Networks ◽  
Dimensional Space ◽  
Scale Free ◽  
Network Representation ◽  
Disease Related Genes ◽  
Representation Method ◽  
Low Dimensional

The identification of disease related genes plays essential roles in bioinformatics. To achieve this, many powerful machine learning methods have been proposed from various computational aspects, such as biological network analysis, classification, regression, deep learning, etc. Among them, deep learning based methods have gained big success in identifying disease related genes in terms of higher accuracy and efficiency. However, these methods rarely handle the following two issues very well, which are (1) the multifunctions of many genes; and (2) the scale-free property of biological networks. To overcome these, we propose a novel network representation method to transfer individual vertices together with their surrounding topological structures into image-like datasets. It takes each node-induced sub-network as a represented candidate, and adds its environmental characteristics to generate a low-dimensional space as its representation. This image-like datasets can be applied directly in a Convolutional Neural Network-based method for identifying cancer-related genes. The numerical experiments show that the proposed method can achieve the AUC value at 0.9256 in a single network and at 0.9452 in multiple networks, which outperforms many existing methods.

scholarly journals Network Representation Learning-Based Drug Mechanism Discovery and Anti-Inflammatory Response Against COVID-19

2021 ◽  
Author(s):  
Wang Xiaoqi ◽  
Bin Xin ◽  
Zhijian Xu ◽  
Kenli LI ◽  
Fei Li ◽  
...  
Keyword(s):  
Inflammatory Response ◽  
Inflammatory Responses ◽  
Representation Learning ◽  
Binding Modes ◽  
Network Representation ◽  
Drug Mechanism ◽  
Docking Program ◽  
Therapeutic Development ◽  
Anti Inflammatory ◽  
Low Dimensional

<p>Recent studies have been demonstrated that the excessive inflammatory response is an important factor of death in COVID-19 patients. In this study, we proposed a network representation learning-based methodology, termed AIdrug2cov, to discover drug mechanism and anti-inflammatory response for patients with COVID-19. This work explores the multi-hub characteristic of a heterogeneous drug network integrating 8 unique networks. Inspired by the multi-hub characteristic, we design three billion special meta paths to train a deep representation model for learning low-dimensional vectors that integrate long-range structure dependency and complex semantic relation among network nodes. Using the representation vectors, AIdrug2cov identifies 40 potential targets and 22 high-confidence drugs that bind to tumor necrosis factor(TNF)-α or interleukin(IL)-6 to prevent excessive inflammatory responses in COVID-19 patients. Finally, we analyze mechanisms of action based on PubMed publications and ongoing clinical trials, and explore the possible binding modes between the new predicted drugs and targets via docking program. In addition, the results in 5 pharmacological application suggested that AIdrug2cov significantly outperforms 5 other state-of-the-art network representation approaches, future demonstrating the availability of AIdrug2cov in drug development field. In summary, AIdrug2cov is practically useful for accelerating COVID-19 therapeutic development. The source code and data can be downloaded from https://github.com/pengsl-lab/AIdrug2cov.git.</p>

scholarly journals Feature Hashing for Network Representation Learning

2018 ◽  
Author(s):  
Qixiang Wang ◽  
Shanfeng Wang ◽  
Maoguo Gong ◽  
Yue Wu
Keyword(s):  
Link Prediction ◽  
Feature Space ◽  
Representation Learning ◽  
Learning Approaches ◽  
Network Representation ◽  
Proximity Matrix ◽  
Low Dimensional ◽  
Vector Representations ◽  
Feature Hashing ◽  
Node Embeddings

The goal of network representation learning is to embed nodes so as to encode the proximity structures of a graph into a continuous low-dimensional feature space. In this paper, we propose a novel algorithm called node2hash based on feature hashing for generating node embeddings. This approach follows the encoder-decoder framework. There are two main mapping functions in this framework. The first is an encoder to map each node into high-dimensional vectors. The second is a decoder to hash these vectors into a lower dimensional feature space. More specifically, we firstly derive a proximity measurement called expected distance as target which combines position distribution and co-occurrence statistics of nodes over random walks so as to build a proximity matrix, then introduce a set of T different hash functions into feature hashing to generate uniformly distributed vector representations of nodes from the proximity matrix. Compared with the existing state-of-the-art network representation learning approaches, node2hash shows a competitive performance on multi-class node classification and link prediction tasks on three real-world networks from various domains.

scholarly journals TransPath: Representation Learning for Heterogeneous Information Networks via Translation Mechanism

2018 ◽  
Author(s):  
Yang Fang ◽  
Xiang Zhao ◽  
Zhen Tan
Keyword(s):  
Large Scale ◽  
Representation Learning ◽  
Information Networks ◽  
Heterogeneous Information ◽  
Structure Information ◽  
Heterogeneous Information Networks ◽  
Network Representation ◽  
Meta Path ◽  
Translation Mechanism ◽  
Real World Datasets

In this paper, we propose a novel network representation learning model TransPath to encode heterogeneous information networks (HINs). Traditional network representation learning models aim to learn the embeddings of a homogeneous network. TransPath is able to capture the rich semantic and structure information of a HIN via meta-paths. We take advantage of the concept of translation mechanism in knowledge graph which regards a meta-path, instead of an edge, as a translating operation from the first node to the last node. Moreover, we propose a user-guided meta-path sampling strategy which takes users' preference as a guidance, which could explore the semantics of a path more precisely, and meanwhile improve model efficiency via the avoidance of other noisy and meaningless meta-paths. We evaluate our model on two large-scale real-world datasets DBLP and YELP, and two benchmark tasks similarity search and node classification. We observe that TransPath outperforms other state-of-the-art baselines consistently and significantly.

scholarly journals An Information-Explainable Random Walk Based Unsupervised Network Representation Learning Framework on Node Classification Tasks

Mathematics ◽  
2021 ◽  
Vol 9 (15) ◽  
pp. 1767
Author(s):  
Xin Xu ◽  
Yang Lu ◽  
Yupeng Zhou ◽  
Zhiguo Fu ◽  
Yanjie Fu ◽  
...  
Keyword(s):  
Random Walk ◽  
Representation Learning ◽  
Local Information ◽  
Learning Framework ◽  
Network Representation ◽  
Label Node ◽  
Label Information ◽  
Classification Tasks ◽  
Node Classification ◽  
Low Dimensional

Network representation learning aims to learn low-dimensional, compressible, and distributed representational vectors of nodes in networks. Due to the expensive costs of obtaining label information of nodes in networks, many unsupervised network representation learning methods have been proposed, where random walk strategy is one of the wildly utilized approaches. However, the existing random walk based methods have some challenges, including: 1. The insufficiency of explaining what network knowledge in the walking path-samplings; 2. The adverse effects caused by the mixture of different information in networks; 3. The poor generality of the methods with hyper-parameters on different networks. This paper proposes an information-explainable random walk based unsupervised network representation learning framework named Probabilistic Accepted Walk (PAW) to obtain network representation from the perspective of the stationary distribution of networks. In the framework, we design two stationary distributions based on nodes’ self-information and local-information of networks to guide our proposed random walk strategy to learn representational vectors of networks through sampling paths of nodes. Numerous experimental results demonstrated that the PAW could obtain more expressive representation than the other six widely used unsupervised network representation learning baselines on four real-world networks in single-label and multi-label node classification tasks.

scholarly journals Inferring the Disease-Associated miRNAs Based on Network Representation Learning and Convolutional Neural Networks

2019 ◽  
Vol 20 (15) ◽  
pp. 3648 ◽  
Author(s):  
Xuan ◽  
Sun ◽  
Wang ◽  
Zhang ◽  
Pan
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Prediction Models ◽  
Prediction Method ◽  
Feature Space ◽  
Representation Learning ◽  
Superior Performance ◽  
Network Representation ◽  
Disease Associations ◽  
Low Dimensional

Identification of disease-associated miRNAs (disease miRNAs) are critical for understanding etiology and pathogenesis. Most previous methods focus on integrating similarities and associating information contained in heterogeneous miRNA-disease networks. However, these methods establish only shallow prediction models that fail to capture complex relationships among miRNA similarities, disease similarities, and miRNA-disease associations. We propose a prediction method on the basis of network representation learning and convolutional neural networks to predict disease miRNAs, called CNNMDA. CNNMDA deeply integrates the similarity information of miRNAs and diseases, miRNA-disease associations, and representations of miRNAs and diseases in low-dimensional feature space. The new framework based on deep learning was built to learn the original and global representation of a miRNA-disease pair. First, diverse biological premises about miRNAs and diseases were combined to construct the embedding layer in the left part of the framework, from a biological perspective. Second, the various connection edges in the miRNA-disease network, such as similarity and association connections, were dependent on each other. Therefore, it was necessary to learn the low-dimensional representations of the miRNA and disease nodes based on the entire network. The right part of the framework learnt the low-dimensional representation of each miRNA and disease node based on non-negative matrix factorization, and these representations were used to establish the corresponding embedding layer. Finally, the left and right embedding layers went through convolutional modules to deeply learn the complex and non-linear relationships among the similarities and associations between miRNAs and diseases. Experimental results based on cross validation indicated that CNNMDA yields superior performance compared to several state-of-the-art methods. Furthermore, case studies on lung, breast, and pancreatic neoplasms demonstrated the powerful ability of CNNMDA to discover potential disease miRNAs.

Sign in / Sign up

Export Citation Format

Share Document