A new stochastic gradient descent possibilistic clustering algorithm

2021 ◽  
pp. 1-18
Author(s):  
Angeliki Koutsimpela ◽  
Konstantinos D. Koutroumbas
Keyword(s):  
Cost Function ◽  
Gradient Descent ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Real Data ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Convergence Results ◽  
Possibilistic Clustering

Several well known clustering algorithms have their own online counterparts, in order to deal effectively with the big data issue, as well as with the case where the data become available in a streaming fashion. However, very few of them follow the stochastic gradient descent philosophy, despite the fact that the latter enjoys certain practical advantages (such as the possibility of (a) running faster than their batch processing counterparts and (b) escaping from local minima of the associated cost function), while, in addition, strong theoretical convergence results have been established for it. In this paper a novel stochastic gradient descent possibilistic clustering algorithm, called O- PCM 2 is introduced. The algorithm is presented in detail and it is rigorously proved that the gradient of the associated cost function tends to zero in the L 2 sense, based on general convergence results established for the family of the stochastic gradient descent algorithms. Furthermore, an additional discussion is provided on the nature of the points where the algorithm may converge. Finally, the performance of the proposed algorithm is tested against other related algorithms, on the basis of both synthetic and real data sets.

Stochastic Gradient Descent Based K-Means Algorithm on Large Scale Data Clustering

2014 ◽  
Vol 687-691 ◽  
pp. 1342-1345 ◽  
Author(s):  
Jie Ding ◽  
Li Peng Zhu ◽  
Bin Hu ◽  
Ren Long Hang ◽  
Yu Bao Sun
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Distance Matrix ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Data Sets ◽  
Human Beings ◽  
Large Scale Data ◽  
Scale Data

With the rapid advance of data collection and storage technique, it is easy to acquire tens of millions or even billions of data sets. How to explore and exploit the useful or interesting information for human beings from these data sets has become an urgent issue. Traditional k-means clustering algorithm has been widely used in data mining community. First, randomly initialize k clustering centres. Then, all instances are classified into k different classes according to their distances to clustering centres. Lastly, update the clustering centres by the mean of its corresponding constituent instances. This whole process will be iterated until convergence. Obviously, at each iteration, distance matrix from all instances to k clustering centres must be calculated which will cost so much time when encounter large scale data sets. To address this issue, in this paper, we proposed a fast optimization algorithm based on stochastic gradient descent (SGD). At each iteration, randomly choose an instance, search its corresponding clustering centre and then update it immediately. Experimental results show that our proposed method achieves a competitive clustering results with less time cost.

Credibilistic Clustering: The Model and Algorithms

2015 ◽  
Vol 23 (04) ◽  
pp. 545-564 ◽  
Author(s):  
Jian Zhou ◽  
Qina Wang ◽  
C.-C. Hung ◽  
Xiajie Yi
Keyword(s):  
Probability Measure ◽  
Set Theory ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Credibility Measure ◽  
Data Sets ◽  
Possibility Measure ◽  
Possibilistic Clustering ◽  
Intuitive Concept ◽  
Robust To Noise

Fuzzy clustering is a widely used approach for data classification by using the fuzzy set theory. The probability measure and the possibility measure are two popular measures which have been used in the fuzzy [Formula: see text]-means algorithm (FCM) and the possibilistic clustering algorithms (PCAs), respectively. However, the numerical experiments revealed that FCM and its derivatives lack the intuitive concept of degree of belongingness, and PCAs suffer from the “coincident problem” and cannot provide very stable results for some data sets. In this study, we propose a new clustering algorithm, called the credibilistic clustering algorithm (CCA), based on the credibility measure. The credibility measure provides some unique properties which can solve the “coincident problem” and noise issue compared with the probability measure and possibility measure. Based on some randomly generated data sets, experimental results compared with FCM and PCA show that CCA can deal with the “coincident problem” with good clustering results, and it is more robust to noise than PCA.

scholarly journals GRAPH BASED CLUSTERING WITH CONSTRAINTS AND ACTIVE LEARNING

2021 ◽  
Vol 37 (1) ◽  
pp. 71-89
Author(s):  
Vu-Tuan Dang ◽  
Viet-Vu Vu ◽  
Hong-Quan Do ◽  
Thi Kieu Oanh Le
Keyword(s):  
Active Learning ◽  
Clustering Algorithm ◽  
Side Information ◽  
Clustering Algorithms ◽  
Real Data ◽  
Data Sets ◽  
Data Set ◽  
Supervised Clustering ◽  
Class Labels ◽  
Graph Based Clustering

During the past few years, semi-supervised clustering has emerged as a new interesting direction in machine learning research. In a semi-supervised clustering algorithm, the clustering results can be significantly improved by using side information, which is available or collected from users. There are two main kinds of side information that can be learned in semi-supervised clustering algorithms: the class labels - called seeds or the pairwise constraints. The first semi-supervised clustering was introduced in 2000, and since that, many algorithms have been presented in literature. However, it is not easy to use both types of side information in the same algorithm. To address the problem, this paper proposes a semi-supervised graph based clustering algorithm that tries to use seeds and constraints in the clustering process, called MCSSGC. Moreover, we introduces a simple but efficient active learning method to collect the constraints that can boost the performance of MCSSGC, named KMMFFQS. In order to verify effectiveness of the proposed algorithm, we conducted a series of experiments not only on real data sets from UCI, but also on a document data set applied in an Information Extraction of Vietnamese documents. These obtained results show that the proposed algorithm can significantly improve the clustering process compared to some recent algorithms.

scholarly journals Parameter calibration with stochastic gradient descent for interacting particle systems driven by neural networks

2021 ◽  
Author(s):  
Simone Göttlich ◽  
Claudia Totzeck
Keyword(s):  
Gradient Descent ◽  
Interacting Particle Systems ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Force Model ◽  
Data Sets ◽  
Optimal Controls ◽  
Parameter Calibration ◽  
Descent Algorithm ◽  
Gradient Descent Algorithm

AbstractWe propose a neural network approach to model general interaction dynamics and an adjoint-based stochastic gradient descent algorithm to calibrate its parameters. The parameter calibration problem is considered as optimal control problem that is investigated from a theoretical and numerical point of view. We prove the existence of optimal controls, derive the corresponding first-order optimality system and formulate a stochastic gradient descent algorithm to identify parameters for given data sets. To validate the approach, we use real data sets from traffic and crowd dynamics to fit the parameters. The results are compared to forces corresponding to well-known interaction models such as the Lighthill–Whitham–Richards model for traffic and the social force model for crowd motion.

scholarly journals On the Convergence Properties of a K-step Averaging Stochastic Gradient Descent Algorithm for Nonconvex Optimization

2018 ◽  
Author(s):  
Fan Zhou ◽  
Guojing Cong
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Learning Problems ◽  
Stochastic Gradient Descent ◽  
Convergence Properties ◽  
Descent Algorithm ◽  
Convergence Results ◽  
Gradient Descent Algorithm ◽  
Parallel Stochastic Gradient Descent

We adopt and analyze a synchronous K-step averaging stochastic gradient descent algorithm which we call K-AVG  for solving large scale machine learning problems. We establish the convergence results of K-AVG for nonconvex objectives. Our analysis of K-AVG applies to many existing variants of synchronous SGD.  We explain why the K-step delay is necessary and leads to better performance than traditional parallel stochastic gradient descent which is equivalent to K-AVG with $K=1$. We also show that K-AVG scales better with the number of learners than asynchronous stochastic gradient descent (ASGD). Another advantage of K-AVG over ASGD is that it allows larger stepsizes and facilitates faster convergence. On a cluster of $128$ GPUs, K-AVG is faster than ASGD implementations and achieves better accuracies and faster convergence for training with the CIFAR-10 dataset.

scholarly journals Neutrosophic Clustering Algorithm Based on Sparse Regular Term Constraint

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Dan Zhang ◽  
Yingcang Ma ◽  
Hu Zhao ◽  
Xiaofei Yang
Keyword(s):  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Large Data ◽  
Real Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Artificial Data ◽  
Research Topics ◽  
Regularization Parameters ◽  
Better Than

Clustering algorithm is one of the important research topics in the field of machine learning. Neutrosophic clustering is the generalization of fuzzy clustering and has been applied to many fields. This paper presents a new neutrosophic clustering algorithm with the help of regularization. Firstly, the regularization term is introduced into the FC-PFS algorithm to generate sparsity, which can reduce the complexity of the algorithm on large data sets. Secondly, we propose a method to simplify the process of determining regularization parameters. Finally, experiments show that the clustering results of this algorithm on artificial data sets and real data sets are mostly better than other clustering algorithms. Our clustering algorithm is effective in most cases.

User Power Behavior Similarity Clustering Based on Unsupervised Extreme Learning Machine Algorithm

2020 ◽  
Vol 13 (5) ◽  
pp. 641-649
Author(s):  
Yuancheng Li ◽  
Yaqi Cui ◽  
Xiaolong Zhang
Keyword(s):  
Extreme Learning Machine ◽  
Clustering Algorithm ◽  
Characteristic Curve ◽  
Clustering Algorithms ◽  
Data Sets ◽  
Residential Areas ◽  
Processing Power ◽  
Learning Machine ◽  
Advanced Metering ◽  
Matlab Programming

Background: Advanced Metering Infrastructure (AMI) for the smart grid is growing rapidly which results in the exponential growth of data collected and transmitted in the device. By clustering this data, it can give the electricity company a better understanding of the personalized and differentiated needs of the user. Objective: The existing clustering algorithms for processing data generally have some problems, such as insufficient data utilization, high computational complexity and low accuracy of behavior recognition. Methods: In order to improve the clustering accuracy, this paper proposes a new clustering method based on the electrical behavior of the user. Starting with the analysis of user load characteristics, the user electricity data samples were constructed. The daily load characteristic curve was extracted through improved extreme learning machine clustering algorithm and effective index criteria. Moreover, clustering analysis was carried out for different users from industrial areas, commercial areas and residential areas. The improved extreme learning machine algorithm, also called Unsupervised Extreme Learning Machine (US-ELM), is an extension and improvement of the original Extreme Learning Machine (ELM), which realizes the unsupervised clustering task on the basis of the original ELM. Results: Four different data sets have been experimented and compared with other commonly used clustering algorithms by MATLAB programming. The experimental results show that the US-ELM algorithm has higher accuracy in processing power data. Conclusion: The unsupervised ELM algorithm can greatly reduce the time consumption and improve the effectiveness of clustering.

Sign in / Sign up

Export Citation Format

Share Document