DBSCAN Algorithm for Document Clustering

2019 ◽  
Vol 9 (1) ◽  
pp. 58-66
Author(s):  
Radu G. Creţulescu ◽  
Daniel I. Morariu ◽  
Macarie Breazu ◽  
Daniel Volovici
Keyword(s):  
Clustering Algorithm ◽  
Information Gain ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Feature Selection Method ◽  
Web Pages ◽  
Dbscan Algorithm ◽  
User Query ◽  
The Web

AbstractDocument clustering is a problem of automatically grouping similar document into categories based on some similarity metrics. Almost all available data, usually on the web, are unclassified so we need powerful clustering algorithms that work with these types of data. All common search engines return a list of pages relevant to the user query. This list needs to be generated fast and as correct as possible. For this type of problems, because the web pages are unclassified, we need powerful clustering algorithms. In this paper we present a clustering algorithm called DBSCAN – Density-Based Spatial Clustering of Applications with Noise – and its limitations on documents (or web pages) clustering. Documents are represented using the “bag-of-words” representation (word occurrence frequency). For this type o representation usually a lot of algorithms fail. In this paper we use Information Gain as feature selection method and evaluate the DBSCAN algorithm by its capacity to integrate in the clusters all the samples from the dataset.

scholarly journals An Enhanced Spectral Clustering Algorithm with S-Distance

Symmetry ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 596
Author(s):  
Krishna Kumar Sharma ◽  
Ayan Seal ◽  
Enrique Herrera-Viedma ◽  
Ondrej Krejcar
Keyword(s):  
Spectral Clustering ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
Rank Test ◽  
Customer Churn ◽  
Signed Rank ◽  
Signed Rank Test ◽  
Spectral Clustering Algorithm ◽  
Industrial Databases

Calculating and monitoring customer churn metrics is important for companies to retain customers and earn more profit in business. In this study, a churn prediction framework is developed by modified spectral clustering (SC). However, the similarity measure plays an imperative role in clustering for predicting churn with better accuracy by analyzing industrial data. The linear Euclidean distance in the traditional SC is replaced by the non-linear S-distance (Sd). The Sd is deduced from the concept of S-divergence (SD). Several characteristics of Sd are discussed in this work. Assays are conducted to endorse the proposed clustering algorithm on four synthetics, eight UCI, two industrial databases and one telecommunications database related to customer churn. Three existing clustering algorithms—k-means, density-based spatial clustering of applications with noise and conventional SC—are also implemented on the above-mentioned 15 databases. The empirical outcomes show that the proposed clustering algorithm beats three existing clustering algorithms in terms of its Jaccard index, f-score, recall, precision and accuracy. Finally, we also test the significance of the clustering results by the Wilcoxon’s signed-rank test, Wilcoxon’s rank-sum test, and sign tests. The relative study shows that the outcomes of the proposed algorithm are interesting, especially in the case of clusters of arbitrary shape.

Biomedical Document Clustering Based on Accelerated Symbiotic Organisms Search Algorithm

2021 ◽  
Vol 12 (4) ◽  
pp. 169-185
Author(s):  
Saida Ishak Boushaki ◽  
Omar Bendjeghaba ◽  
Nadjet Kamel
Keyword(s):  
Clustering Algorithm ◽  
Search Algorithm ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Latent Semantic Indexing ◽  
Research Area ◽  
Semantic Indexing ◽  
Local Optima ◽  
Symbiotic Organisms Search ◽  
Symbiotic Organisms

Clustering is an important unsupervised analysis technique for big data mining. It finds its application in several domains including biomedical documents of the MEDLINE database. Document clustering algorithms based on metaheuristics is an active research area. However, these algorithms suffer from the problems of getting trapped in local optima, need many parameters to adjust, and the documents should be indexed by a high dimensionality matrix using the traditional vector space model. In order to overcome these limitations, in this paper a new documents clustering algorithm (ASOS-LSI) with no parameters is proposed. It is based on the recent symbiotic organisms search metaheuristic (SOS) and enhanced by an acceleration technique. Furthermore, the documents are represented by semantic indexing based on the famous latent semantic indexing (LSI). Conducted experiments on well-known biomedical documents datasets show the significant superiority of ASOS-LSI over five famous algorithms in terms of compactness, f-measure, purity, misclassified documents, entropy, and runtime.

Document Clustering

2017 ◽  
pp. 264-281
Author(s):  
Harsha Patil ◽  
R. S. Thakur
Keyword(s):  
Text Mining ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Web Pages ◽  
Digital Form ◽  
Search Query ◽  
Text Documents ◽  
Keen Interest ◽  
Use Of Internet

As we know use of Internet flourishes with its full velocity and in all dimensions. Enormous availability of Text documents in digital form (email, web pages, blog post, news articles, ebooks and other text files) on internet challenges technology to appropriate retrieval of document as a response for any search query. As a result there has been an eruption of interest in people to mine these vast resources and classify them properly. It invigorates researchers and developers to work on numerous approaches of document clustering. Researchers got keen interest in this problem of text mining. The aim of this chapter is to summarised different document clustering algorithms used by researchers.

An Intelligent Web Search Using Multi-Document Summarization

2016 ◽  
Vol 6 (2) ◽  
pp. 41-65 ◽  
Author(s):  
Sheetal A. Takale ◽  
Prakash J. Kulkarni ◽  
Sahil K. Shah
Keyword(s):  
Search Engine ◽  
Web Search ◽  
Document Clustering ◽  
The Internet ◽  
Web Pages ◽  
Extractive Summarization ◽  
Text Understanding ◽  
User Query ◽  
Sentence Clustering

Information available on the internet is huge, diverse and dynamic. Current Search Engine is doing the task of intelligent help to the users of the internet. For a query, it provides a listing of best matching or relevant web pages. However, information for the query is often spread across multiple pages which are returned by the search engine. This degrades the quality of search results. So, the search engines are drowning in information, but starving for knowledge. Here, we present a query focused extractive summarization of search engine results. We propose a two level summarization process: identification of relevant theme clusters, and selection of top ranking sentences to form summarized result for user query. A new approach to semantic similarity computation using semantic roles and semantic meaning is proposed. Document clustering is effectively achieved by application of MDL principle and sentence clustering and ranking is done by using SNMF. Experiments conducted demonstrate the effectiveness of system in semantic text understanding, document clustering and summarization.

SeqPAM

2008 ◽  
pp. 17-38 ◽  
Author(s):  
Pradeep Kumar Kumar ◽  
Raju S. Bapi ◽  
P. Radha Krishna
Keyword(s):  
Similarity Measure ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Similarity Measures ◽  
Sequential Data ◽  
Cluster Validation ◽  
Web Personalization ◽  
Similarity Preserving ◽  
Validation Technique ◽  
The Web

With the growth in the number of web users and necessity for making information available on the web, the problem of web personalization has become very critical and popular. Developers are trying to customize a web site to the needs of specific users with the help of knowledge acquired from user navigational behavior. Since user page visits are intrinsically sequential in nature, efficient clustering algorithms for sequential data are needed. In this paper, we introduce a similarity preserving function called sequence and set similarity measure S3M that captures both the order of occurrence of page visits as well as the content of pages. We conducted pilot experiments comparing the results of PAM, a standard clustering algorithm, with two similarity measures: Cosine and S3M. The goodness of the clusters resulting from both the measures was computed using a cluster validation technique based on average levensthein distance. Results on pilot dataset established the effectiveness of S3M for sequential data. Based on these results, we proposed a new clustering algorithm, SeqPAM for clustering sequential data. We tested the new algorithm on two datasets namely, cti and msnbc datasets. We provided recommendations for web personalization based on the clusters obtained from SeqPAM for msnbc dataset.

Multi-Radius Density Clustering Algorithm Based on Outlier Factor

2014 ◽  
Vol 472 ◽  
pp. 427-431
Author(s):  
Zong Lin Ye ◽  
Hui Cao ◽  
Li Xin Jia ◽  
Yan Bin Zhang ◽  
Gang Quan Si
Keyword(s):  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Similar Process ◽  
The Core ◽  
Dbscan Algorithm ◽  
Proposed Model ◽  
Density Clustering ◽  
Relationship Of ◽  
Core Points ◽  
The Relationship

This paper proposes a novel multi-radius density clustering algorithm based on outlier factor. The algorithm first calculates the density-similar-neighbor-based outlier factor (DSNOF) for each point in the dataset according to the relationship of the density of the point and its neighbors, and then treats the point whose DSNOF is smaller than 1 as a core point. Second, the core points are used for clustering by the similar process of the density based spatial clustering application with noise (DBSCAN) to get some sub-clusters. Third, the proposed algorithm merges the obtained sub-clusters into some clusters. Finally, the points whose DSNOF are larger than 1 are assigned into these clusters. Experiments are performed on some real datasets of the UCI Machine Learning Repository and the experiments results verify that the effectiveness of the proposed model is higher than the DBSCAN algorithm and k-means algorithm and would not be affected by the parameter greatly.

scholarly journals Fine-Tuning an Algorithm for Semantic Document Clustering Using a Similarity Graph

2016 ◽  
Vol 10 (04) ◽  
pp. 527-555
Author(s):  
Lubomir Stanchev
Keyword(s):  
English Language ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Fine Tuning ◽  
Human Judgment ◽  
Multiple Parameters ◽  
Similarity Graph ◽  
Multiple Metrics

In this article, we examine an algorithm for document clustering using a similarity graph. The graph stores words and common phrases from the English language as nodes and it can be used to compute the degree of semantic similarity between any two phrases. One application of the similarity graph is semantic document clustering, that is, grouping documents based on the meaning of the words in them. Since our algorithm for semantic document clustering relies on multiple parameters, we examine how fine-tuning these values affects the quality of the result. Specifically, we use the Reuters-21578 benchmark, which contains [Formula: see text] newswire stories that are grouped in 82 categories using human judgment. We apply the k-means clustering algorithm to group the documents using a similarity metric that is based on keywords matching and one that uses the similarity graph. We evaluate the results of the clustering algorithms using multiple metrics, such as precision, recall, f-score, entropy, and purity.

Research on Parallel DBSCAN Algorithm Design Based on MapReduce

2011 ◽  
Vol 301-303 ◽  
pp. 1133-1138 ◽  
Author(s):  
Yan Xiang Fu ◽  
Wei Zhong Zhao ◽  
Hui Fang Ma
Keyword(s):  
Data Clustering ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Algorithm Design ◽  
Document Retrieval ◽  
Commodity Hardware ◽  
Dbscan Clustering ◽  
Dbscan Algorithm ◽  
Parallel Clustering

Data clustering has been received considerable attention in many applications, such as data mining, document retrieval, image segmentation and pattern classification. The enlarging volumes of information emerging by the progress of technology, makes clustering of very large scale of data a challenging task. In order to deal with the problem, more researchers try to design efficient parallel clustering algorithms. In this paper, we propose a parallel DBSCAN clustering algorithm based on Hadoop, which is a simple yet powerful parallel programming platform. The experimental results demonstrate that the proposed algorithm can scale well and efficiently process large datasets on commodity hardware.

scholarly journals DeepDBSCAN: Deep Density-Based Clustering for Geo-Tagged Photos

2021 ◽  
Vol 10 (8) ◽  
pp. 548
Author(s):  
Jang-You Park ◽  
Dong-June Ryu ◽  
Kwang-Woo Nam ◽  
Insung Jang ◽  
Minseok Jang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Clustering Algorithm ◽  
Nearest Neighbor ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
Learning Technology ◽  
Neighbor Graph ◽  
Density Based Clustering ◽  
Real Objects ◽  
Nearest Neighbor Graph

Density-based clustering algorithms have been the most commonly used algorithms for discovering regions and points of interest in cities using global positioning system (GPS) information in geo-tagged photos. However, users sometimes find more specific areas of interest using real objects captured in pictures. Recent advances in deep learning technology make it possible to recognize these objects in photos. However, since deep learning detection is a very time-consuming task, simply combining deep learning detection with density-based clustering is very costly. In this paper, we propose a novel algorithm supporting deep content and density-based clustering, called deep density-based spatial clustering of applications with noise (DeepDBSCAN). DeepDBSCAN incorporates object detection by deep learning into the density clustering algorithm using the nearest neighbor graph technique. Additionally, this supports a graph-based reduction algorithm that reduces the number of deep detections. We performed experiments with pictures shared by users on Flickr and compared the performance of multiple algorithms to demonstrate the excellence of the proposed algorithm.

A novel dual-domain clustering algorithm for inhomogeneous spatial point event

2020 ◽  
Vol 54 (5) ◽  
pp. 603-623
Author(s):  
Jie Zhu ◽  
Jing Yang ◽  
Shaoning Di ◽  
Jiazhu Zheng ◽  
Leying Zhang
Keyword(s):  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
Spatial Domain ◽  
Spatial Proximity ◽  
Land Use Classification ◽  
Content Type ◽  
Spatial Point ◽  
Spatial Attributes ◽  
Dual Domain

PurposeThe spatial and non-spatial attributes are the two important characteristics of a spatial point, which belong to the two different attribute domains in many Geographic Information Systems applications. The dual clustering algorithms take into account both spatial and non-spatial attributes, where a cluster has not only high proximity in spatial domain but also high similarity in non-spatial domain. In a geographical dataset, traditional dual spatial clustering algorithms discover homogeneous spatially adjacent clusters suffering from the between-cluster inhomogeneity where those spatial points are described in non-spatial domain. To overcome this limitation, a novel dual-domain clustering algorithm (DDCA) is proposed by considering both spatial proximity and attribute similarity with the presence of inhomogeneity.Design/methodology/approachIn this algorithm, Delaunay triangulation with edge length constraints is first employed to construct spatial proximity relationships amongst objects. Then, a clustering strategy based on statistical change detection is designed to obtain clusters with similar attributes.FindingsThe effectiveness and practicability of the proposed algorithm are illustrated by experiments on both simulated datasets and real spatial events. It is found that the proposed algorithm can adaptively and accurately detect clusters with spatial proximity and similar non-spatial attributes under the consideration of inhomogeneity.Originality/valueTraditional dual spatial clustering algorithms discover homogeneous spatially adjacent clusters suffering from the between-cluster inhomogeneity where those spatial points are described in non-spatial domain. The research here is a contribution to developing a dual spatial clustering method considering both spatial proximity and attribute similarity with the presence of inhomogeneity. The detection of these clusters is useful to understand the local patterns of geographical phenomena, such as land use classification, spatial patterns research and big geo-data analysis.

Sign in / Sign up

Export Citation Format

Share Document