scholarly journals Adapting Hidden Naive Bayes for Text Classification

Mathematics ◽  
2021 ◽  
Vol 9 (19) ◽  
pp. 2378
Author(s):  
Shengfeng Gan ◽  
Shiqi Shao ◽  
Long Chen ◽  
Liangjun Yu ◽  
Liangxiao Jiang
Keyword(s):  
Text Classification ◽  
Conditional Independence ◽  
Structure Learning ◽  
Naive Bayes ◽  
Learning Algorithm ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Efficiency And Effectiveness ◽  
The One ◽  
Structure Extension

Due to its simplicity, efficiency, and effectiveness, multinomial naive Bayes (MNB) has been widely used for text classification. As in naive Bayes (NB), its assumption of the conditional independence of features is often violated and, therefore, reduces its classification performance. Of the numerous approaches to alleviating its assumption of the conditional independence of features, structure extension has attracted less attention from researchers. To the best of our knowledge, only structure-extended MNB (SEMNB) has been proposed so far. SEMNB averages all weighted super-parent one-dependence multinomial estimators; therefore, it is an ensemble learning model. In this paper, we propose a single model called hidden MNB (HMNB) by adapting the well-known hidden NB (HNB). HMNB creates a hidden parent for each feature, which synthesizes all the other qualified features’ influences. For HMNB to learn, we propose a simple but effective learning algorithm without incurring a high-computational-complexity structure-learning process. Our improved idea can also be used to improve complement NB (CNB) and the one-versus-all-but-one model (OVA), and the resulting models are simply denoted as HCNB and HOVA, respectively. The extensive experiments on eleven benchmark text classification datasets validate the effectiveness of HMNB, HCNB, and HOVA.

scholarly journals Structure Extension of Tree-Augmented Naive Bayes

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 721 ◽  
Author(s):  
YuGuang Long ◽  
LiMin Wang ◽  
MingHui Sun
Keyword(s):  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Training Data ◽  
Conditional Probability Distribution ◽  
Independence Assumption ◽  
Bayesian Network Classifiers ◽  
Leibler Divergence ◽  
The Difference ◽  
Structure Extension

Due to the simplicity and competitive classification performance of the naive Bayes (NB), researchers have proposed many approaches to improve NB by weakening its attribute independence assumption. Through the theoretical analysis of Kullback–Leibler divergence, the difference between NB and its variations lies in different orders of conditional mutual information represented by these augmenting edges in the tree-shaped network structure. In this paper, we propose to relax the independence assumption by further generalizing tree-augmented naive Bayes (TAN) from 1-dependence Bayesian network classifiers (BNC) to arbitrary k-dependence. Sub-models of TAN that are built to respectively represent specific conditional dependence relationships may “best match” the conditional probability distribution over the training data. Extensive experimental results reveal that the proposed algorithm achieves bias-variance trade-off and substantially better generalization performance than state-of-the-art classifiers such as logistic regression.

Classification Algorithm for Naïve Bayes Based on Validity and Correlation

2013 ◽  
Vol 303-306 ◽  
pp. 1609-1612
Author(s):  
Huai Lin Dong ◽  
Xiao Dan Zhu ◽  
Qing Feng Wu ◽  
Juan Juan Huang
Keyword(s):  
Classification Accuracy ◽  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Classification Algorithm ◽  
Experimental Results ◽  
Training Data ◽  
Improved Method ◽  
Naive Bayes Classification ◽  
The One

Naïve Bayes classification algorithm based on validity (NBCABV) optimizes the training data by eliminating the noise samples of training data with validity to improve the effect of classification, while it ignores the associations of properties. In consideration of the associations of properties, an improved method that is classification algorithm for Naïve Bayes based on validity and correlation (CANBBVC) is proposed to delete more noise samples with validity and correlation, thus resulting in better classification performance. Experimental results show this model has higher classification accuracy comparing the one based on validity solely.

scholarly journals A Machine Learning Framework for Improving Classification Performance on Credit Approval

2021 ◽  
Vol 10 (1) ◽  
pp. 47-52
Author(s):  
Pulung Hendro Prastyo ◽  
Septian Eko Prasetyo ◽  
Shindy Arti
Keyword(s):  
Machine Learning ◽  
Naive Bayes ◽  
Information Gain ◽  
Learning Algorithm ◽  
Confusion Matrix ◽  
Credit Scoring ◽  
Research Work ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Bayes Algorithm

Credit scoring is a model commonly used in the decision-making process to refuse or accept loan requests. The credit score model depends on the type of loan or credit and is complemented by various credit factors. At present, there is no accurate model for determining which creditors are eligible for loans. Therefore, an accurate and automatic model is needed to make it easier for banks to determine appropriate creditors. To address the problem, we propose a new approach using the combination of a machine learning algorithm (Naïve Bayes), Information Gain (IG), and discretization in classifying creditors. This research work employed an experimental method using the Weka application. Australian Credit Approval data was used as a dataset, which contains 690 instances of data. In this study, Information Gain is employed as a feature selection to select relevant features so that the Naïve Bayes algorithm can work optimally. The confusion matrix is used as an evaluator and 10-fold cross-validation as a validator. Based on experimental results, our proposed method could improve the classification performance, which reached the highest performance in average accuracy, precision, recall, and f-measure with the value of 86.29%, 86.33%, 86.29%, 86.30%, and 91.52%, respectively. Besides, the proposed method also obtains 91.52% of the ROC area. It indicates that our proposed method can be classified as an excellent classification.

EXPLORING CONDITIONS FOR THE OPTIMALITY OF NAÏVE BAYES

2005 ◽  
Vol 19 (02) ◽  
pp. 183-198 ◽  
Author(s):  
HARRY ZHANG
Keyword(s):  
Conditional Independence ◽  
Naive Bayes ◽  
Inductive Learning ◽  
Learning Algorithms ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Necessary Condition ◽  
Conditional Independence Assumption ◽  
Good Classification Performance ◽  
Open Question

Naïve Bayes is one of the most efficient and effective inductive learning algorithms for machine learning and data mining. Its competitive performance in classification is surprising, because the conditional independence assumption on which it is based is rarely true in real-world applications. An open question is: what is the true reason for the surprisingly good performance of Naïve Bayes in classification? In this paper, we propose a novel explanation for the good classification performance of Naïve Bayes. We show that, essentially, dependence distribution plays a crucial role. Here dependence distribution means how the local dependence of an attribute distributes in each class, evenly or unevenly, and how the local dependences of all attributes work together, consistently (supporting a certain classification) or inconsistently (canceling each other out). Specifically, we show that no matter how strong the dependences among attributes are, Naïve Bayes can still be optimal if the dependences distribute evenly in classes, or if the dependences cancel each other out. We propose and prove a sufficient and necessary condition for the optimality of Naïve Bayes. Further, we investigate the optimality of Naïve Bayes under the Gaussian distribution. We present and prove a sufficient condition for the optimality of Naïve Bayes, in which the dependences among attributes exist. This provides evidence that dependences may cancel each other out. Our theoretic analysis can be used in designing learning algorithms. In fact, a major class of learning algorithms for Bayesian networks are conditional independence-based (or CI-based), which are essentially based on dependence. We design a dependence distribution-based algorithm by extending the ChowLiu algorithm, a widely used CI based algorithm. Our experiments show that the new algorithm outperforms the ChowLiu algorithm, which also provides empirical evidence to support our new explanation.

scholarly journals Bayesian Prediction Model Based on Attribute Weighting and Kernel Density Estimations

2015 ◽  
Vol 2015 ◽  
pp. 1-7
Author(s):  
Zhong-Liang Xiang ◽  
Xiang-Ru Yu ◽  
Dae-Ki Kang
Keyword(s):  
Conditional Independence ◽  
Naive Bayes ◽  
Learning Algorithm ◽  
Kernel Method ◽  
Likelihood Estimation ◽  
Naïve Bayes ◽  
Real World Data ◽  
Weighting Method ◽  
Attribute Weighting ◽  
Smooth Kernel

Although naïve Bayes learner has been proven to show reasonable performance in machine learning, it often suffers from a few problems with handling real world data. First problem is conditional independence; the second problem is the usage of frequency estimator. Therefore, we have proposed methods to solve these two problems revolving around naïve Bayes algorithms. By using an attribute weighting method, we have been able to handle conditional independence assumption issue, whereas, for the case of the frequency estimators, we have found a way to weaken the negative effects through our proposed smooth kernel method. In this paper, we have proposed a compact Bayes model, in which a smooth kernel augments weights on likelihood estimation. We have also chosen an attribute weighting method which employs mutual information metric to cooperate with the framework. Experiments have been conducted on UCI benchmark datasets and the accuracy of our proposed learner has been compared with that of standard naïve Bayes. The experimental results have demonstrated the effectiveness and efficiency of our proposed learning algorithm.

scholarly journals TEXT CLASSIFICATION USING NAIVE BAYES UPDATEABLE ALGORITHM IN SBMPTN TEST QUESTIONS

Telematika ◽  
2017 ◽  
Vol 13 (2) ◽  
pp. 123 ◽  
Author(s):  
Ristu Saptono ◽  
Meianto Eko Sulistyo ◽  
Nur Shobriana Trihabsari
Keyword(s):  
Text Classification ◽  
Naive Bayes ◽  
Feature Selection Method ◽  
Classification Performance ◽  
Selection Method ◽  
Naïve Bayes ◽  
Naive Bayes Classifier ◽  
Bayes Classifier ◽  
Naïve Bayes Classifier ◽  
Existing Data

Document classification is a growing interest in the research of text mining. Classification can be done based on the topics, languages, and so on. This study was conducted to determine how Naive Bayes Updateable performs in classifying the SBMPTN exam questions based on its theme. Increment model of one classification algorithm often used in text classification Naive Bayes classifier has the ability to learn from new data introduces with the system even after the classifier has been produced with the existing data. Naive Bayes Classifier classifies the exam questions based on the theme of the field of study by analyzing keywords that appear on the exam questions. One of feature selection method DF-Thresholding is implemented for improving the classification performance. Evaluation of the classification with Naive Bayes classifier algorithm produces 84,61% accuracy.

scholarly journals A Novel Hybrid Approach: Instance Weighted Hidden Naive Bayes

Mathematics ◽  
2021 ◽  
Vol 9 (22) ◽  
pp. 2982
Author(s):  
Liangjun Yu ◽  
Shengfeng Gan ◽  
Yu Chen ◽  
Dechun Luo
Keyword(s):  
Structure Learning ◽  
Naive Bayes ◽  
Hybrid Approach ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Network Algorithms ◽  
Probability Estimates ◽  
Conditional Independence Assumption ◽  
Instance Weighting ◽  
Hidden Naive Bayes

Naive Bayes (NB) is easy to construct but surprisingly effective, and it is one of the top ten classification algorithms in data mining. The conditional independence assumption of NB ignores the dependency between attributes, so its probability estimates are often suboptimal. Hidden naive Bayes (HNB) adds a hidden parent to each attribute, which can reflect dependencies from all the other attributes. Compared with other Bayesian network algorithms, it offers significant improvements in classification performance and avoids structure learning. However, the assumption that HNB regards each instance equivalent in terms of probability estimation is not always true in real-world applications. In order to reflect different influences of different instances in HNB, the HNB model is modified into the improved HNB model. The novel hybrid approach called instance weighted hidden naive Bayes (IWHNB) is proposed in this paper. IWHNB combines instance weighting with the improved HNB model into one uniform framework. Instance weights are incorporated into the improved HNB model to calculate probability estimates in IWHNB. Extensive experimental results show that IWHNB obtains significant improvements in classification performance compared with NB, HNB and other state-of-the-art competitors. Meanwhile, IWHNB maintains the low time complexity that characterizes HNB.

scholarly journals Multiple Naïve Bayes Classifiers Ensemble for Traffic Incident Detection

2014 ◽  
Vol 2014 ◽  
pp. 1-16 ◽  
Author(s):  
Qingchao Liu ◽  
Jian Lu ◽  
Shuyan Chen ◽  
Kangjia Zhao
Keyword(s):  
Decision Tree ◽  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Classifier Ensemble ◽  
Optimal Threshold ◽  
Incident Detection ◽  
Bayes Classifier ◽  
Traffic Incident ◽  
Better Than

This study presents the applicability of the Naïve Bayes classifier ensemble for traffic incident detection. The standard Naive Bayes (NB) has been applied to traffic incident detection and has achieved good results. However, the detection result of the practically implemented NB depends on the choice of the optimal threshold, which is determined mathematically by using Bayesian concepts in the incident-detection process. To avoid the burden of choosing the optimal threshold and tuning the parameters and, furthermore, to improve the limited classification performance of the NB and to enhance the detection performance, we propose an NB classifier ensemble for incident detection. In addition, we also propose to combine the Naïve Bayes and decision tree (NBTree) to detect incidents. In this paper, we discuss extensive experiments that were performed to evaluate the performances of three algorithms: standard NB, NB ensemble, and NBTree. The experimental results indicate that the performances of five rules of the NB classifier ensemble are significantly better than those of standard NB and slightly better than those of NBTree in terms of some indicators. More importantly, the performances of the NB classifier ensemble are very stable.

Sign in / Sign up

Export Citation Format

Share Document