A Hybrid Imputation Method Based on Denoising Restricted Boltzmann Machine

2018 ◽  
Vol 10 (2) ◽  
pp. 1-13
Author(s):  
Jiang Xu ◽  
Siqian Liu ◽  
Zhikui Chen ◽  
Yonglin Leng
Keyword(s):  
Incomplete Data ◽  
Imputation Method ◽  
Restricted Boltzmann Machine ◽  
Data Sets ◽  
Data Imputation ◽  
Boltzmann Machine ◽  
Machine Model ◽  
Robust Feature Extraction ◽  
Effectiveness And Efficiency ◽  
Occurrence Matrix

Data imputation is an important issue in data processing and analysis which has serious impact on the results of data mining and learning. Most of the existing algorithms are either utilizing whole data sets for imputation or only considering the correlation among records. Aiming at these problems, the article proposes a hybrid method to fill incomplete data. In order to reduce interference and computation, denoising restricted Boltzmann machine model is developed for robust feature extraction from incomplete data and clustering. Then, the article proposes partial-distance and co-occurrence matrix strategies to measure correlation between records and attributes, respectively. Finally, quantifiable correlation is converted to weights for imputation. Compared with different algorithms, the experimental results confirm the effectiveness and efficiency of the proposed method in data imputation.

Interdependences of Products in Market Baskets: Comparing the Conditional Restricted Boltzmann Machine to the Multivariate Logit Model

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Harald Hruschka
Keyword(s):  
Hidden Variables ◽  
Restricted Boltzmann Machine ◽  
Future Research ◽  
Data Sets ◽  
Boltzmann Machine ◽  
Consumer Durables ◽  
Product Categories ◽  
Simulation Based ◽  
Out Of Sample ◽  
Average Probability

AbstractWe analyze market baskets of individual households in two consumer durables categories (music, computer related products) by the multivariate logit (MVL) model, its finite mixture extension (FM-MVL) and the conditional restricted Boltzmann machine (CRBM). The CRBM attains a vastly better out-of-sample performance than MVL and FM-MVL models. Based on simulation-based likelihood ratio tests we prefer the CRBM to the FM-MVL model. To interpret hidden variables of conditional Boltzmann machines we look at their average probability differences between purchase and non-purchases of any sub-category across all baskets. To measure interdependences we compute cross effects between sub-categories for the best performing FM-MVL model and CRBM. In both product categories the CRBM indicates more or higher positive cross effects than the FM-MVL model. Finally, we suggest appropriate future research based on larger and more detailed data sets.

scholarly journals Genetic Programming for Symbolic Regression on Incomplete Data

2021 ◽  
Author(s):  
◽  
Baligh Al-Helali
Keyword(s):  
Feature Selection ◽  
Transfer Learning ◽  
Incomplete Data ◽  
Missing Values ◽  
Selection Method ◽  
Symbolic Regression ◽  
Instance Selection ◽  
Data Sets ◽  
Data Imputation ◽  
Regression Methods

<p><b>Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously.</b></p> <p>Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning.</p> <p>Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values.</p> <p>Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete.</p> <p>The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning.</p> <p>This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods.</p> <p>This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone.</p> <p>High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data.</p> <p>Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data.</p> <p>In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.</p>

Context-aware restricted Boltzmann machine meets collaborative filtering

2018 ◽  
Vol 44 (2) ◽  
pp. 455-476
Author(s):  
Jingshuai Zhang ◽  
Yuanxin Ouyang ◽  
Weizhu Xie ◽  
Wenge Rong ◽  
Zhang Xiong
Keyword(s):  
Collaborative Filtering ◽  
Contextual Information ◽  
Restricted Boltzmann Machine ◽  
Combination Method ◽  
Data Sets ◽  
Context Aware ◽  
Boltzmann Machine ◽  
Content Type ◽  
Recommendation Accuracy ◽  
Hybrid Recommendation

Purpose The purpose of this paper is to propose an approach to incorporate contextual information into collaborative filtering (CF) based on the restricted Boltzmann machine (RBM) and deep belief networks (DBNs). Traditionally, neither the RBM nor its derivative model has been applied to modeling contextual information. In this work, the authors analyze the RBM and explore how to utilize a user’s occupation information to enhance recommendation accuracy. Design/methodology/approach The proposed approach is based on the RBM. The authors employ user occupation information as a context to design a context-aware RBM and stack the context-aware RBM to construct DBNs for recommendations. Findings The experiments on the MovieLens data sets show that the user occupation-aware RBM outperforms other CF models, and combinations of different context-aware models by mutual information can obtain better accuracy. Moreover, the context-aware DBNs model is superior to baseline methods, indicating that deep networks have more qualifications for extracting preference features. Originality/value To improve recommendation accuracy through modeling contextual information, the authors propose context-aware CF approaches based on the RBM. Additionally, the authors attempt to introduce hybrid weights based on information entropy to combine context-aware models. Furthermore, the authors stack the RBM to construct a context-aware multilayer network model. The results of the experiments not only convey that the context-aware RBM has potential in terms of contextual information but also demonstrate that the combination method, the hybrid recommendation and the multilayer neural network extension have significant benefits for the recommendation quality.

scholarly journals Automated arrhythmia detection from electrocardiogram signal using stacked restricted Boltzmann machine model

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
Saroj Kumar Pandey ◽  
Rekh Ram Janghel ◽  
Aditya Vikram Dev ◽  
Pankaj Kumar Mishra
Keyword(s):  
Deep Learning ◽  
Binary Classification ◽  
Restricted Boltzmann Machine ◽  
Patient Specific ◽  
Second Phase ◽  
Ecg Signal ◽  
List Type ◽  
Independent Data ◽  
Boltzmann Machine ◽  
Machine Model

AbstractSignificant advances in deep learning techniques have made it possible to offer technologically advanced methods to detect cardiac abnormalities. In this study, we have proposed a new deep learning based Restricted Boltzmann machine (RBM) model for the classification of arrhythmias from Electrocardiogram (ECG) signal. The work is divided into three phases where, in the first phase, signal processing is performed, including the normalization of the heartbeats as well as the segmentation of the heartbeats. In the second phase, the stacked RBM model is implemented which extracts the essential features from the ECG signal. Finally, a SoftMax activation function is used that classifies the ECG signal into four types of heartbeat classes according to ANSI/AAMA standards. This stacked RBM model is offered as three types of experiments, patient independent data classification for multi-class, patient independent data for binary classification, and patient specific classification. The best result was obtained using patient independent binary classification with an overall accuracy of 99.61%. For Patient Independent Multi Class classification, accuracy obtained was 98.61% and for patient specific data, the accuracy was 95.13%. The experimental results shows that the developed RBM model has better performance in terms of accuracy, sensitivity and specificity as compared to work mentioned in the other research papers.Article highlights The proposed RBM model is skilled to automatically classify ECG heartbeat according to the ANSI- AAMI standards with accuracy, Recall, specificity. The performance of the RBM model to correctly classify heartbeat classes was found to be improved. The model is fully automatic, hence there is no requirement of additional system like feature extraction, feature selection, and classification.

Restricted Boltzmann Machine based on a Fermi sea

2021 ◽  
Author(s):  
Yana Lyakhova ◽  
Evgeny Alexandrovich Polyakov ◽  
Alexey N Rubtsov
Keyword(s):  
Degrees Of Freedom ◽  
Cold Atoms ◽  
Laws Of Nature ◽  
Restricted Boltzmann Machine ◽  
Data Sets ◽  
Boltzmann Machine ◽  
Log Likelihood ◽  
Observable Data ◽  
Lattice Fermions ◽  
Machine Learning Models

Abstract In recent years, there has been an intensive research on how to exploit the quantum laws of nature in the machine learning. Models have been put forward which employ spins, photons, and cold atoms. In this work we study the possibility of using the lattice fermions to learn the classical data. We propose an alternative to the quantum Boltzmann Machine, the so-called Spin-Fermion Machine (SFM), in which the spins represent the degrees of freedom of the observable data (to be learned), and the fermions represent the correlations between the data. The coupling is linear in spins and quadratic in fermions. The fermions are allowed to tunnel between the lattice sites. The training of SFM can be eciently implemented since there are closed expressions for the log- likelihood gradient. We nd that SFM is more powerful than the classical Restricted Boltzmann Machine (RBM) with the same number of physical degrees of freedom. The reason is that SFM has additional freedom due to the rotation of the Fermi sea. We show examples for several data sets.

scholarly journals Genetic Programming for Symbolic Regression on Incomplete Data

2021 ◽  
Author(s):  
◽  
Baligh Al-Helali
Keyword(s):  
Feature Selection ◽  
Transfer Learning ◽  
Incomplete Data ◽  
Missing Values ◽  
Selection Method ◽  
Symbolic Regression ◽  
Instance Selection ◽  
Data Sets ◽  
Data Imputation ◽  
Regression Methods

<p><b>Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously.</b></p> <p>Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning.</p> <p>Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values.</p> <p>Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete.</p> <p>The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning.</p> <p>This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods.</p> <p>This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone.</p> <p>High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data.</p> <p>Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data.</p> <p>In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.</p>

Sign in / Sign up

Export Citation Format

Share Document