Generalization-error-bound-based discriminative dictionary learning

2021 ◽  
Author(s):  
Kaifang Zhang ◽  
Xiaoming Wang ◽  
Tao Xu ◽  
Yajun Du ◽  
Zengxi Huang
Keyword(s):  
Error Bound ◽  
Dictionary Learning ◽  
Generalization Error ◽  
Generalization Error Bound ◽  
Discriminative Dictionary Learning

scholarly journals A Novel Radial Basis Function Neural Network with High Generalization Performance for Nonlinear Process Modelling

Processes ◽  
2022 ◽  
Vol 10 (1) ◽  
pp. 140
Author(s):  
Yanxia Yang ◽  
Pu Wang ◽  
Xuejin Gao
Keyword(s):  
Neural Network ◽  
Radial Basis Function ◽  
Error Bound ◽  
Basis Function ◽  
Nonlinear Process ◽  
Generalization Error ◽  
Sample Mean ◽  
Generalization Ability ◽  
Radial Basis ◽  
Generalization Error Bound

A radial basis function neural network (RBFNN), with a strong function approximation ability, was proven to be an effective tool for nonlinear process modeling. However, in many instances, the sample set is limited and the model evaluation error is fixed, which makes it very difficult to construct an optimal network structure to ensure the generalization ability of the established nonlinear process model. To solve this problem, a novel RBFNN with a high generation performance (RBFNN-GP), is proposed in this paper. The proposed RBFNN-GP consists of three contributions. First, a local generalization error bound, introducing the sample mean and variance, is developed to acquire a small error bound to reduce the range of error. Second, the self-organizing structure method, based on a generalization error bound and network sensitivity, is established to obtain a suitable number of neurons to improve the generalization ability. Third, the convergence of this proposed RBFNN-GP is proved theoretically in the case of structure fixation and structure adjustment. Finally, the performance of the proposed RBFNN-GP is compared with some popular algorithms, using two numerical simulations and a practical application. The comparison results verified the effectiveness of RBFNN-GP.

scholarly journals The Generalization Error Bound for the Multiclass Analytical Center Classifier

2013 ◽  
Vol 2013 ◽  
pp. 1-5
Author(s):  
Zeng Fanzi ◽  
Ma Xiaolong
Keyword(s):  
Error Bound ◽  
Upper Bound ◽  
Linear Optimization ◽  
Generalization Error ◽  
Single Class ◽  
Benchmark Datasets ◽  
Generalization Error Bound ◽  
Multiclass Classifier ◽  
Feasible Space ◽  
Analytical Center

This paper presents the multiclass classifier based on analytical center of feasible space (MACM). This multiclass classifier is formulated as quadratic constrained linear optimization and does not need repeatedly constructing classifiers to separate a single class from all the others. Its generalization error upper bound is proved theoretically. The experiments on benchmark datasets validate the generalization performance of MACM.

scholarly journals Improving the Robustness of Wasserstein Embedding by Adversarial PAC-Bayesian Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 3791-3800
Author(s):  
Daizong Ding ◽  
Mi Zhang ◽  
Xudong Pan ◽  
Min Yang ◽  
Xiangnan He
Keyword(s):  
Error Bound ◽  
Upper Bound ◽  
Bayesian Learning ◽  
State Of The Art ◽  
Comprehensive Analysis ◽  
Graph Analysis ◽  
Generalization Error ◽  
New Challenges ◽  
Generalization Error Bound ◽  
First Time

Node embedding is a crucial task in graph analysis. Recently, several methods are proposed to embed a node as a distribution rather than a vector to capture more information. Although these methods achieved noticeable improvements, their extra complexity brings new challenges. For example, the learned representations of nodes could be sensitive to external noises on the graph and vulnerable to adversarial behaviors. In this paper, we first derive an upper bound on generalization error for Wasserstein embedding via the PAC-Bayesian theory. Based on this, we propose an algorithm called Adversarial PAC-Bayesian Learning (APBL) in order to minimize the generalization error bound. Furthermore, we provide a model called Regularized Adversarial Wasserstein Embedding Network (RAWEN) as an implementation of APBL. Besides our comprehensive analysis of the robustness of RAWEN, our work for the first time explores more kinds of embedded distributions. For evaluations, we conduct extensive experiments to demonstrate the effectiveness and robustness of our proposed embedding model compared with the state-of-the-art methods.

scholarly journals Bridging the Gap between Few-Shot and Many-Shot Learning via Distribution Calibration

2021 ◽  
Author(s):  
Shuo Yang ◽  
Songhua Wu ◽  
Tongliang Liu ◽  
Min Xu
Keyword(s):  
Error Bound ◽  
Estimation Error ◽  
Data Distribution ◽  
Approximation Error ◽  
Ground Truth ◽  
Generalization Error ◽  
Learning To Learn ◽  
Ground Truth Data ◽  
Generalization Error Bound ◽  
Training Examples

A major gap between few-shot and many-shot learning is the data distribution empirically observed by the model during training. In few-shot learning, the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples, while the ground-truth data distribution is more accurately uncovered in many-shot learning to learn a well-generalized model. In this paper, we propose to calibrate the distribution of these few-sample classes to be more unbiased to alleviate such an over-fitting problem. The distribution calibration is achieved by transferring statistics from the classes with sufficient examples to those few-sample classes. After calibration, an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. Extensive experiments on three datasets, miniImageNet, tieredImageNet, and CUB, show that a simple linear classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy by a large margin. We also establish a generalization error bound for the proposed distribution-calibration-based few-shot learning, which consists of the distribution assumption error, the distribution approximation error, and the estimation error. This generalization error bound theoretically justifies the effectiveness of the proposed method.

Sign in / Sign up

Export Citation Format

Share Document