Canal-LASSO: A sparse noise-resilient online linear regression model

2020 ◽  
Vol 24 (5) ◽  
pp. 993-1010
Author(s):  
Hejie Lei ◽  
Xingke Chen ◽  
Ling Jian
Keyword(s):  
Variable Selection ◽  
Regression Model ◽  
Concept Drift ◽  
Noisy Data ◽  
Penalized Regression ◽  
Streaming Data ◽  
Shrinkage Estimation ◽  
Absolute Deviation ◽  
Explanatory Variables ◽  
Selection Tasks

Least absolute shrinkage and selection operator (LASSO) is one of the most commonly used methods for shrinkage estimation and variable selection. Robust variable selection methods via penalized regression, such as least absolute deviation LASSO (LAD-LASSO), etc., have gained growing attention in works of literature. However those penalized regression procedures are still sensitive to noisy data. Furthermore, “concept drift” makes learning from streaming data fundamentally different from the traditional batch learning. Focusing on the shrinkage estimation and variable selection tasks on noisy streaming data, this paper presents a noise-resilient online learning regression model, i.e. canal-LASSO. Comparing with the LASSO and LAD-LASSO, canal-LASSO is resistant to noisy data in both explanatory variables and response variables. Extensive simulation studies demonstrate satisfactory sparseness and noise-resilient performances of canal-LASSO.

scholarly journals Proposing Robust LAD-Atan Penalty of Regression Model Estimation for High Dimensional Data

2020 ◽  
Vol 17 (2) ◽  
pp. 0550
Author(s):  
Ali Hameed Yousef ◽  
Omar Abdulmohsin Ali
Keyword(s):  
Variable Selection ◽  
Regression Model ◽  
High Dimensional Data ◽  
Real Data ◽  
Penalized Regression ◽  
Good Method ◽  
Superior Performance ◽  
High Dimensional ◽  
Absolute Deviation ◽  
Heavy Tailed

         The issue of penalized regression model has received considerable critical attention to variable selection. It plays an essential role in dealing with high dimensional data. Arctangent denoted by the Atan penalty has been used in both estimation and variable selection as an efficient method recently. However, the Atan penalty is very sensitive to outliers in response to variables or heavy-tailed error distribution. While the least absolute deviation is a good method to get robustness in regression estimation. The specific objective of this research is to propose a robust Atan estimator from combining these two ideas at once. Simulation experiments and real data applications show that the proposed LAD-Atan estimator has superior performance compared with other estimators.  

A Fuzzy Penalized Regression Model with Variable Selection

2021 ◽  
pp. 114696
Author(s):  
M. Kashani ◽  
M. Arashi ◽  
M.R. Rabiei ◽  
P.D’Urso ◽  
L. De Giovanni
Keyword(s):  
Variable Selection ◽  
Regression Model ◽  
Penalized Regression

scholarly journals Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights

Entropy ◽  
2020 ◽  
Vol 23 (1) ◽  
pp. 33
Author(s):  
Edmore Ranganai ◽  
Innocent Mudhombo
Keyword(s):  
Variable Selection ◽  
Quantile Regression ◽  
Data Sets ◽  
Least Absolute Deviation ◽  
Minimum Covariance Determinant ◽  
Absolute Deviation ◽  
Leverage Points ◽  
Influential Points ◽  
Computationally Intensive ◽  
Space Data

The importance of variable selection and regularization procedures in multiple regression analysis cannot be overemphasized. These procedures are adversely affected by predictor space data aberrations as well as outliers in the response space. To counter the latter, robust statistical procedures such as quantile regression which generalizes the well-known least absolute deviation procedure to all quantile levels have been proposed in the literature. Quantile regression is robust to response variable outliers but very susceptible to outliers in the predictor space (high leverage points) which may alter the eigen-structure of the predictor matrix. High leverage points that alter the eigen-structure of the predictor matrix by creating or hiding collinearity are referred to as collinearity influential points. In this paper, we suggest generalizing the penalized weighted least absolute deviation to all quantile levels, i.e., to penalized weighted quantile regression using the RIDGE, LASSO, and elastic net penalties as a remedy against collinearity influential points and high leverage points in general. To maintain robustness, we make use of very robust weights based on the computationally intensive high breakdown minimum covariance determinant. Simulations and applications to well-known data sets from the literature show an improvement in variable selection and regularization due to the robust weighting formulation.

scholarly journals L2,1-norm regularized multivariate regression model with applications to genomic prediction

2021 ◽  
Author(s):  
Alain J Mbebi ◽  
Hao Tong ◽  
Zoran Nikoloski
Keyword(s):  
Variable Selection ◽  
Regression Model ◽  
Multivariate Regression ◽  
Statistical Testing ◽  
Supplementary Information ◽  
Nucleotide Polymorphisms ◽  
Multiple Traits ◽  
Multivariate Regression Model ◽  
Proposed Model ◽  
Iterative Optimization Algorithm

AbstractMotivationGenomic selection (GS) is currently deemed the most effective approach to speed up breeding of agricultural varieties. It has been recognized that consideration of multiple traits in GS can improve accuracy of prediction for traits of low heritability. However, since GS forgoes statistical testing with the idea of improving predictions, it does not facilitate mechanistic understanding of the contribution of particular single nucleotide polymorphisms (SNP).ResultsHere, we propose a L2,1-norm regularized multivariate regression model and devise a fast and efficient iterative optimization algorithm, called L2,1-joint, applicable in multi-trait GS. The usage of the L2,1-norm facilitates variable selection in a penalized multivariate regression that considers the relation between individuals, when the number of SNPs is much larger than the number of individuals. The capacity for variable selection allows us to define master regulators that can be used in a multi-trait GS setting to dissect the genetic architecture of the analyzed traits. Our comparative analyses demonstrate that the proposed model is a favorable candidate compared to existing state-of-the-art approaches. Prediction and variable selection with datasets from Brassica napus, wheat and Arabidopsis thaliana diversity panels are conducted to further showcase the performance of the proposed model.Availability and implementation: The model is implemented using R programming language and the code is freely available from https://github.com/alainmbebi/L21-norm-GS.Supplementary informationSupplementary data are available at Bioinformatics online.

scholarly journals Handling adversarial concept drift in streaming data

2018 ◽  
Vol 97 ◽  
pp. 18-40 ◽  
Author(s):  
Tegjyot Singh Sethi ◽  
Mehmed Kantardzic
Keyword(s):  
Concept Drift ◽  
Streaming Data
Sign in / Sign up

Export Citation Format

Share Document