Improved Guarantees and a Multiple-descent Curve for Column Subset Selection and the Nystrom Method (Extended Abstract)

The Column Subset Selection Problem (CSSP) and the Nystrom method are among the leading tools for constructing interpretable low-rank approximations of large datasets by selecting a small but representative set of features or instances. A fundamental question in this area is: what is the cost of this interpretability, i.e., how well can a data subset of size k compete with the best rank k approximation? We develop techniques which exploit spectral properties of the data matrix to obtain improved approximation guarantees which go beyond the standard worst-case analysis. Our approach leads to significantly better bounds for datasets with known rates of singular value decay, e.g., polynomial or exponential decay. Our analysis also reveals an intriguing phenomenon: the cost of interpretability as a function of k may exhibit multiple peaks and valleys, which we call a multiple-descent curve. A lower bound we establish shows that this behavior is not an artifact of our analysis, but rather it is an inherent property of the CSSP and Nystrom tasks. Finally, using the example of a radial basis function (RBF) kernel, we show that both our improved bounds and the multiple-descent curve can be observed on real datasets simply by varying the RBF parameter.

Multi-Nyström Method Based on Multiple Kernel Learning for Large Scale Imbalanced Classification

Extensions of kernel methods for the class imbalance problems have been extensively studied. Although they work well in coping with nonlinear problems, the high computation and memory costs severely limit their application to real-world imbalanced tasks. The Nyström method is an effective technique to scale kernel methods. However, the standard Nyström method needs to sample a sufficiently large number of landmark points to ensure an accurate approximation, which seriously affects its efficiency. In this study, we propose a multi-Nyström method based on mixtures of Nyström approximations to avoid the explosion of subkernel matrix, whereas the optimization to mixture weights is embedded into the model training process by multiple kernel learning (MKL) algorithms to yield more accurate low-rank approximation. Moreover, we select subsets of landmark points according to the imbalance distribution to reduce the model’s sensitivity to skewness. We also provide a kernel stability analysis of our method and show that the model solution error is bounded by weighted approximate errors, which can help us improve the learning process. Extensive experiments on several large scale datasets show that our method can achieve a higher classification accuracy and a dramatical speedup of MKL algorithms.

Nyström Method to Solve Two-Dimensional Volterra Integral Equation with Discontinuous Kernel

In this paper, a linear two-dimensional Volterra integral equation of the second kind with the discontinuous kernel is considered. The conditions for ensuring the existence of a unique continuous solution are mentioned. The product Nystrom method, as a well-known method of solving singular integral equations, is presented. Therefore, the Nystrom method is applied to the linear Volterra integral equation with the discontinuous kernel to convert it to a linear algebraic system. Some formulas are expanded in two dimensions. Weights’ functions of the Nystrom method are obtained for kernels of logarithmic and Carleman types. Some numerical applications are presented to show the efficiency and accuracy of the proposed method. Maple18 is used to compute numerical solutions. The estimated error is calculated in each case. The Nystrom method is useful and effective in treating the two-dimensional singular Volterra integral equation. Finally, we conclude that the time factor and the parameter v have a clear effect on the results.