Large-Scale Local Online Similarity/Distance Learning Framework Based on Passive/Aggressive

Similarity/distance measures play a key role in many machine learning, pattern recognition, and data mining algorithms, which leads to the emergence of the metric learning field. Many metric learning algorithms learn a global distance function from data that satisfies the constraints of the problem. However, in many real-world datasets, where the discrimination power of features varies in the different regions of input space, a global metric is often unable to capture the complexity of the task. To address this challenge, local metric learning methods are proposed which learn multiple metrics across the different regions of the input space. Some advantages of these methods include high flexibility and learning a nonlinear mapping, but they typically achieve at the expense of higher time requirements and overfitting problems. To overcome these challenges, this research presents an online multiple metric learning framework. Each metric in the proposed framework is composed of a global and a local component learned simultaneously. Adding a global component to a local metric efficiently reduces the problem of overfitting. The proposed framework is also scalable with both sample size and the dimension of input data. To the best of our knowledge, this is the first local online similarity/distance learning framework based on Passive/Aggressive (PA). In addition, for scalability with the dimension of input data, Dual Random Projection (DRP) is extended for local online learning in the present work. It enables our methods to run efficiently on high-dimensional datasets while maintaining their predictive performance. The proposed framework provides a straightforward local extension to any global online similarity/distance learning algorithm based on PA. Experimental results on some challenging datasets from machine vision community confirm that the extended methods considerably enhance the performance of the related global ones without increasing the time complexity.

Adjustable Coins

In this paper we consider a scenario where there are several algorithms for solving a given problem. Each algorithm is associated with a probability of success and a cost, and there is also a penalty for failing to solve the problem. The user may run one algorithm at a time for the specified cost, or give up and pay the penalty. The probability of success may be implied by randomization in the algorithm, or by assuming a probability distribution on the input space, which lead to different variants of the problem. The goal is to minimize the expected cost of the process under the assumption that the algorithms are independent. We study several variants of this problem, and present possible solution strategies and a hardness result.

A Manifold Learning Perspective on Representation Learning: Learning Decoder and Representations without an Encoder

Autoencoders are commonly used in representation learning. They consist of an encoder and a decoder, which provide a straightforward method to map n-dimensional data in input space to a lower m-dimensional representation space and back. The decoder itself defines an m-dimensional manifold in input space. Inspired by manifold learning, we showed that the decoder can be trained on its own by learning the representations of the training samples along with the decoder weights using gradient descent. A sum-of-squares loss then corresponds to optimizing the manifold to have the smallest Euclidean distance to the training samples, and similarly for other loss functions. We derived expressions for the number of samples needed to specify the encoder and decoder and showed that the decoder generally requires much fewer training samples to be well-specified compared to the encoder. We discuss the training of autoencoders in this perspective and relate it to previous work in the field that uses noisy training examples and other types of regularization. On the natural image data sets MNIST and CIFAR10, we demonstrated that the decoder is much better suited to learn a low-dimensional representation, especially when trained on small data sets. Using simulated gene regulatory data, we further showed that the decoder alone leads to better generalization and meaningful representations. Our approach of training the decoder alone facilitates representation learning even on small data sets and can lead to improved training of autoencoders. We hope that the simple analyses presented will also contribute to an improved conceptual understanding of representation learning.

SpecAugment++: A Hidden Space Data Augmentation Method for AcousticScene Classification

In this paper, we present SpecAugment++, a novel data aug-mentation method for deep neural networks based acousticscene classification (ASC). Different from other popular dataaugmentation methods such as SpecAugment and mixup thatonly work on the input space, SpecAugment++ is applied toboth the input space and the hidden space of the deep neuralnetworks to enhance the input and the intermediate feature rep-resentations. For an intermediate hidden state, the augmentationtechniques consist of masking blocks of frequency channels andmasking blocks of time frames, which improve generalizationby enabling a model to attend not only to the most discrimina-tive parts of the feature, but also the entire parts. Apart fromusing zeros for masking, we also examine two approaches formasking based on the use of other samples within the mini-batch, which helps introduce noises to the networks to makethem more discriminative for classification. The experimentalresults on the DCASE 2018 Task1 dataset and DCASE 2019Task1 dataset show that our proposed method can obtain3.6%and4.7%accuracy gains over a strong baseline without aug-mentation (i.e.CP-ResNet) respectively, and outperforms otherprevious data augmentation methods.

Machine Learning Bias in Predicting High School Grades: A Knowledge Perspective

This study focuses on the machine learning bias when predicting teacher grades. The experimental phase consists of predicting the student grades of 11th and 12thgrade Portuguese high school grades and computing the bias and variance decomposition. In the base implementation, only the academic achievement critical factors are considered. In the second implementation, the preceding year’s grade is appended as an input variable. The machine learning algorithms in use are random forest, support vector machine, and extreme boosting machine. The reasons behind the poor performance of the machine learning algorithms are either the input space poor preciseness or the lack of a sound record of student performance. We introduce the new concept of knowledge bias and a new predictive model classification. Precision education would reduce bias by providing low-bias intensive-knowledge models. To avoid bias, it is not necessary to add knowledge to the input space. Low-bias extensive-knowledge models are achievable simply by appending the student’s earlier performance record to the model. The low-bias intensive-knowledge learning models promoted by precision education are suited to designing new policies and actions toward academic attainments. If the aim is solely prediction, deciding for a low bias knowledge-extensive model can be appropriate and correct. Doi: 10.28991/esj-2021-01298 Full Text: PDF

Multiclass recognition of Alzheimer’s and Parkinson’s disease using various machine learning techniques: A study

The aging population is primarily affected by Alzheimer’s disease (AD) that is an incurable neurodegenerative disorder. There is a need for an automated efficient technique to diagnose Alzheimer’s in its early stage. Various techniques are used to diagnose AD. EEG and neuroimaging methodologies are widely used to highlight changes in the electrical activity of the brain signals that are helpful for early diagnosis. Parkinson’s disease (PD) is a major neurological disease that results in an average of 50,000 new clinical diagnoses worldwide every year. The voice features are majorly used as the main means to diagnose PD. The major symptoms of PD are loss of intensity, the monotony of loudness and pitch, reduction in stress, unidentified silences, and dysphonia. Even though various innovative models are proposed by explorers about Alzheimer’s and Parkinson’s classification diseases, still there is a need for efficient learning methodologies and techniques. This paper provides a review on using machine learning (ML) together with several feature extraction techniques that is helpful in the early detection of AD with Parkinson’s. The novelty and objective of this study are that the CAD technique is used to improve the accuracy of early diagnosis of AD. The proposed technique depends on the nonlinear process for data dimension reduction, feature removal, and classification using kernel-based support vector machine (SVM) classifiers. The dimension of the input space is radically diminished with kernel methods. As the learning set is labeled, it creates sense to utilize this information to make a dependable method of dropping the input space dimension. The different techniques of ML are explained under the major approaches viz. SVM, artificial neural network (ANN), deep learning (DL), and ensemble methods. A comprehensive assessment is presented at SVM, ANN, and DL approaches for better detection of Alzheimer’s with PD highlighting future insights.

Outage Estimation in Electric Power Distribution Systems Using a Neural Network Ensemble

Outages in an overhead power distribution system are caused by multiple environmental factors, such as weather, trees, and animal activity. Since they form a major portion of the outages, the ability to accurately estimate these outages is a significant step towards enhancing the reliability of power distribution systems. Earlier research with statistical models, neural networks, and committee machines to estimate weather-related and animal-related outages has reported some success. In this paper, a deep neural network ensemble model for outage estimation is proposed. The entire input space is partitioned with a distinct neural network in the ensemble performing outage estimate in each partition. A novel algorithm is proposed to train the neural networks in the ensemble, while simultaneously partitioning the input space in a suitable manner. The proposed approach has been compared with the earlier approaches for outage estimation for four U.S. cities. The results suggest that the proposed method significantly improves the estimates of outages caused by wind and lightning in power distribution systems. A comparative analysis with a previously published model for animal-related outages further establishes the overall effectiveness of the deep neural network ensemble.

Handling Variables, via Inversion of Partial Least Squares Models for Class-Modelling, to Bring Defective Items to Non-Defective Ones

In the context of binary class-modelling techniques, the paper presents the computation in the input space of linear boundaries of a class-model constructed with given values of sensitivity and specificity. This is done by inversion of a decision threshold, set with these values of sensitivity and specificity, in the probabilistic class-models computed by means of PLS-CM (Partial Least Squares for Class-Modelling). The characterization of the boundary hyperplanes, in the latent space (space spanned by the selected latent variables of the fitted PLS model) or in the input space, makes it possible to calculate directions that can be followed to move objects toward the class-model of interest. Different points computed along these directions will show how to modify the input variables (provided they can be manipulated) so that, eventually, a computed ‘object’ would be inside the class-model, in terms of the prediction with the PLS model. When the class of interest is that of “adequate” objects, as for example in some process control or product formulation, the proposed procedure helps in answering the question about how to modify the input variables so that a defective object would be inside the class-model of the adequate (non-defective) ones. This is the situation illustrated with some examples, taken from the literature when modelling the class of adequate objects.