Accelerating DNN Training Through Selective Localized Learning

Training Deep Neural Networks (DNNs) places immense compute requirements on the underlying hardware platforms, expending large amounts of time and energy. We propose LoCal+SGD, a new algorithmic approach to accelerate DNN training by selectively combining localized or Hebbian learning within a Stochastic Gradient Descent (SGD) based training framework. Back-propagation is a computationally expensive process that requires 2 Generalized Matrix Multiply (GEMM) operations to compute the error and weight gradients for each layer. We alleviate this by selectively updating some layers' weights using localized learning rules that require only 1 GEMM operation per layer. Further, since localized weight updates are performed during the forward pass itself, the layer activations for such layers do not need to be stored until the backward pass, resulting in a reduced memory footprint. Localized updates can substantially boost training speed, but need to be used judiciously in order to preserve accuracy and convergence. We address this challenge through a Learning Mode Selection Algorithm, which gradually selects and moves layers to localized learning as training progresses. Specifically, for each epoch, the algorithm identifies a Localized→SGD transition layer that delineates the network into two regions. Layers before the transition layer use localized updates, while the transition layer and later layers use gradient-based updates. We propose both static and dynamic approaches to the design of the learning mode selection algorithm. The static algorithm utilizes a pre-defined scheduler function to identify the position of the transition layer, while the dynamic algorithm analyzes the dynamics of the weight updates made to the transition layer to determine how the boundary between SGD and localized updates is shifted in future epochs. We also propose a low-cost weak supervision mechanism that controls the learning rate of localized updates based on the overall training loss. We applied LoCal+SGD to 8 image recognition CNNs (including ResNet50 and MobileNetV2) across 3 datasets (Cifar10, Cifar100, and ImageNet). Our measurements on an Nvidia GTX 1080Ti GPU demonstrate upto 1.5× improvement in end-to-end training time with ~0.5% loss in Top-1 classification accuracy.

A New Method for Detecting Sleep Apnea

Sleep apnea is a sleep disorder that can induce hypertension, coronary heart disease, stroke and other diseases, so the detection of sleep apnea is clinically important for the prevention of these diseases. In order to improve the detection performance and verify which physiological signals are better for sleep apnea detection, this paper uses multi-channel signal superposition and channel summation to improve the content of valid information in the original signal. Thirty features are analyzed by Relief feature selection algorithm. Finally, 15 features were used to build a classification model and support vector machine (SVM) was used for classification. The experimental results showed that the highest accuracy of 96.24% was achieved when electrocardiogram (X2) and electroencephalogram (C3-A2) channels were used for channel summation.

Modified Dominance-based soft set Approach for Feature Selection

Big data analysis applications in the field of medical image processing have recently increased rapidly. Feature reduction plays a significant role in eliminating irrelevant features and creating a successful research model for Big Data applications. Fuzzy clustering is used for the segment of the nucleus. Various features, including shape, texture, and color-based features, have been used to address the segmented nucleus. The Modified Dominance Soft Set Feature Selection Algorithm (MDSSA) is intended in this paper to determine the most important features for the classification of leukaemia images. The results of the MDSSA are evaluated using the variance analysis called ANOVA. In the dataset extracted function, the MDSSA selected 17 percent of the features that were more promising than the existing reduction algorithms. The proposed approach also reduces the time needed for further analysis of Big Data. The experimental findings confirm that the performance of the proposed reduction approach is higher than other approaches.

Hybrid Rough Set with Black Hole Optimization Based Feature Selection Algorithm for Protein Structure Prediction

In this paper, a new approach for hybridizing Rough Set Quick Reduct and Relative Reduct approaches with Black Hole optimization algorithm is proposed. This algorithm is inspired of black holes. A black hole is a region of spacetime where the gravitational field is so strong that nothing— not even light— that enters this region can ever escape from it. Every black hole has a mass and charge. In this Algorithm, each solution of problem is considered as a black hole and gravity force is used for global search and the electrical force is used for local search. The proposed algorithm is compared with leading algorithms such as, Rough Set Quick Reduct, Rough Set Relative Reduct, Rough Set particle swarm optimization based Quick Reduct, Rough Set based PSO Relative Reduct, Rough Set Harmony Search based Quick Reduct, and Rough Set Harmony Search based Relative Reduct.