scholarly journals Improved Loss Function for Image Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Chenrui Wen ◽  
Xinhao Yang ◽  
Ke Zhang ◽  
Jiahui Zhang
Keyword(s):  
Image Classification ◽  
Loss Function ◽  
Classification Performance ◽  
Cross Entropy ◽  
Loss Functions ◽  
Network Architectures ◽  
Sampling Function ◽  
Network Training ◽  
Sampling Procedures ◽  
Loss Experiment

An improved loss function free of sampling procedures is proposed to improve the ill-performed classification by sample shortage. Adjustable parameters are used to expand the loss scope, minimize the weight of easily classified samples, and further substitute the sampling function, which are added to the cross-entropy loss and the SoftMax loss. Experiment results indicate that improvements in all classification performance of our loss function are shown in various network architectures and on different datasets. To summarize, compared with traditional loss functions, our improved version not only elevates classification performance but also lowers the difficulty of network training.

A Densely Connected Network Based on U-Net for Medical Image Segmentation

2021 ◽  
Vol 17 (3) ◽  
pp. 1-14
Author(s):  
Zhenzhen Yang ◽  
Pengfei Xu ◽  
Yongpeng Yang ◽  
Bing-Kun Bao
Keyword(s):  
Feature Extraction ◽  
Image Segmentation ◽  
Loss Function ◽  
Network Architecture ◽  
Medical Image ◽  
Medical Image Segmentation ◽  
Cross Entropy ◽  
Loss Functions ◽  
Feature Maps ◽  
Different Levels

The U-Net has become the most popular structure in medical image segmentation in recent years. Although its performance for medical image segmentation is outstanding, a large number of experiments demonstrate that the classical U-Net network architecture seems to be insufficient when the size of segmentation targets changes and the imbalance happens between target and background in different forms of segmentation. To improve the U-Net network architecture, we develop a new architecture named densely connected U-Net (DenseUNet) network in this article. The proposed DenseUNet network adopts a dense block to improve the feature extraction capability and employs a multi-feature fuse block fusing feature maps of different levels to increase the accuracy of feature extraction. In addition, in view of the advantages of the cross entropy and the dice loss functions, a new loss function for the DenseUNet network is proposed to deal with the imbalance between target and background. Finally, we test the proposed DenseUNet network and compared it with the multi-resolutional U-Net (MultiResUNet) and the classic U-Net networks on three different datasets. The experimental results show that the DenseUNet network has significantly performances compared with the MultiResUNet and the classic U-Net networks.

scholarly journals Progressive loss functions for speech enhancement with deep neural networks

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Jorge Llombart ◽  
Dayana Ribas ◽  
Antonio Miguel ◽  
Luis Vicente ◽  
Alfonso Ortega ◽  
...  
Keyword(s):  
Speech Enhancement ◽  
Loss Function ◽  
Network Performance ◽  
Deep Neural Networks ◽  
Function Optimization ◽  
Loss Functions ◽  
Network Architectures ◽  
Optimization Criteria ◽  
Promising Strategy ◽  
Progressive Loss

AbstractThe progressive paradigm is a promising strategy to optimize network performance for speech enhancement purposes. Recent works have shown different strategies to improve the accuracy of speech enhancement solutions based on this mechanism. This paper studies the progressive speech enhancement using convolutional and residual neural network architectures and explores two criteria for loss function optimization: weighted and uniform progressive. This work carries out the evaluation on simulated and real speech samples with reverberation and added noise using REVERB and VoiceHome datasets. Experimental results show a variety of achievements among the loss function optimization criteria and the network architectures. Results show that the progressive design strengthens the model and increases the robustness to distortions due to reverberation and noise.

scholarly journals Novel loss functions for ensemble-based medical image classification

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0261307
Author(s):  
Sivaramakrishnan Rajaraman ◽  
Ghada Zamzmi ◽  
Sameer K. Antani
Keyword(s):  
Loss Function ◽  
State Of The Art ◽  
Model Performance ◽  
Classification Performance ◽  
Loss Functions ◽  
Weighted Averaging ◽  
Classification Task ◽  
Source Distribution ◽  
Multi Class Classification ◽  
The Individual

Medical images commonly exhibit multiple abnormalities. Predicting them requires multi-class classifiers whose training and desired reliable performance can be affected by a combination of factors, such as, dataset size, data source, distribution, and the loss function used to train deep neural networks. Currently, the cross-entropy loss remains the de-facto loss function for training deep learning classifiers. This loss function, however, asserts equal learning from all classes, leading to a bias toward the majority class. Although the choice of the loss function impacts model performance, to the best of our knowledge, we observed that no literature exists that performs a comprehensive analysis and selection of an appropriate loss function toward the classification task under study. In this work, we benchmark various state-of-the-art loss functions, critically analyze model performance, and propose improved loss functions for a multi-class classification task. We select a pediatric chest X-ray (CXR) dataset that includes images with no abnormality (normal), and those exhibiting manifestations consistent with bacterial and viral pneumonia. We construct prediction-level and model-level ensembles to improve classification performance. Our results show that compared to the individual models and the state-of-the-art literature, the weighted averaging of the predictions for top-3 and top-5 model-level ensembles delivered significantly superior classification performance (p < 0.05) in terms of MCC (0.9068, 95% confidence interval (0.8839, 0.9297)) metric. Finally, we performed localization studies to interpret model behavior and confirm that the individual models and ensembles learned task-specific features and highlighted disease-specific regions of interest. The code is available at https://github.com/sivaramakrishnan-rajaraman/multiloss_ensemble_models.

scholarly journals Genderpredictions using Convolution Neural Networks

2020 ◽  
Vol 9 (3) ◽  
pp. 537-540
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Loss Function ◽  
Daily Life ◽  
Cross Entropy ◽  
Convolution Neural Networks ◽  
Max Pooling ◽  
Vast Amount ◽  
Massive Growth

Nowadays Deep learning was advanced so much in our daily life. From 2014, there is massive growth in this technology as there is a vast amount of data present. We are even getting better results from whatever we may do. In my work, I have used Convolution Neural Networks as my project depends on image classification. So what I’m trying to do is I’m using two classes in which one class is male and one class is female. I’m classifying both the classes and trying to predict who is male and who is female. For this, I have been using layers like Sequential, Convolution2D, Max-pooling, Flattening, and finally Dense. So, I connect all of these layers. I have been using two more extra layers which are Convolution2D and max-pooling connected as one layer for better classifications. In my model, I’m using Adam optimizer as I’m having only two classes and in my experiments, I found Adam as a good optimizer and I use binary cross entropy as my loss function as I’m using only two classes if we have more than two classes we can use categorical loss function and the images which I use for predictions will be converted into 64*64 matrix form. In the end, I will be getting predictions as 1 for male and 0 for female.

Neural Network Training with Global Optimization Techniques

2003 ◽  
Vol 13 (02) ◽  
pp. 77-86 ◽  
Author(s):  
Akio Yamazaki ◽  
Teresa B. Ludermir
Keyword(s):  
Neural Network ◽  
Low Complexity ◽  
Classification Performance ◽  
Optimization Techniques ◽  
Fine Tuning ◽  
Simultaneous Optimization ◽  
Network Architectures ◽  
Odor Recognition ◽  
Artificial Nose ◽  
Network Training

This paper presents an approach of using Simulated Annealing and Tabu Search for the simultaneous optimization of neural network architectures and weights. The problem considered is the odor recognition in an artificial nose. Both methods have produced networks with high classification performance and low complexity. Generalization has been improved by using the backpropagation algorithm for fine tuning. The combination of simple and traditional search methods has shown to be very suitable for generating compact and efficient networks.

scholarly journals Generalization of Cross-Entropy Loss Function for Image Classification

2021 ◽  
Vol 3 ◽  
pp. 3-10
Author(s):  
Valeria Andreieva ◽  
Nadiia Shvai
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Image Classification ◽  
Loss Function ◽  
Network Architecture ◽  
Cross Entropy ◽  
Classification Task ◽  
Neural Network Architecture ◽  
Training Set ◽  
Rényi Divergence

Classification task is one of the most common tasks in machine learning. This supervised learning problem consists in assigning each input to one of a finite number of discrete categories. Classification task appears naturally in numerous applications, such as medical image processing, speech recognition, maintenance systems, accident detection, autonomous driving etc.In the last decade methods of deep learning have proven to be extremely efficient in multiple machine learning problems, including classification. Whereas the neural network architecture might depend a lot on data type and restrictions posed by the nature of the problem (for example, real-time applications), the process of its training (i.e. finding model’s parameters) is almost always presented as loss function optimization problem.Cross-entropy is a loss function often used for multiclass classification problems, as it allows to achieve high accuracy results.Here we propose to use a generalized version of this loss based on Renyi divergence and entropy. We remark that in case of binary labels proposed generalization is reduced to cross-entropy, thus we work in the context of soft labels. Specifically, we consider a problem of image classification being solved by application of convolution neural networks with mixup regularizer. The latter expands the training set by taking convex combination of pairs of data samples and corresponding labels. Consequently, labels are no longer binary (corresponding to single class), but have a form of vector of probabilities. In such settings cross-entropy and proposed generalization with Renyi divergence and entropy are distinct, and their comparison makes sense.To measure effectiveness of the proposed loss function we consider image classification problem on benchmark CIFAR-10 dataset. This dataset consists of 60000 images belonging to 10 classes, where images are color and have the size of 32×32. Training set consists of 50000 images, and the test set contains 10000 images.For the convolution neural network, we follow [1] where the same classification task was studied with respect to different loss functions and consider the same neural network architecture in order to obtain comparable results.Experiments demonstrate superiority of the proposed method over cross-entropy for loss function parameter value α < 1. For parameter value α > 1 proposed method shows worse results than cross-entropy loss function. Finally, parameter value α = 1 corresponds to cross-entropy.

scholarly journals Investigation of Loss Functions for Improving Deep Segmentation of Abdominal Organs from MRI

2020 ◽  
Author(s):  
Pedro Furtado
Keyword(s):  
Loss Function ◽  
Magnetic Resonance Images ◽  
Cross Entropy ◽  
Loss Functions ◽  
Multiple Views ◽  
Percentage Points ◽  
Abdominal Organs ◽  
Automated Learning ◽  
Future Work ◽  
Practical Implications

Segmentation of Magnetic Resonance Images (MRI) of abdominal organs is useful for analysis prior to surgical procedures and for further processing. Deep Learning (DL) has become the standard, researchers have proposed improvements that include multiple views, ensembles and voting. Loss function alternatives, while being crucial to guide automated learning, have not been compared in detail. In this work we analyze limitations of popular metrics and their use as loss, study alternative loss variations based on those and other modifications and search for the best approach. An experimental setup was necessary to assess the alternatives. Results for the top scoring network and top scoring loss show improvements between 2 and 11 percentage points (pp) in Jaccard Index (JI), depending on organ and patient (sequence), for a total of 22 pp over 4 organs, all this being obtained just by choosing the best performing loss function instead of cross-entropy or dice. Our results apply directly to MRI of abdominal organs, with important practical implications for other architectures, as they can be applied easily to any of them. They also show the worth of variants of loss function and loss tuning, with future work needed to generalize and test in other contexts.

scholarly journals Breast Cancer Image Multi-Classification Using Random Patch Aggregation and Depth-Wise Convolution based Deep-Net Model

2021 ◽  
Vol 17 (01) ◽  
pp. 83
Author(s):  
Vandana Kate ◽  
Pragya Shukla
Keyword(s):  
Breast Cancer ◽  
Image Classification ◽  
Majority Vote ◽  
Network Models ◽  
Classification Performance ◽  
Network Architectures ◽  
Neural Network Models ◽  
Proposed Model ◽  
Optical Magnification ◽  
Histopathology Images

Adapting the profound, deep convolutional neural network models for large image classification can result in the layout of network architectures with a large number of learnable parameters and tuning of those varied parameters can considerably grow the complexity of the model. To address this problem a convolutional Deep-Net Model based on the extraction of random patches and enforcing depth-wise convolutions is proposed for training and classification of widely known benchmark Breast Cancer histopathology images. The classification result of these patches is aggregated using majority vote casting in deciding the final image classification type. It has been observed that the proposed Deep-Net model implementation results when compared with classification results of the VGG Net(16 layers) learned features, outclasses in terms of accuracy when applied to breast tumor Histopathology images. The objective of this work is to examine and comprehensively analyze the sub-class classification performance of the proposed model across all optical magnification frontiers.

Valence electron spectroscopy of inhomogeneous media

1992 ◽  
Vol 50 (2) ◽  
pp. 1150-1151
Author(s):  
A. Howie ◽  
D.W. McComb
Keyword(s):  
Specific Gravity ◽  
Loss Function ◽  
Electron Spectroscopy ◽  
Valence Electron ◽  
Peak Height ◽  
Loss Functions ◽  
Loss Peak ◽  
High Energies ◽  
Valence Electron Density ◽  
Electron Microscopist

The bulk loss function Im(-l/ε (ω)), a well established tool for the interpretation of valence loss spectra, is being progressively adapted to the wide variety of inhomogeneous samples of interest to the electron microscopist. Proportionality between n, the local valence electron density, and ε-1 (Sellmeyer's equation) has sometimes been assumed but may not be valid even in homogeneous samples. Figs. 1 and 2 show the experimentally measured bulk loss functions for three pure silicates of different specific gravity ρ - quartz (ρ = 2.66), coesite (ρ = 2.93) and a zeolite (ρ = 1.79). Clearly, despite the substantial differences in density, the shift of the prominent loss peak is very small and far less than that predicted by scaling e for quartz with Sellmeyer's equation or even the somewhat smaller shift given by the Clausius-Mossotti (CM) relation which assumes proportionality between n (or ρ in this case) and (ε - 1)/(ε + 2). Both theories overestimate the rise in the peak height for coesite and underestimate the increase at high energies.

scholarly journals Semi-Supervised PolSAR Image Classification Based on Self-Training and Superpixels

2019 ◽  
Vol 11 (16) ◽  
pp. 1933 ◽  
Author(s):  
Yangyang Li ◽  
Ruoting Xing ◽  
Licheng Jiao ◽  
Yanqiao Chen ◽  
Yingte Chai ◽  
...  
Keyword(s):  
Image Classification ◽  
State Of The Art ◽  
Sample Selection ◽  
Spatial Relations ◽  
Speckle Noise ◽  
Classification Performance ◽  
Selection Strategy ◽  
Training Set ◽  
Polarimetric Synthetic Aperture Radar ◽  
Unlabeled Sample

Polarimetric synthetic aperture radar (PolSAR) image classification is a recent technology with great practical value in the field of remote sensing. However, due to the time-consuming and labor-intensive data collection, there are few labeled datasets available. Furthermore, most available state-of-the-art classification methods heavily suffer from the speckle noise. To solve these problems, in this paper, a novel semi-supervised algorithm based on self-training and superpixels is proposed. First, the Pauli-RGB image is over-segmented into superpixels to obtain a large number of homogeneous areas. Then, features that can mitigate the effects of the speckle noise are obtained using spatial weighting in the same superpixel. Next, the training set is expanded iteratively utilizing a semi-supervised unlabeled sample selection strategy that elaborately makes use of spatial relations provided by superpixels. In addition, a stacked sparse auto-encoder is self-trained using the expanded training set to obtain classification results. Experiments on two typical PolSAR datasets verified its capability of suppressing the speckle noise and showed excellent classification performance with limited labeled data.

Sign in / Sign up

Export Citation Format

Share Document