Empirical study on label smoothing in neural networks

Date issued

2018

Journal Title

Journal ISSN

Volume Title

Publisher

Václav Skala - UNION Agency

Abstract

Neural networks are now day routinely employed in the classification of sets of objects, which consists in predicting the class label of an object. The softmax function is a popular choice of the output function in neural networks. It is a probability distribution of the class labels and the label with maximum probability represents the prediction of the neural network, given the object being classified. The softmax function is also used to compute the loss function, which evaluates the error made by the network in the classification task. In this paper we consider a simple modification to the loss function, called label smoothing. We experimented this modification by training a neural network using 12 data sets, all containing a total of about 1:5 106 images. We show that this modification allow a neural network to achieve a better accuracy in the classification task.

Description

Subject(s)

neuronové sítě, vyhlazování štítků, regulace, softmax, viuální domény

Citation

WSCG '2018: short communications proceedings: The 26th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2016 in co-operation with EUROGRAPHICS: University of West Bohemia, Plzen, Czech Republic May 28 - June 1 2018, p. 200-205.
OPEN License Selector