Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation

Madan Ravi Ganesh, Jason J. Corso

Research output: Contribution to conferencePaperpeer-review

1 Scopus citations

Abstract

Like humans, deep networks have been shown to learn better when samples are organized and introduced in a meaningful order or curriculum [37]. Conventional curriculum learning schemes introduce samples in their order of difficulty. This forces models to begin learning from a subset of the available data while adding the external overhead of evaluating the difficulty of samples. In this work, we propose Learning with Incremental Labels and Adaptive Compensation (LILAC), a two-phase method that incrementally increases the number of unique output labels rather than the difficulty of samples while consistently using the entire dataset throughout training. In the first phase, Incremental Label Introduction, we partition data into mutually exclusive subsets, one that contains a subset of the ground-truth labels and another that contains the remaining data attached to a pseudo-label. Throughout the training process, we recursively reveal unseen ground-truth labels in fixed increments until all the labels are known to the model. In the second phase, Adaptive Compensation, we optimize the loss function using altered target vectors for previously misclassified samples. The target vectors of such samples are modified to a smoother distribution to help models learn better. On evaluating across three standard image benchmarks, CIFAR-10, CIFAR-100, and STL-10, we show that LILAC outperforms all comparable baselines. Further, we detail the importance of pacing the introduction of new labels to a model as well as the impact of using a smooth target vector.

Original languageEnglish
StatePublished - 2020
Event31st British Machine Vision Conference, BMVC 2020 - Virtual, Online
Duration: 7 Sep 202010 Sep 2020

Conference

Conference31st British Machine Vision Conference, BMVC 2020
CityVirtual, Online
Period7/09/2010/09/20

Fingerprint

Dive into the research topics of 'Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation'. Together they form a unique fingerprint.

Cite this