Bagging by design (on the suboptimality of bagging)

Periklis A. Papakonstantinou, Jia Xu, Zhu Cao

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Bagging (Breiman 1996) and its variants is one of the most popular methods in aggregating classifiers and regressors. Originally, its analysis assumed that the bootstraps are built from an unlimited, independent source of samples, therefore we call this form of bagging ideal-bagging. However in the real world, base predictors are trained on data subsampled from a limited number of training samples and thus they behave very differently. We analyze the effect of intersections between bootstraps, obtained by subsampling, to train different base predictors. Most importantly, we provide an alternative subsampling method called design-bagging based on a new construction of combinatorial designs, and prove it universally better than bagging. Methodologically, we succeed at this level of generality because we compare the prediction accuracy of bagging and design-bagging relative to the accuracy ideal-bagging. This finds potential applications in more involved bagging-based methods. Our analytical results are backed up by experiments on classification and regression settings.

Original languageEnglish
Title of host publicationProceedings of the National Conference on Artificial Intelligence
Pages2041-2047
Number of pages7
ISBN (Electronic)9781577356790
StatePublished - 2014
Event28th AAAI Conference on Artificial Intelligence, AAAI 2014, 26th Innovative Applications of Artificial Intelligence Conference, IAAI 2014 and the 5th Symposium on Educational Advances in Artificial Intelligence, EAAI 2014 - Quebec City, Canada
Duration: 27 Jul 201431 Jul 2014

Publication series

NameProceedings of the National Conference on Artificial Intelligence
Volume3

Conference

Conference28th AAAI Conference on Artificial Intelligence, AAAI 2014, 26th Innovative Applications of Artificial Intelligence Conference, IAAI 2014 and the 5th Symposium on Educational Advances in Artificial Intelligence, EAAI 2014
Country/TerritoryCanada
CityQuebec City
Period27/07/1431/07/14

Fingerprint

Dive into the research topics of 'Bagging by design (on the suboptimality of bagging)'. Together they form a unique fingerprint.

Cite this