Abstract

Ensemble classification techniques combine the predictions of multiple machine learning models (such as decision trees or artificial neural networks) into a single prediction. Popular methods such as bagging and boosting produce models that are often significantly more accurate on unseen data. This accuracy depends on both the strength and the diversity of the component classifiers.
The first half of this talk presents a novel ensemble learning method that uses feature selection to promote diversity among the classifiers. We use a two-level evolutionary search strategy, in which individual classifiers compete to correctly predict training points, while multiple ensembles compete based on estimated accuracy. Computational results compare favorably to bagging and boosting.
Methods that require any form of data resampling are inappropriate for data mining applications with large-scale or streaming input data. The second half of the talk presents an ensemble technique that combines classifiers built on consecutive chunks of data. This method builds ensembles that are comparable in accuracy to a single classifier, and does so in a single pass of the data using bounded memory. It is also robust against concept drift.