Dynamic Weighted Majority: A new ensemble method for tracking concept drift

Jeremy Z. Kolter and Marcus A. Maloof

Algorithms for tracking concept drift are important for many applications. We present a general method based on the Weighted Majority algorithm for using any on-line learner for concept drift. Dynamic Weighted Majority (DWM) maintains an ensemble of base learners, predicts using a weighted-majority vote of these "experts", and dynamically creates and deletes experts in response to changes in performance. We empirically evaluated two experimental systems based on the method using incremental naive Bayes and Incremental Tree Inducer (ITI) as experts. For the sake of comparison, we also included Blum's implementation of Weighted Majority. On the STAGGER Concepts and on the SEA Concepts, results suggest that the ensemble method learns drifting concepts almost as well as the base algorithms learn each concept individually. Indeed, we report the best overall results for these problems to date.

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

See also kolter.jmlr.07.

Paper available in PostScript (gzipped) and PDF.

Source code available in Java

Kolter, J. Z., & Maloof, M. A. (2003). Dynamic Weighted Majority: A new ensemble method for tracking concept drift. Proceedings of the Third International IEEE Conference on Data Mining, 123–130. Los Alamitos, CA: IEEE Press.

@inproceedings{kolter.icdm.03,
  author = "Kolter, J. Z. and Maloof, M. A.",
  title = "Dynamic Weighted Majority: A new ensemble method for tracking
    concept drift",
  booktitle = "{Proceedings of the Third IEEE International Conference
    on Data Mining}",
  year = 2003,
  pages = "123--130",
  publisher = "IEEE Press",
  address = "Los Alamitos, CA",
  annote = {
  }}