Dynamic Weighted Majority: An ensemble method for drifting concepts

J. Zico Kolter and Marcus A. Maloof

We present an ensemble method for concept drift that dynamically creates and removes weighted experts in response to changes in performance. The method, dynamic weighted majority (DWM), uses four mechanisms to cope with concept drift: It trains online learners of the ensemble, it weights those learners based on their performance, it removes them, also based on their performance, and it adds new experts based on the global performance of the ensemble. After an extensive evaluation—consisting of five experiments, eight learners, and thirty data sets that varied in type of target concept, size, presence of noise, and the like—we concluded that DWM outperformed other learners that only incrementally learn concept descriptions, that maintain and use previously encountered examples, and that employ an unweighted, fixed-size ensemble of experts.

Paper available from JMLR.

Source code available in Java

  author = "Kolter, J. Z. and Maloof, M. A.",
  title = "Dynamic Weighted Majority: An ensemble method for drifting concepts",
  journal = "Journal of Machine Learning Research",
  year = 2007,
  volume = 8,
  month = "Dec"
  pages = "2755--2790",
  annote = {