Dynamic Weighted Majority: A new ensemble method for tracking concept drift

Jeremy Z. Kolter and Marcus A. Maloof

Algorithms for tracking concept drift are important for many applications. We present a general method based on the Weighted Majority algorithm for using any on-line learner for concept drift. Dynamic Weighted Majority (DWM) maintains an ensemble of base learners, predicts using a weighted-majority vote of these "experts", and dynamically creates and deletes experts in response to changes in performance. We empirically evaluated two experimental systems based on the method using incremental naive Bayes and Incremental Tree Inducer (ITI) as experts. For the sake of comparison, we also included Blum's implementation of Weighted Majority. On the STAGGER Concepts and on the SEA Concepts, results suggest that the ensemble method learns drifting concepts almost as well as the base algorithms learn each concept individually. Indeed, we report the best overall results for these problems to date.

See Kolter & Maloof 2003b.

Paper available in PDF. 

@techreport{kolter.tr.03,
  author = "Kolter, J. Z. and Maloof, M. A.",
  title = "Dynamic Weighted Majority: A new ensemble method for tracking
    concept drift",
  type = "Technical Report",
  number = "CSTR-20030610-3",
  month = "June",
  year = 2003,
  institution = "Department of Computer Science, Georgetown University",
  address = "Washington, DC",
  note = "http://daruma.georgetown.edu/techreports/techrep\_data/cstr-20030610-3.pdf"
}