Progressive partial memory learning

Marcus A. Maloof

A learning methodology called Progressive Partial Memory Learning (PPML) is presented. PPML takes a partial memory approach to progressive inductive learning problems in which training examples are distributed over time. The partial memory approach retains and uses representative examples and induced concept descriptions for future learning. Representative examples are those training examples that maximally expand and constrain concept descriptions in the representation space. Mechanisms such as the selection of representative examples, forgetting, and aging allow the system to efficiently learn concepts over time and to track changing concepts through a representation space. Key components of the methodology are implemented in an experimental system called AQ-Partial Memory (AQ-PM), which is based on the AQ15c inductive learning system. AQ-PM is experimentally validated against a baseline AQ15c learning algorithm using both synthetic and real-world data sets. Synthetic data sets include problems in which concepts change or drift in the representation space. Real-world problems include applications to dynamic knowledge bases (i.e., computer intrusion detection), intelligent agents (i.e., email sorting), and computer vision (i.e., blasting cap detection in X-ray images). Results demonstrate that the methodology is able to perform well on a variety of problems. Specifically, the method considerably improves memory requirements and learning time with slight decreases in predictive accuracy when compared to the baseline learner.

Furthermore, the method is also able to track concept drift. Results are presented for the STAGGER concepts in which AQ-PM achieves predictive accuracies comparable to the FLORA systems, but requires much less memory. Comparisons are made to the AQ11 and GEM incremental learning systems using the blasting cap detection and computer intrusion detection problems in which they learned more predictive, but more complex concept descriptions than AQ-PM. Partial memory learning is also conducted using the incremental learning algorithms of AQ11 and GEM for the blasting cap detection and computer intrusion detection problems. Results for the partial memory versions of these learners, when compared to the unmodified versions, show slight decreases in predictive accuracy and concept complexity with considerable decreases in learning time.

Paper available in PostScript (gzipped) and PDF.

Slides from the defense available in PostScript (gzipped) and PDF. 

@phdthesis{maloof.phd.96,
  author = "Maloof, M.A.",
  title = "Progressive partial memory learning",
  year = 1996,
  school = "School of Information Technology and Engineering, George
    Mason University",
  address = "Fairfax, VA"
}