Reference: Pfleger, K. Learning Predictive Compositional Hierarchies. Knowledge Systems Laboratory, June, 2001.
Abstract: This paper explores the vital but overlooked problem of learning compositional hierarchies in predictive models and presents a new sequential learning paradigm in which to study such models. Hierarchical compositional structure, like taxonomic structure, is a critical representation tool for Artificial Intelligence. Prominent existing work with hand-built systems demonstrates the potential of predictive models based on compositional hierarchies for making inferences that smoothly integrate bottom-up and top-down influences and for enabling the processing of representations spanning multiple levels of spatial or temporal resolution. Additionally, like taxonomic hierarchies, compositional hierarchies can be learned purely from primitive data in a general, unsupervised fashion and subsequently used to make predictions about unseen data. However, unlike taxonomies, for which numerous foundational learning algorithms exist, there has not been analogous foundational work on learning predictive compositional hierarchies. The core aim of learning such models is to identify in a bottom-up fashion frequently occurring repeated patterns, enabling the future discovery of even larger patterns. This process holds the potential to scale up automatically from fine-grained, low-level data to coarser, high-level representations, bridging a gap that has proved to be one of the biggest stumbling blocks on the way to creating significantly more complex and intelligent autonomous agents.
Notes: A shorter, preliminary version appeared in the proceedings of the AAAI workshop on New Research Problems for Machine Learning, August, 2000.
Full paper available as ps.