Agglomerative Multivariate Information Bottleneck

N. Slonim, N. Friedman, and N. Tishby

Neural Information Processing Systems (NIPS 01), 2001.



The Information bottleneck method is an unsupervised model independent data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. In a recent paper, we introduced a general principled framework for multivariate extensions of the information bottleneck method that allows us to consider multiple systems of data partitions that are inter-related. In this paper, we present a new family of simple agglomerative algorithms to construct such systems of inter-related clusters. We analyze the behavior of these algorithms and apply them to several real-life datasets.