Agglomerative Info-Clustering : Maximizing Normalized Total Correlation

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)2001-2011
Journal / PublicationIEEE Transactions on Information Theory
Volume67
Issue number3
Online published25 Nov 2020
Publication statusPublished - Mar 2021

Abstract

We show that, under the info-clustering framework, correlated random variables can be clustered in an agglomerative manner. While the existing divisive approach successively segregates the random variables into subsets with increasing multivariate mutual information, our agglomerative approach successively merges subsets of random variables sharing a large amount of normalized total correlation. We show that both approaches result in the same hierarchy of clusters, but the agglomerative approach is an order of magnitude faster than the divisive one. The uniqueness of the hierarchy produced by the two approaches is due to a fundamental connection that we uncover between the well-known total correlation and the recently proposed measure of multivariate mutual information. We implement the new algorithm and provide a data structure for efficient storage and retrieval of the hierarchical clustering solution.

Research Area(s)

  • agglomerative clustering, Clustering algorithms, Correlation, Entropy, Lattices, minimum norm base, multivariate mutual information, Mutual information, principal sequence, principal sequence of partitions, Random variables, Turning

Citation Format(s)

Agglomerative Info-Clustering : Maximizing Normalized Total Correlation. / Chan, Chung; Al-Bashabsheh, Ali; Zhou, Qiaoqiao.

In: IEEE Transactions on Information Theory, Vol. 67, No. 3, 03.2021, p. 2001-2011.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review