Constructing Dynamic Topic Models Based on Variational Antoencoder and Factor Graph

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

4 Scopus Citations
View graph of relations

Author(s)

  • Zhinan Gou
  • Lixin Han
  • Ling Sun
  • Jun Zhu
  • Hong Yan

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)53102-53111
Journal / PublicationIEEE Access
Volume6
Online published13 Sep 2018
Publication statusOnline published - 13 Sep 2018

Abstract

Topic models are widely used in various fields of machine learning and statistics. Among them, the dynamic topic model (DTM) is the most popular time-series topic model for the dynamic repre-sentations of text corpora. A major challenge is that the posterior distribution of DTM requires a complex reasoning process with the high cost of computing time in modeling, and even a tiny change of model re-quires restructuring. For these reasons, the variability and generality of DTM is so poor that DTM is diffi-cult to be carried out. In this paper, we introduce a new method for constructing DTM based on variational antoencoder and factor graphs. This model uses re-parameterization of the variational lower bound to gen-erate a lower bound estimator which is optimized by standard stochastic gradient descent method directly. At the same time, the optimization process is simplified by integrating the dynamic factor graph in the state space to achieve a better model. The experimental dataset uses a journal paper corpus that mainly focuses on natural language processing and spans twenty-five years (1984-2009) from DBLP. Experiment results indicate that the proposed method is effective and feasible by comparing several state-of-the-art baselines.

Research Area(s)

  • 2-layer neural network, Bayes methods, Cognition, Computational modeling, dynamic topic model, factor graph, Heuristic algorithms, Inference algorithms, Optimization, Probabilistic logic, variational antoencoder

Citation Format(s)

Constructing Dynamic Topic Models Based on Variational Antoencoder and Factor Graph. / Gou, Zhinan; Han, Lixin; Sun, Ling; Zhu, Jun; Yan, Hong.

In: IEEE Access, Vol. 6, 13.09.2018, p. 53102-53111.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review