Latent Variable Models for Sequential Data: Optimization, Representation, and Applications

Abstract

This thesis explores a spectrum of methodologies for uncovering patterns in complex, high-dimensional, and sequential data, bridging traditional statistical techniques with modern deep learning approaches. The research begins with the introduction of PLS-Lasso, a novel regression method that employs convex optimization to integrate dimensionality reduction directly into the regression framework. As a conventional statistical approach, PLS-Lasso operates as a static model—a sophisticated variant of Lasso—demonstrating its utility in applications such as financial index tracking, where it achieves higher sparsity and lower tracking error compared to standard Lasso.

Transitioning to the challenges of sequential data, the thesis then delves into deep probabilistic models designed to explicitly capture temporal dynamics. This shift is marked by the development of Deep Dynamic Probabilistic Canonical Correlation Analysis (D²PCCA), which harnesses neural networks to model nonlinear latent dynamics in time series, with its effectiveness validated on financial datasets. Building on D²PCCA, InfoDPCCA extends this framework by incorporating an information-theoretic objective, improving the extraction of shared latent structures between paired sequences and demonstrating superior performance on synthetic and medical fMRI datasets.

The thesis further contributes to sequential modeling by proposing a hierarchical taxonomy for Deep State Space Models (DSSMs), classifying existing models based on conditional independence and Markov properties. It introduces the Autoregressive State-Space Model (ArSSM) as a competitive approach for applications such as speech and music modeling.

Together, these contributions bridge traditional statistical methods and advanced deep learning techniques, offering innovative tools for analyzing complex data across domains including finance, healthcare, and signal processing.

Date of Award	20 Aug 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Yining DONG (Supervisor) & S Joe QIN (Co-supervisor)