Exploiting Graph Convolutional Networks for Light Field Image Compression
利用圖卷積網絡進行光場圖像壓縮
Student thesis: Master's Thesis
Author(s)
Related Research Unit(s)
Detail(s)
Awarding Institution | |
---|---|
Supervisors/Advisors |
|
Award date | 23 Feb 2024 |
Link(s)
Permanent Link | https://scholars.cityu.edu.hk/en/theses/theses(ccc1f0b2-0882-4e84-8b9c-afed3f971247).html |
---|---|
Other link(s) | Links |
Abstract
The application of light field technology has gained traction due to its unparalleled ability to capture comprehensive scene details. Nonetheless, the sizable volume of data generated by this method presents formidable storage and transmission challenges.
To address this issue, we present several novel approaches via GCN. This thesis first proposes a novel Light Field (LF) compression method via a variational graph auto-encoder (VGAE), aiming to exploit better the structural information of edges and vertices of the graph LF image. More specifically, the graph adjacency matrix and feature matrix are derived from the original graph data in the encoder. Subsequently, a graph convolutional network (GCN) is utilized to determine a multi-dimensional Gaussian distribution, from which the latent representation can be derived by sampling. Finally, the graph LF image can be reconstructed by the inner product of the latent variable in the decoder. The distinct characteristics of the proposed scheme lie in that VGAE encoder applies GCN as a function, which can better alleviate the loss of compression. Moreover, the divergence between the original and the reconstructed signals is evaluated using KL divergence to ensure that the estimator is unbiased, leading to better adaptability.
Then, Graph sample and aggregate algorithm (GraphSAGE), a potent graph neural network model that learns node embeddings on graphs is used to improve. This method represents each view of the light field as a node in a graph and uses GraphSAGE to acquire a compressed set of node embeddings that effectively capture the light field. In order to assess the effectiveness of our approach, we benchmark it against current leading light field compression techniques such as HEVC, graph-based learning methodologies, and prior research. The outcomes of our experiments show that our suggested strategies offer comparable compression efficiency to these state-of-art methods.
To address this issue, we present several novel approaches via GCN. This thesis first proposes a novel Light Field (LF) compression method via a variational graph auto-encoder (VGAE), aiming to exploit better the structural information of edges and vertices of the graph LF image. More specifically, the graph adjacency matrix and feature matrix are derived from the original graph data in the encoder. Subsequently, a graph convolutional network (GCN) is utilized to determine a multi-dimensional Gaussian distribution, from which the latent representation can be derived by sampling. Finally, the graph LF image can be reconstructed by the inner product of the latent variable in the decoder. The distinct characteristics of the proposed scheme lie in that VGAE encoder applies GCN as a function, which can better alleviate the loss of compression. Moreover, the divergence between the original and the reconstructed signals is evaluated using KL divergence to ensure that the estimator is unbiased, leading to better adaptability.
Then, Graph sample and aggregate algorithm (GraphSAGE), a potent graph neural network model that learns node embeddings on graphs is used to improve. This method represents each view of the light field as a node in a graph and uses GraphSAGE to acquire a compressed set of node embeddings that effectively capture the light field. In order to assess the effectiveness of our approach, we benchmark it against current leading light field compression techniques such as HEVC, graph-based learning methodologies, and prior research. The outcomes of our experiments show that our suggested strategies offer comparable compression efficiency to these state-of-art methods.