Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment
Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Number of pages | 10 |
Journal / Publication | IEEE Transactions on Broadcasting |
Online published | 4 Jan 2023 |
Publication status | Online published - 4 Jan 2023 |
Link(s)
Abstract
Generally, 360-degree images have absolute senses of reality and three-dimensionality, providing a wide range of immersive interactions. Due to the novel rendering and display technology of 360-degree images, they have more complex perceptual characteristics than other images. It is challenging to perform comprehensive image quality assessment (IQA) learning by simply stacking multichannel neural network architectures for pre/postprocessing, compression, and rendering tasks. To thoroughly learn the global and local features in 360-degree images, reduce the complexity of multichannel neural network models and simplify the training process, this paper proposes a joint architecture with user perception and an efficient transformer dedicated to 360-degree no-reference (NR) IQA. The input of the proposed method is a 360-degree cube map projection (CMP) image. Furthermore, the proposed 360-degree NRIQA method includes a saliency map-based non-overlapping self-attention selection module and a U-shaped transformer (U-former)-based feature extraction module to account for perceptual region importance and projection distortion. The transformer-based architecture and the weighted average technique are jointly utilized for predicting local perceptual quality. Experimental results obtained on widely used databases show that the proposed model outperforms other state-of-the-art methods in NR 360-degree image quality evaluation cases. Furthermore, a cross-database evaluation and an ablation study also demonstrate the inherent robustness and generalization ability of the proposed model.
Research Area(s)
- Image quality assessment, no-reference image quality assessment, 360-degree image, U-shaped transformer, OMNIDIRECTIONAL IMAGE, SALIENCY, CNN
Citation Format(s)
Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment. / Zhou, Mingliang; Chen, Lei; Wei, Xuekai et al.
In: IEEE Transactions on Broadcasting, 04.01.2023.Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review