Full-Reference Image Quality Assessment : Addressing Content Misalignment Issue by Comparing Order Statistics of Deep Features

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

1 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)305-315
Number of pages11
Journal / PublicationIEEE Transactions on Broadcasting
Volume70
Issue number1
Online published28 Jul 2023
Publication statusPublished - Mar 2024

Abstract

This letter aims to develop advanced full-reference image quality assessment (FR-IQA) models to evaluate content-misaligned image pairs, which are commonly encountered in image reconstruction tasks and texture synthesis tasks. Traditional FR-IQA models tend to be overly sensitive to content shifting and misalignment, thus deviating from subjective evaluations. Herein, we propose a deep order statistical similarity (DOSS) FR-IQA model that compares the order statistics of deep features to address this issue. In DOSS, the reference and distorted images are projected into the deep feature space, and the sorted deep network features are compared with the cosine similarity index to output the final perceptual quality scores. With such a simple design baseline, DOSS offers several advantages. First, it mimics the behavior of the human visual system (HVS) in terms of evaluating content-misaligned image pairs, thereby tolerating slight image shifts and deformations. Second, DOSS possesses an advanced texture perception capability, producing superior quality assessment results on images generated by various texture synthesis algorithms; this indicates that DOSS can be used to select visually appealing texture synthesis results. Finally, experimental results demonstrate that DOSS can also obtain competitive quality assessment results on standard IQA datasets, suggesting that deep feature order statistics can serve as generic features for both content-aligned and content-misaligned IQA. The code for this method is publicly available at https://github.com/Buka-Xing/DOSS. © 2023 IEEE.

Research Area(s)

  • content misalignment, cosine similarity, deep neural network, Feature extraction, full-reference image quality assessment, Image coding, Image quality, Image quality assessment, Indexes, order statistics, Quality assessment, Standards, Task analysis

Citation Format(s)