Full-Reference Image Quality Assessment: Addressing Content Misalignment Issue by Comparing Order Statistics of Deep Features

Xingran Liao, Xuekai Wei*, Mingliang Zhou, Sam Kwong*

*Corresponding author for this work

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

11 Citations (Scopus)

Abstract

This letter aims to develop advanced full-reference image quality assessment (FR-IQA) models to evaluate content-misaligned image pairs, which are commonly encountered in image reconstruction tasks and texture synthesis tasks. Traditional FR-IQA models tend to be overly sensitive to content shifting and misalignment, thus deviating from subjective evaluations. Herein, we propose a deep order statistical similarity (DOSS) FR-IQA model that compares the order statistics of deep features to address this issue. In DOSS, the reference and distorted images are projected into the deep feature space, and the sorted deep network features are compared with the cosine similarity index to output the final perceptual quality scores. With such a simple design baseline, DOSS offers several advantages. First, it mimics the behavior of the human visual system (HVS) in terms of evaluating content-misaligned image pairs, thereby tolerating slight image shifts and deformations. Second, DOSS possesses an advanced texture perception capability, producing superior quality assessment results on images generated by various texture synthesis algorithms; this indicates that DOSS can be used to select visually appealing texture synthesis results. Finally, experimental results demonstrate that DOSS can also obtain competitive quality assessment results on standard IQA datasets, suggesting that deep feature order statistics can serve as generic features for both content-aligned and content-misaligned IQA. The code for this method is publicly available at https://github.com/Buka-Xing/DOSS. © 2023 IEEE.
Original languageEnglish
Pages (from-to)305-315
Number of pages11
JournalIEEE Transactions on Broadcasting
Volume70
Issue number1
Online published28 Jul 2023
DOIs
Publication statusPublished - Mar 2024

Research Keywords

  • content misalignment
  • cosine similarity
  • deep neural network
  • Feature extraction
  • full-reference image quality assessment
  • Image coding
  • Image quality
  • Image quality assessment
  • Indexes
  • order statistics
  • Quality assessment
  • Standards
  • Task analysis

Fingerprint

Dive into the research topics of 'Full-Reference Image Quality Assessment: Addressing Content Misalignment Issue by Comparing Order Statistics of Deep Features'. Together they form a unique fingerprint.

Cite this