Active Fine-Tuning from gMAD Examples Improves Blind Image Quality Assessment

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

12 Scopus Citations
View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)4577-4590
Number of pages14
Journal / PublicationIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume44
Issue number9
Online published8 Apr 2021
Publication statusPublished - Sept 2022

Abstract

The research in image quality assessment (IQA) has a long history, and significant progress has been made by leveraging recent advances in deep neural networks (DNNs). Despite high correlation numbers on existing IQA datasets, DNN-based models may be easily falsified in the group maximum differentiation (gMAD) competition. Here we show that gMAD examples can be used to improve blind IQA (BIQA) methods. Specifically, we first pre-train a DNN-based BIQA model using multiple noisy annotators, and fine-tune it on multiple synthetically distorted images, resulting in a “top-performing” baseline model. We then seek pairs of images by comparing the baseline model with a set of full-reference IQA methods in gMAD. The spotted gMAD examples are most likely to reveal the weaknesses of the baseline, and suggest potential ways for refinement. We query human quality annotations for the selected images in a well-controlled laboratory environment, and further fine-tune the baseline on the combination of human-rated images from gMAD and existing databases. This process may be iterated, enabling active fine-tuning from gMAD examples for BIQA. We demonstrate the feasibility of our active learning scheme on a large-scale unlabeled image set, and show that the fine-tuned quality model achieves improved generalizability in gMAD, without destroying performance on previously seen databases.

Research Area(s)

  • active learning, Adaptation models, Blind image quality assessment, Computational modeling, Databases, deep neural networks, gMAD competition, Image quality, Predictive models, subjective quality assessment, Task analysis, Training