Image Quality Assessment : Unifying Structure and Texture Similarity

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

147 Scopus Citations
View graph of relations


Related Research Unit(s)


Original languageEnglish
Pages (from-to)2567-2581
Journal / PublicationIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number5
Online published18 Dec 2020
Publication statusPublished - May 2022


Objective measures of image quality generally operate by making local comparisons of pixels of a "degraded" image to those of the original. Relative to human observers, these measures are overly sensitive to resampling of texture regions (e.g., replacing one patch of grass with another). Here we develop the first full-reference image quality model with explicit tolerance to texture resampling. Using a convolutional neural network, we construct an injective and differentiable function that transforms images to a multi-scale overcomplete representation. We empirically show that the spatial averages of the feature maps in this representation capture texture appearance, in that they provide a set of sufficient statistical constraints to synthesize a wide variety of texture patterns. We then describe an image quality method that combines correlation of these spatial averages ("texture similarity'') with correlation of the feature maps ("structure similarity''). The parameters of the proposed measure are jointly optimized to match human ratings of image quality, while minimizing the reported distances between subimages cropped from the same texture images. Experiments show that the optimized method explains human perceptual scores, both on conventional image quality databases and texture databases. The measure also offers competitive performance on texture classification and retrieval, and show the robustness to geometric transformations. Code is available at

Research Area(s)

  • Image quality assessment, perceptual optimization, structure similarity, texture similarity