Group Maximum Differentiation Competition : Model Comparison with Few Samples

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

37 Scopus Citations
View graph of relations


  • Zhengfang Duanmu
  • Zhou Wang
  • Qingbo Wu
  • Wentao Liu
  • Hongwei Yong
  • Hongliang Li
  • Lei Zhang


Original languageEnglish
Pages (from-to)851-864
Journal / PublicationIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number4
Online published27 Dec 2018
Publication statusPublished - Apr 2020
Externally publishedYes


In many science and engineering fields that require computational models to predict certain physical quantities, we are often faced with the selection of the best model under the constraint that only a small sample set can be physically measured. One such example is the prediction of human perception of visual quality, where sample images live in a high dimensional space with enormous content variations. We propose a new methodology for model comparison named group maximum differentiation (gMAD) competition. Given multiple computational models, gMAD maximizes the chances of falsifying a "defender" model using the rest models as "attackers". It exploits the sample space to find sample pairs that maximally differentiate the attackers while holding the defender fixed. Based on the results of the attacking-defending game, we introduce two measures, aggressiveness and resistance, to summarize the performance of each model at attacking other models and defending attacks from other models, respectively. We demonstrate the gMAD competition using three examples—image quality, image aesthetics, and streaming video quality-of-experience. Although these examples focus on visually discriminable quantities, the gMAD methodology can be extended to many other fields, and is especially useful when the sample space is large, the physical measurement is expensive and the cost of computational prediction is low.

Research Area(s)

  • gMAD competition, image aesthetics, image quality, Model comparison, streaming video quality-of-experience

Citation Format(s)

Group Maximum Differentiation Competition: Model Comparison with Few Samples. / Ma, Kede; Duanmu, Zhengfang; Wang, Zhou et al.
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, No. 4, 04.2020, p. 851-864.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review