Blind late fusion in multimedia event retrieval

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

3 Scopus Citations
View graph of relations

Author(s)

  • Maaike H. T. de Boer
  • Klamer Schutte
  • Wessel Kraaij

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)203-217
Journal / PublicationInternational Journal of Multimedia Information Retrieval
Volume5
Issue number4
Publication statusPublished - 1 Nov 2016

Abstract

One of the challenges in Multimedia Event Retrieval is the integration of data from multiple modalities. A modality is defined as a single channel of sensory input, such as visual or audio. We also refer to this as data source. Previous research has shown that the integration of different data sources can improve performance compared to only using one source, but a clear insight of success factors of alternative fusion methods is still lacking. We introduce several new blind late fusion methods based on inversions and ratios of the state-of-the-art blind fusion methods and compare performance in both simulations and an international benchmark data set in multimedia event retrieval named TRECVID MED. The results show that five of the proposed methods outperform the state-of-the-art methods in a case with sufficient training examples (100 examples). The novel fusion method named JRER is not only the best method with dependent data sources, but this method is also a robust method in all simulations with sufficient training examples.

Research Area(s)

  • Integration, Late fusion, Multimedia event retrieval, Multimodal

Citation Format(s)

Blind late fusion in multimedia event retrieval. / de Boer, Maaike H. T.; Schutte, Klamer; Zhang, Hao; Lu, Yi-Jie; Ngo, Chong-Wah; Kraaij, Wessel.

In: International Journal of Multimedia Information Retrieval, Vol. 5, No. 4, 01.11.2016, p. 203-217.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review