Is the Reign of Interactive Search Eternal? Findings from the Video Browser Showdown 2020

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

8 Scopus Citations
View graph of relations

Author(s)

  • Jakub LOKOČ
  • Patrik VESELÝ
  • František MEJZLÍK
  • Gregor KOVALČÍK
  • Tomáš SOUČEK
  • Luca ROSSETTO
  • Klaus SCHOEFFMANN
  • Werner BAILER
  • Cathal GURRIN
  • Loris SAUTER
  • Jaeyub SONG
  • Stefanos VROCHIDIS
  • Björn Jþór JÓNSSON

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number91
Journal / PublicationACM Transactions on Multimedia Computing, Communications and Applications
Volume17
Issue number3
Online published22 Jul 2021
Publication statusPublished - Aug 2021

Abstract

Comprehensive and fair performance evaluation of information retrieval systems represents an essential task for the current information age. Whereas Cranfield-based evaluations with benchmark datasets support development of retrieval models, significant evaluation efforts are required also for user-oriented systems that try to boost performance with an interactive search approach. This article presents findings from the 9th Video Browser Showdown, a competition that focuses on a legitimate comparison of interactive search systems designed for challenging known-item search tasks over a large video collection. During previous installments of the competition, the interactive nature of participating systems was a key feature to satisfy known-item search needs, and this article continues to support this hypothesis. Despite the fact that top-performing systems integrate the most recent deep learning models into their retrieval process, interactive searching remains a necessary component of successful strategies for known-item search tasks. Alongside the description of competition settings, evaluated tasks, participating teams, and overall results, this article presents a detailed analysis of query logs collected by the top three performing systems, SOMHunter, VIRET, and vitrivr. The analysis provides a quantitative insight to the observed performance of the systems and constitutes a new baseline methodology for future events. The results reveal that the top two systems mostly relied on temporal queries before a correct frame was identified. An interaction log analysis complements the result log findings and points to the importance of result set and video browsing approaches. Finally, various outlooks are discussed in order to improve the Video Browser Showdown challenge in the future.

Research Area(s)

  • deep learning, interactive search evaluation, Interactive video retrieval

Bibliographic Note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Citation Format(s)

Is the Reign of Interactive Search Eternal? Findings from the Video Browser Showdown 2020. / LOKOČ, Jakub; VESELÝ, Patrik; MEJZLÍK, František; KOVALČÍK, Gregor; SOUČEK, Tomáš; ROSSETTO, Luca; SCHOEFFMANN, Klaus; BAILER, Werner; GURRIN, Cathal; SAUTER, Loris; SONG, Jaeyub; VROCHIDIS, Stefanos; WU, Jiaxin; JÓNSSON, Björn Jþór.

In: ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 17, No. 3, 91, 08.2021.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review