A Task Category Space for User-Centric Comparative Multimedia Search Evaluations

Jakub Lokoč; Werner Bailer; Kai Uwe Barthel; Cathal Gurrin; Silvan Heller; Björn þór Jónsson; Ladislav Peška; Luca Rossetto; Klaus Schoeffmann; Lucia Vadicamo; Stefanos Vrochidis; Jiaxin Wu

doi:10.1007/978-3-030-98358-1_16

A Task Category Space for User-Centric Comparative Multimedia Search Evaluations

Jakub Lokoč^*, Werner Bailer, Kai Uwe Barthel, Cathal Gurrin, Silvan Heller, Björn þór Jónsson, Ladislav Peška, Luca Rossetto, Klaus Schoeffmann, Lucia Vadicamo, Stefanos Vrochidis, Jiaxin Wu

^*Corresponding author for this work

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

17 Citations (Scopus)

Abstract

In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.

Original language	English
Title of host publication	MultiMedia Modeling
Subtitle of host publication	28th International Conference, MMM 2022, Proceedings, Part I
Editors	Björn Þór Jónsson, Cathal Gurrin, Minh-Triet Tran, Duc-Tien Dang-Nguyen, Anita Min-Chun Hu, Binh Huynh Thi Thanh, Benoit Huet
Publisher	Springer
Pages	193-204
ISBN (Electronic)	978-3-030-98358-1
ISBN (Print)	978-3-030-98357-4
DOIs	https://doi.org/10.1007/978-3-030-98358-1_16
Publication status	Published - Jun 2022
Event	28th International Conference on MultiMedia Modeling, MMM 2022 - Phu Quoc, Viet Nam Duration: 6 Jun 2022 → 10 Jun 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13141 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	28th International Conference on MultiMedia Modeling, MMM 2022
Place	Viet Nam
City	Phu Quoc
Period	6/06/22 → 10/06/22

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

Evaluation design
Multimedia retrieval
Task taxonomy

Access Output

10.1007/978-3-030-98358-1_16

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Lokoč, J., Bailer, W., Barthel, K. U., Gurrin, C., Heller, S., Jónsson, B. Þ., Peška, L., Rossetto, L., Schoeffmann, K., Vadicamo, L., Vrochidis, S., & Wu, J. (2022). A Task Category Space for User-Centric Comparative Multimedia Search Evaluations. In B. Þ. Jónsson, C. Gurrin, M.-T. Tran, D.-T. Dang-Nguyen, A. M.-C. Hu, B. H. Thi Thanh, & B. Huet (Eds.), MultiMedia Modeling : 28th International Conference, MMM 2022, Proceedings, Part I (pp. 193-204). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13141 LNCS). Springer . https://doi.org/10.1007/978-3-030-98358-1_16

Lokoč, Jakub ; Bailer, Werner ; Barthel, Kai Uwe et al. / A Task Category Space for User-Centric Comparative Multimedia Search Evaluations. MultiMedia Modeling : 28th International Conference, MMM 2022, Proceedings, Part I. editor / Björn Þór Jónsson ; Cathal Gurrin ; Minh-Triet Tran ; Duc-Tien Dang-Nguyen ; Anita Min-Chun Hu ; Binh Huynh Thi Thanh ; Benoit Huet. Springer , 2022. pp. 193-204 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b014fd1b23cb4026b97a3086b5a40b51,

title = "A Task Category Space for User-Centric Comparative Multimedia Search Evaluations",

abstract = "In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.",

keywords = "Evaluation design, Multimedia retrieval, Task taxonomy",

author = "Jakub Loko{\v c} and Werner Bailer and Barthel, {Kai Uwe} and Cathal Gurrin and Silvan Heller and J{\'o}nsson, {Bj{\"o}rn {\th}{\'o}r} and Ladislav Pe{\v s}ka and Luca Rossetto and Klaus Schoeffmann and Lucia Vadicamo and Stefanos Vrochidis and Jiaxin Wu",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s). ; 28th International Conference on MultiMedia Modeling, MMM 2022 ; Conference date: 06-06-2022 Through 10-06-2022",

year = "2022",

month = jun,

doi = "10.1007/978-3-030-98358-1_16",

language = "English",

isbn = "978-3-030-98357-4",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer ",

pages = "193--204",

editor = "J{\'o}nsson, {Bj{\"o}rn {\TH}{\'o}r } and Cathal Gurrin and Tran, {Minh-Triet } and Dang-Nguyen, {Duc-Tien } and Hu, {Anita Min-Chun } and {Thi Thanh}, {Binh Huynh } and Huet, {Benoit }",

booktitle = "MultiMedia Modeling",

}

Lokoč, J, Bailer, W, Barthel, KU, Gurrin, C, Heller, S, Jónsson, BÞ, Peška, L, Rossetto, L, Schoeffmann, K, Vadicamo, L, Vrochidis, S & Wu, J 2022, A Task Category Space for User-Centric Comparative Multimedia Search Evaluations. in BÞ Jónsson, C Gurrin, M-T Tran, D-T Dang-Nguyen, AM-C Hu, BH Thi Thanh & B Huet (eds), MultiMedia Modeling : 28th International Conference, MMM 2022, Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13141 LNCS, Springer , pp. 193-204, 28th International Conference on MultiMedia Modeling, MMM 2022, Phu Quoc, Viet Nam, 6/06/22. https://doi.org/10.1007/978-3-030-98358-1_16

A Task Category Space for User-Centric Comparative Multimedia Search Evaluations. / Lokoč, Jakub; Bailer, Werner; Barthel, Kai Uwe et al.
MultiMedia Modeling : 28th International Conference, MMM 2022, Proceedings, Part I. ed. / Björn Þór Jónsson; Cathal Gurrin; Minh-Triet Tran; Duc-Tien Dang-Nguyen; Anita Min-Chun Hu; Binh Huynh Thi Thanh; Benoit Huet. Springer , 2022. p. 193-204 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13141 LNCS).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - A Task Category Space for User-Centric Comparative Multimedia Search Evaluations

AU - Lokoč, Jakub

AU - Bailer, Werner

AU - Barthel, Kai Uwe

AU - Gurrin, Cathal

AU - Heller, Silvan

AU - Jónsson, Björn þór

AU - Peška, Ladislav

AU - Rossetto, Luca

AU - Schoeffmann, Klaus

AU - Vadicamo, Lucia

AU - Vrochidis, Stefanos

AU - Wu, Jiaxin

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2022/6

Y1 - 2022/6

N2 - In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.

AB - In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.

KW - Evaluation design

KW - Multimedia retrieval

KW - Task taxonomy

UR - http://www.scopus.com/inward/record.url?scp=85127018549&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85127018549&origin=recordpage

U2 - 10.1007/978-3-030-98358-1_16

DO - 10.1007/978-3-030-98358-1_16

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-3-030-98357-4

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 193

EP - 204

BT - MultiMedia Modeling

A2 - Jónsson, Björn Þór

A2 - Gurrin, Cathal

A2 - Tran, Minh-Triet

A2 - Dang-Nguyen, Duc-Tien

A2 - Hu, Anita Min-Chun

A2 - Thi Thanh, Binh Huynh

A2 - Huet, Benoit

PB - Springer

T2 - 28th International Conference on MultiMedia Modeling, MMM 2022

Y2 - 6 June 2022 through 10 June 2022

ER -

Lokoč J, Bailer W, Barthel KU, Gurrin C, Heller S, Jónsson BÞ et al. A Task Category Space for User-Centric Comparative Multimedia Search Evaluations. In Jónsson BÞ, Gurrin C, Tran MT, Dang-Nguyen DT, Hu AMC, Thi Thanh BH, Huet B, editors, MultiMedia Modeling : 28th International Conference, MMM 2022, Proceedings, Part I. Springer . 2022. p. 193-204. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2022 Mar 15. doi: 10.1007/978-3-030-98358-1_16

A Task Category Space for User-Centric Comparative Multimedia Search Evaluations

Abstract

Publication series

Conference

Bibliographical note

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this