Concept-based interactive search system

Yi-Jie Lu; Phuong Anh Nguyen; Hao Zhang; Chong-Wah Ngo

doi:10.1007/978-3-319-51814-5_42

Concept-based interactive search system

Yi-Jie Lu^*, Phuong Anh Nguyen, Hao Zhang, Chong-Wah Ngo

^*Corresponding author for this work

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

18 Citations (Scopus)

Abstract

Our successful multimedia event detection system at TRECVID 2015 showed its strength on handling complex concepts in a query. The system was based on a large number of pre-trained concept detectors for textual-to-visual relation. In this paper, we enhance the system by enabling human-in-the-loop. In order to facilitate a user to quickly find an information need, we incorporate concept screening, video reranking by highlighted concepts, relevance feedback and color sketch to refine a coarse retrieval result. The aim is to eventually come up with a system suitable for both Ad-hoc Video Search and Known-Item Search. In addition, as the increasing awareness of difficulty in distinguishing shots of very similar scenes, we also explore the automatic story annotation along the timeline of a video, so that a user can quickly grasp the story happened in the context of a target shot and reject shots with incorrect context. With the story annotation, a user can refine the search result as well by simply adding a few keywords in a special “context field” of a query.

Original language	English
Title of host publication	MultiMedia Modeling
Subtitle of host publication	23rd International Conference, MMM 2017, Proceedings
Editors	Cathal Gurrin, Björn Thór Jónsson, Laurent Amsaleg, Shin’ichi Satoh, Gylfi Thór Gudmundsson
Publisher	Springer Verlag
Pages	463-468
Volume	10133 LNCS
ISBN (Print)	9783319518138
DOIs	https://doi.org/10.1007/978-3-319-51814-5_42
Publication status	Published - 31 Dec 2016
Event	23rd International Conference on MultiMedia Modeling, MMM 2017 - Reykjavik, Iceland Duration: 4 Jan 2017 → 6 Jan 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10133 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	23rd International Conference on MultiMedia Modeling, MMM 2017
Place	Iceland
City	Reykjavik
Period	4/01/17 → 6/01/17

Research Keywords

Concept bank
Known-item search
Semantic query
Story annotation
Video reranking
Video search

Access Output

10.1007/978-3-319-51814-5_42

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Lu, Y.-J., Nguyen, P. A., Zhang, H., & Ngo, C.-W. (2016). Concept-based interactive search system. In C. Gurrin, B. T. Jónsson, L. Amsaleg, S. Satoh, & G. T. Gudmundsson (Eds.), MultiMedia Modeling: 23rd International Conference, MMM 2017, Proceedings (Vol. 10133 LNCS, pp. 463-468). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10133 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-51814-5_42

Lu, Yi-Jie ; Nguyen, Phuong Anh ; Zhang, Hao et al. / Concept-based interactive search system. MultiMedia Modeling: 23rd International Conference, MMM 2017, Proceedings. editor / Cathal Gurrin ; Björn Thór Jónsson ; Laurent Amsaleg ; Shin’ichi Satoh ; Gylfi Thór Gudmundsson. Vol. 10133 LNCS Springer Verlag, 2016. pp. 463-468 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{37f36c98554a4b36a3c29a5290ad4ab0,

title = "Concept-based interactive search system",

abstract = "Our successful multimedia event detection system at TRECVID 2015 showed its strength on handling complex concepts in a query. The system was based on a large number of pre-trained concept detectors for textual-to-visual relation. In this paper, we enhance the system by enabling human-in-the-loop. In order to facilitate a user to quickly find an information need, we incorporate concept screening, video reranking by highlighted concepts, relevance feedback and color sketch to refine a coarse retrieval result. The aim is to eventually come up with a system suitable for both Ad-hoc Video Search and Known-Item Search. In addition, as the increasing awareness of difficulty in distinguishing shots of very similar scenes, we also explore the automatic story annotation along the timeline of a video, so that a user can quickly grasp the story happened in the context of a target shot and reject shots with incorrect context. With the story annotation, a user can refine the search result as well by simply adding a few keywords in a special “context field” of a query.",

keywords = "Concept bank, Known-item search, Semantic query, Story annotation, Video reranking, Video search",

author = "Yi-Jie Lu and Nguyen, {Phuong Anh} and Hao Zhang and Chong-Wah Ngo",

year = "2016",

month = dec,

day = "31",

doi = "10.1007/978-3-319-51814-5_42",

language = "English",

isbn = "9783319518138",

volume = "10133 LNCS",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "463--468",

editor = "Cathal Gurrin and J{\'o}nsson, {Bj{\"o}rn Th{\'o}r} and Laurent Amsaleg and Shin{\textquoteright}ichi Satoh and Gudmundsson, {Gylfi Th{\'o}r}",

booktitle = "MultiMedia Modeling",

address = "Germany",

note = "23rd International Conference on MultiMedia Modeling, MMM 2017 ; Conference date: 04-01-2017 Through 06-01-2017",

}

Lu, Y-J , Nguyen, PA , Zhang, H & Ngo, C-W 2016, Concept-based interactive search system. in C Gurrin, BT Jónsson, L Amsaleg, S Satoh & GT Gudmundsson (eds), MultiMedia Modeling: 23rd International Conference, MMM 2017, Proceedings. vol. 10133 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10133 LNCS, Springer Verlag, pp. 463-468, 23rd International Conference on MultiMedia Modeling, MMM 2017, Reykjavik, Iceland, 4/01/17. https://doi.org/10.1007/978-3-319-51814-5_42

Concept-based interactive search system. / Lu, Yi-Jie ; Nguyen, Phuong Anh ; Zhang, Hao et al.
MultiMedia Modeling: 23rd International Conference, MMM 2017, Proceedings. ed. / Cathal Gurrin; Björn Thór Jónsson; Laurent Amsaleg; Shin’ichi Satoh; Gylfi Thór Gudmundsson. Vol. 10133 LNCS Springer Verlag, 2016. p. 463-468 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10133 LNCS).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Concept-based interactive search system

AU - Lu, Yi-Jie

AU - Nguyen, Phuong Anh

AU - Zhang, Hao

AU - Ngo, Chong-Wah

PY - 2016/12/31

Y1 - 2016/12/31

N2 - Our successful multimedia event detection system at TRECVID 2015 showed its strength on handling complex concepts in a query. The system was based on a large number of pre-trained concept detectors for textual-to-visual relation. In this paper, we enhance the system by enabling human-in-the-loop. In order to facilitate a user to quickly find an information need, we incorporate concept screening, video reranking by highlighted concepts, relevance feedback and color sketch to refine a coarse retrieval result. The aim is to eventually come up with a system suitable for both Ad-hoc Video Search and Known-Item Search. In addition, as the increasing awareness of difficulty in distinguishing shots of very similar scenes, we also explore the automatic story annotation along the timeline of a video, so that a user can quickly grasp the story happened in the context of a target shot and reject shots with incorrect context. With the story annotation, a user can refine the search result as well by simply adding a few keywords in a special “context field” of a query.

AB - Our successful multimedia event detection system at TRECVID 2015 showed its strength on handling complex concepts in a query. The system was based on a large number of pre-trained concept detectors for textual-to-visual relation. In this paper, we enhance the system by enabling human-in-the-loop. In order to facilitate a user to quickly find an information need, we incorporate concept screening, video reranking by highlighted concepts, relevance feedback and color sketch to refine a coarse retrieval result. The aim is to eventually come up with a system suitable for both Ad-hoc Video Search and Known-Item Search. In addition, as the increasing awareness of difficulty in distinguishing shots of very similar scenes, we also explore the automatic story annotation along the timeline of a video, so that a user can quickly grasp the story happened in the context of a target shot and reject shots with incorrect context. With the story annotation, a user can refine the search result as well by simply adding a few keywords in a special “context field” of a query.

KW - Concept bank

KW - Known-item search

KW - Semantic query

KW - Story annotation

KW - Video reranking

KW - Video search

UR - http://www.scopus.com/inward/record.url?scp=85009932228&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85009932228&origin=recordpage

U2 - 10.1007/978-3-319-51814-5_42

DO - 10.1007/978-3-319-51814-5_42

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9783319518138

VL - 10133 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 463

EP - 468

BT - MultiMedia Modeling

A2 - Gurrin, Cathal

A2 - Jónsson, Björn Thór

A2 - Amsaleg, Laurent

A2 - Satoh, Shin’ichi

A2 - Gudmundsson, Gylfi Thór

PB - Springer Verlag

T2 - 23rd International Conference on MultiMedia Modeling, MMM 2017

Y2 - 4 January 2017 through 6 January 2017

ER -

Lu YJ , Nguyen PA , Zhang H , Ngo CW. Concept-based interactive search system. In Gurrin C, Jónsson BT, Amsaleg L, Satoh S, Gudmundsson GT, editors, MultiMedia Modeling: 23rd International Conference, MMM 2017, Proceedings. Vol. 10133 LNCS. Springer Verlag. 2016. p. 463-468. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-51814-5_42

Concept-based interactive search system

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Multimedia Hyperlinking: From Similarity Ranking to Entity Linking

GRF: Beyond Relevancy: Multimedia Analytics for Exploratory Video Search

Interactive Video Retrieval

Cite this

Concept-based interactive search system

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Multimedia Hyperlinking: From Similarity Ranking to Entity Linking

GRF: Beyond Relevancy: Multimedia Analytics for Exploratory Video Search

Student theses

Interactive Video Retrieval

Cite this