Towards textually describing complex video contents with audio-visual concept classifiers

Chun Chet Tan; Yu-Gang Jiang; Chong-Wah Ngo

doi:10.1145/2072298.2072411

Towards textually describing complex video contents with audio-visual concept classifiers

Chun Chet Tan, Yu-Gang Jiang, Chong-Wah Ngo

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

32 Citations (Scopus)

Abstract

Automatically generating compact textual descriptions of complex video contents has wide applications. With the recent advancements in automatic audio-visual content recognition, in this paper we explore the technical feasibility of the challenging issue of precisely recounting video contents. Based on cutting-edge automatic recognition techniques, we start from classifying a variety of visual and audio concepts in video contents. According to the classification results, we apply simple rule-based methods to generate textual descriptions of video contents. Results are evaluated by conducting carefully designed user studies. We find that the state-of-the-art visual and audio concept classification, although far from perfect, is able to provide very useful clues indicating what is happening in the videos. Most users involved in the evaluation confirmed the informativeness of our machine-generated descriptions. © 2011 ACM.

Original language	English
Title of host publication	MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops
Pages	655-658
DOIs	https://doi.org/10.1145/2072298.2072411
Publication status	Published - 2011
Event	19th ACM International Conference on Multimedia (ACM Multimedia Conference 2011) - Scottsdale, United States Duration: 28 Nov 2011 → 1 Dec 2011 Conference number: 19 https://dl.acm.org/doi/proceedings/10.1145/2072298

Conference

Conference	19th ACM International Conference on Multimedia (ACM Multimedia Conference 2011)
Abbreviated title	MM'11
Place	United States
City	Scottsdale
Period	28/11/11 → 1/12/11
Internet address	https://dl.acm.org/doi/proceedings/10.1145/2072298

Research Keywords

Audio-visual concept classification
Textual descriptions of video content

Access Output

10.1145/2072298.2072411

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{5056973118864b0cbdc8a2dc117c3b20,

title = "Towards textually describing complex video contents with audio-visual concept classifiers",

abstract = "Automatically generating compact textual descriptions of complex video contents has wide applications. With the recent advancements in automatic audio-visual content recognition, in this paper we explore the technical feasibility of the challenging issue of precisely recounting video contents. Based on cutting-edge automatic recognition techniques, we start from classifying a variety of visual and audio concepts in video contents. According to the classification results, we apply simple rule-based methods to generate textual descriptions of video contents. Results are evaluated by conducting carefully designed user studies. We find that the state-of-the-art visual and audio concept classification, although far from perfect, is able to provide very useful clues indicating what is happening in the videos. Most users involved in the evaluation confirmed the informativeness of our machine-generated descriptions. {\textcopyright} 2011 ACM.",

keywords = "Audio-visual concept classification, Textual descriptions of video content",

author = "Tan, {Chun Chet} and Yu-Gang Jiang and Chong-Wah Ngo",

year = "2011",

doi = "10.1145/2072298.2072411",

language = "English",

isbn = "9781450306164",

pages = "655--658",

booktitle = "MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops",

note = "19th ACM International Conference on Multimedia (ACM Multimedia Conference 2011) , MM'11 ; Conference date: 28-11-2011 Through 01-12-2011",

url = "https://dl.acm.org/doi/proceedings/10.1145/2072298",

}

Tan, CC, Jiang, Y-G & Ngo, C-W 2011, Towards textually describing complex video contents with audio-visual concept classifiers. in MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops. pp. 655-658, 19th ACM International Conference on Multimedia (ACM Multimedia Conference 2011) , Scottsdale, Arizona, United States, 28/11/11. https://doi.org/10.1145/2072298.2072411

Towards textually describing complex video contents with audio-visual concept classifiers. / Tan, Chun Chet; Jiang, Yu-Gang; Ngo, Chong-Wah.
MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops. 2011. p. 655-658.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Towards textually describing complex video contents with audio-visual concept classifiers

AU - Tan, Chun Chet

AU - Jiang, Yu-Gang

AU - Ngo, Chong-Wah

N1 - Conference code: 19

PY - 2011

Y1 - 2011

N2 - Automatically generating compact textual descriptions of complex video contents has wide applications. With the recent advancements in automatic audio-visual content recognition, in this paper we explore the technical feasibility of the challenging issue of precisely recounting video contents. Based on cutting-edge automatic recognition techniques, we start from classifying a variety of visual and audio concepts in video contents. According to the classification results, we apply simple rule-based methods to generate textual descriptions of video contents. Results are evaluated by conducting carefully designed user studies. We find that the state-of-the-art visual and audio concept classification, although far from perfect, is able to provide very useful clues indicating what is happening in the videos. Most users involved in the evaluation confirmed the informativeness of our machine-generated descriptions. © 2011 ACM.

AB - Automatically generating compact textual descriptions of complex video contents has wide applications. With the recent advancements in automatic audio-visual content recognition, in this paper we explore the technical feasibility of the challenging issue of precisely recounting video contents. Based on cutting-edge automatic recognition techniques, we start from classifying a variety of visual and audio concepts in video contents. According to the classification results, we apply simple rule-based methods to generate textual descriptions of video contents. Results are evaluated by conducting carefully designed user studies. We find that the state-of-the-art visual and audio concept classification, although far from perfect, is able to provide very useful clues indicating what is happening in the videos. Most users involved in the evaluation confirmed the informativeness of our machine-generated descriptions. © 2011 ACM.

KW - Audio-visual concept classification

KW - Textual descriptions of video content

UR - http://www.scopus.com/inward/record.url?scp=84455192418&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-84455192418&origin=recordpage

U2 - 10.1145/2072298.2072411

DO - 10.1145/2072298.2072411

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781450306164

SP - 655

EP - 658

BT - MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

T2 - 19th ACM International Conference on Multimedia (ACM Multimedia Conference 2011)

Y2 - 28 November 2011 through 1 December 2011

ER -

Towards textually describing complex video contents with audio-visual concept classifiers

Abstract

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this