Dynamic graph convolutional network for multi-video summarization

Jiaxin Wu, Sheng-hua Zhong*, Yan Liu

*Corresponding author for this work

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

52 Citations (Scopus)

Abstract

Multi-video summarization is an effective tool for users to browse multiple videos. In this paper, multi-video summarization is formulated as a graph analysis problem and a dynamic graph convolutional network is proposed to measure the importance and relevance of each video shot in its own video as well as in the whole video collection. Two strategies are proposed to solve the inherent class imbalance problem of video summarization task. Moreover, we propose a diversity regularization to encourage the model to generate a diverse summary. Extensive experiments are conducted, and the comparisons are carried out with the state-of-the-art video summarization methods, the traditional and novel graph models. Our method achieves state-of-the-art performances on two standard video summarization datasets. The results demonstrate the effectiveness of our proposed model in generating a representative summary for multiple videos with good diversity.
Original languageEnglish
Article number107382
JournalPattern Recognition
Volume107
Online published20 Jun 2020
DOIs
Publication statusPublished - Nov 2020

Funding

This work was supported by the Natural Science Foundation of Guangdong Province (No. 2019A1515011181 ), the Science and Technology Innovation Commission of Shenzhen under Grant (No. JCYJ20190808162613130 ), the Hong Kong Polytechnic University Grant G-UAEU, the communication platform at the Third Afficated Hospital of SUN Yat-Sen University, and the Shenzhen high-level talents program.

Research Keywords

  • Class imbalance problem
  • Graph convolutional network
  • Multi-video summarization

Fingerprint

Dive into the research topics of 'Dynamic graph convolutional network for multi-video summarization'. Together they form a unique fingerprint.

Cite this