Abstract
Multi-video summarization is an effective tool for users to browse multiple videos. In this paper, multi-video summarization is formulated as a graph analysis problem and a dynamic graph convolutional network is proposed to measure the importance and relevance of each video shot in its own video as well as in the whole video collection. Two strategies are proposed to solve the inherent class imbalance problem of video summarization task. Moreover, we propose a diversity regularization to encourage the model to generate a diverse summary. Extensive experiments are conducted, and the comparisons are carried out with the state-of-the-art video summarization methods, the traditional and novel graph models. Our method achieves state-of-the-art performances on two standard video summarization datasets. The results demonstrate the effectiveness of our proposed model in generating a representative summary for multiple videos with good diversity.
Original language | English |
---|---|
Article number | 107382 |
Journal | Pattern Recognition |
Volume | 107 |
Online published | 20 Jun 2020 |
DOIs | |
Publication status | Published - Nov 2020 |
Funding
This work was supported by the Natural Science Foundation of Guangdong Province (No. 2019A1515011181 ), the Science and Technology Innovation Commission of Shenzhen under Grant (No. JCYJ20190808162613130 ), the Hong Kong Polytechnic University Grant G-UAEU, the communication platform at the Third Afficated Hospital of SUN Yat-Sen University, and the Shenzhen high-level talents program.
Research Keywords
- Class imbalance problem
- Graph convolutional network
- Multi-video summarization