Data-level information enhancement : Motion-patch-based Siamese Convolutional Neural Networks for human activity recognition in videos
Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Article number | 113203 |
Journal / Publication | Expert Systems with Applications |
Volume | 147 |
Online published | 14 Jan 2020 |
Publication status | Published - 1 Jun 2020 |
Link(s)
Abstract
Data augmentation is critical for deep learning-based human activity recognition (HAR) systems. However, conventional data augmentation methods, such as random-cropping, may generate bad samples that are unrelated to a particular activity (e.g. the background patches without saliency motion information). As a result, the random-cropping based data augmentation may affect negatively the overall performance of HAR systems. Humans, in turn, tend to pay more attention to motion information when recognizing activities. In this work, we attempt to enhance the motion information in HAR systems and mitigate the influence of bad samples through a Siamese architecture, termed as Motion-patch-based Siamese Convolutional Neural Network (MSCNN). The term motion patch is defined as a specific square region that includes critical motion information in the video. We propose a simple yet effective method for selecting those regions. To evaluate the proposed MSCNN, we conduct a number of experiments on the popular datasets UCF-101 and HMDB-51. The mathematical model and experimental results show that the proposed architecture is capable of enhancing the motion information and achieves comparable performance.
Research Area(s)
- Human activity recognition, data augmentation, deep learning, 3D Convolutional Neural Networks
Citation Format(s)
Data-level information enhancement : Motion-patch-based Siamese Convolutional Neural Networks for human activity recognition in videos. / Zhang, Yujia; Po, Lai Man; Liu, Mengyang; Rehman, Yasar Abbas Ur; Ou, Weifeng; Zhao, Yuzhi.
In: Expert Systems with Applications, Vol. 147, 113203, 01.06.2020.Research output: Journal Publications and Reviews (RGC: 21, 22, 62) › 21_Publication in refereed journal › peer-review