Action recognition from depth sequence using depth motion maps-based local ternary patterns and CNN

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

12 Scopus Citations
View graph of relations


Related Research Unit(s)


Original languageEnglish
Pages (from-to)19587–19601
Journal / PublicationMultimedia Tools and Applications
Issue number14
Online published15 Feb 2019
Publication statusPublished - Jul 2019


This paper presents a method for human action recognition from depth sequences captured by the depth camera. The main idea of the method is the action mapping image classification via convolutional neural network (CNN) based approach. Firstly, we project the raw frames onto three orthogonal Cartesian planes and stack the results into three still images (corresponding to the front, side, and top views) to form the Depth Motion Maps (DMMs). Secondly, Local Ternary Pattern (LTP) is introduced as an image filter for DMMs, thus to improve the distinguishability of similar actions. Finally, we apply CNN to action recognition by classifying corresponding LTP-encoded images. Experimental results on the popular and challenging benchmark MSR-Action 3D and MSR-Gesture dataset show the effectiveness of the presented method and meet real-time action recognition task requirements.

Research Area(s)

  • Convolutional neural network, Depth motion maps, Human action recognition, Local ternary pattern