Two-Stream 3D MobileNetV3 for Pedestrians Intent Prediction Based on Monocular Camera

Recent developments in the field of autonomous vehicles have led to a renewed interest in combining deep learning and autonomous driving, aiming to support vision-related safety applications in Intelligent Transportation System (ITS). Pedestrians intent prediction is critical in improving the efficiency and safety of autonomous driving since complex traffic scenarios are ubiquitous, specifically facing pedestrian intrusion and other emergencies. Substantial studies have focused on using skeleton features as input to the neural network to extract pedestrians’ features. However, most of them cannot avoid the loss of feature points and design a bloated neural network. In this paper, we propose a spatio-temporal MobileNetV3 based on Two-Stream to predict pedestrian intention. Based on the pedestrian images cropped by the bounding box, we use RGB images to generate optical flow to replace the pedestrian’s skeleton features as the network’s input. With the benefit of 3D depthwise separable convolution, our network’s Multiply-Accumulate Operations (MACs) is one-fifth of the baseline model. Finally, we conduct experiments on real traffic datasets to verify our proposed method’s effectiveness in different road environments.