Leveraging long-term predictions and online learning in agent-based multiple person tracking

We present a multiple-person tracking algorithm, based on combining particle filters (PFs) and reciprocal velocity obstacle (RVO), an agent-based crowd model that infers collision-free velocities so as to predict a pedestrian's motion. In addition to position and velocity, our tracking algorithm can estimate the internal goals (desired destination or desired velocity) of the tracked pedestrian in an online manner, thus removing the need to specify this information beforehand. Furthermore, we leverage the longer term predictions of RVO by deriving a higher order PF, which aggregates multiple predictions from different prior time steps. This yields a tracker that can recover from short-term occlusions and spurious noise in the appearance model. Experimental results show that our tracking algorithm is suitable for predicting pedestrians' behaviors online without needing scene priors or hand-annotated goal information, and improves tracking in real-world crowded scenes under low frame rates.