Due to the popularity of smartphones, there is a great need to deploy appropriate authentication mechanisms to safeguard users’ sensitive data. Touch dynamics-based authentication has been developed to verify smartphone users and detect imposters. These schemes usually employ machine learning techniques to detect behavioral anomalies by comparing current behavioral actions with the stored normal model. However, we notice that machine learning classifiers often have an unstable performance, which would greatly reduce the system usability, i.e., causing a high false rejection. In this work, we are motivated by this challenge and design a cost-based intelligent mechanism that can choose a less costly algorithm for user authentication. In the evaluation, we conduct a user study with a total of 60 users to investigate the performance of our mechanism with a lightweight touch gesture-based scheme on smartphones. Experimental results demonstrate that our approach can help achieve a relatively higher and more stable authentication accuracy, as compared to the use of a sole classifier.