Maintaining knowledge-based systems against adversarial attacks : A differential game approach

This paper analyzes strategic interactions between afirm that seeks to optimally maintain the accuracy of a knowledge-based system and an adversary who seeks to impair the accuracy of the system. Thefirm maintains the system to reduce misclassifications. On the other hand, the adversary acts to reduce system accuracy to increase the likelihood that the system does not identify a malicious event. We formulate the problem using a differential game model, in which the optimal effort levels of maintaining and attacking the system are determined in a continuous time problem. Our solution shows that, over time the optimal accuracy level converges to a steady state level. Depending on the adversary's cost function and the initial system accuracy, thefirm and the adversary may exert different amount of effort in equilibrium before reaching a steady level of accuracy.