Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

45 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)1429-1440
Journal / PublicationIEEE Transactions on Neural Networks and Learning Systems
Volume33
Issue number4
Online published22 Dec 2020
Publication statusPublished - Apr 2022

Abstract

In this article, we study a multiplayer Stackelberg-Nash game (SNG) pertaining to a nonlinear dynamical system, including one leader and multiple followers. At the higher level, the leader makes its decision preferentially with consideration of the reaction functions of all followers, while, at the lower level, each of the followers reacts optimally to the leader's strategy simultaneously by playing a Nash game. First, the optimal strategies for the leader and the followers are derived from down to the top, and these strategies are further shown to constitute the Stackelberg-Nash equilibrium points. Subsequently, to overcome the difficulty in calculating the equilibrium points analytically, we develop a novel two-level value iteration-based integral reinforcement learning (VI-IRL) algorithm that relies only upon partial information of system dynamics. We establish that the proposed method converges asymptotically to the equilibrium strategies under the weak coupling conditions. Moreover, we introduce effective termination criteria to guarantee the admissibility of the policy (strategy) profile obtained from a finite number of iterations of the proposed algorithm. In the implementation of our scheme, we employ neural networks (NNs) to approximate the value functions and invoke the least-squares methods to update the involved weights. Finally, the effectiveness of the developed algorithm is verified by two simulation examples.

Research Area(s)

  • Integral reinforcement learning (IRL), multiplayer Stackelberg-Nash game (SNG), neural networks (NNs), nonlinear system, value iteration (VI)

Citation Format(s)

Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning. / Li, Man; Qin, Jiahu; Freris, Nikolaos M. et al.
In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, No. 4, 04.2022, p. 1429-1440.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review