Off-policy reinforcement learning-based decentralized stabilization for interconnected nonlinear systems

In this paper, we consider a class of interconnected systems, where each subsystem is a continuous-time nonlinear system, with both state interactions and input interactions among them. Given the high cost and complexity associated with designing centralized controllers, making decisions based on local information from subsystems has become a common alternative. Therefore, we aim to design a set of decentralized control strategies that ensure system stability. To this end, we integrate off-policy adaptive dynamic programming with neural network approximators. First, by defining an auxiliary cost function for each subsystem, a decentralized control policy is obtained, which turns out to be the subsystem's best response to the policies of its neighbors. Due to the difficulty in obtaining analytical solutions for nonlinear cases, a constructive procedure is presented to approximate the decentralized control policy using the system trajectory. Second, the stability of the interconnected system is proven using the Lyapunov theory. Our approach is synthesized into Algorithm 1. Two simulation examples based on interconnected nonlinear systems validate our theoretical analysis, highlighting the superiority of the proposed approach over reference methods. © 2025 Published by Elsevier Inc.