Improving Autonomous Behavior Strategy Learning in an Unmanned Swarm System Through Knowledge Enhancement

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

5 Scopus Citations
View graph of relations

Author(s)

  • Lai Chai
  • Shenshen Wang
  • Junyu Jin
  • Aiguo Song
  • Yushi Lan

Detail(s)

Original languageEnglish
Pages (from-to)763-774
Number of pages12
Journal / PublicationIEEE Transactions on Reliability
Volume71
Issue number2
Online published28 Mar 2022
Publication statusPublished - Jun 2022
Externally publishedYes

Abstract

An unmanned swarm system (UWS) is a multiagent system that can fulfill task requirements through autonomous and cooperative behavior strategy learning. However, learning instability is inevitable in a dynamic mission setting, as the agents continuously adapt to an evolving mission objective. This article proposes several knowledge enhancement mechanisms to improve the training efficiency and learning stability of a UWS in a confined-space confrontation mission. Specifically, a punishment for transcending action-space boundary and a reward for satisfying agent space-time distance constraints are introduced as training reward enhancements. Meanwhile, experience sharing among agents is optimized for unanimous behavior. We apply these novel mechanisms to several representative single-agent and multiagent reinforcement learning algorithms and verify their effectiveness on our proprietary, SwarmFlow, simulation system. Simulations show that the proposed mechanisms improve existing algorithms' convergence speed and performance stability. The increase is more prominent for multiagent reinforcement learning algorithms than single-agent algorithms where the convergence time is halved, and the mission success rates increase by 3-4%.

Research Area(s)

  • Training, Task analysis, Heuristic algorithms, Reinforcement learning, Convergence, Convolutional neural networks, Complexity theory, Behavioral decision, confrontation mission, multiagent reinforcement learning, reward mechanism, SwarmFlow, unmanned aerial vehicle (UAV), unmanned swarm system (UWS), UAV, COMMUNICATION, OPTIMIZATION, ALLOCATION, DEPLOYMENT

Citation Format(s)