Human-Machine Bidding Strategy for Distributed Energy Resources Based on Multiagent Inverse Reinforcement Learning

In recent years, the rapid growth of distributed energy resources (DERs) and the emergence of local energy markets (LEMs) have dramatically transformed the energy trading landscape, emphasizing the crucial role of DER aggregators in optimizing bidding strategies. Traditional model-based methods for optimizing DER aggregator bidding in LEMs face significant challenges, including information asymmetry, an inability to adapt to changing market dynamics, and issues with computational scalability in real-time decision-making. Recognized as a promising alternative, deep reinforcement learning (DRL) forms the basis of our proposed solution. This article introduces a human-machine (HM) framework that utilizes a multiagent adversarial inverse reinforcement learning (MA-AIRL) approach to address these challenges. The HM framework enables the DER agent to imitate human demonstrations and leverages a HM hybrid experiment to augment insufficient data, effectively tackling the problem of data inadequacy in new market environments. Concurrently, the MA-AIRL algorithm employs inverse reinforcement learning to capture underlying reward functions, risk preferences, and expert knowledge, significantly enhancing the model's adaptability to dynamic market conditions. Additionally, the adversarial learning component allows the DER agent to robustly respond to uncertainties and the strategic maneuvers of rival agents, thereby mitigating information asymmetry. Moreover, this DRL-based approach is designed to ensure rapid responsiveness without compromising scalability in real-time contexts. Through extensive case studies, we have verified that the proposed HM framework and MA-AIRL algorithm offer a more robust, data-efficient, and adaptive approach for optimizing DER aggregator bidding in LEMs.

© 2025 IEEE. All rights reserved, including rights for text and data mining, and training of artificial intelligence and similar technologies. Personal use is permitted, but republication/redistribution requires IEEE permission.