Deep Reinforcement Learning Based Residential Demand Side Management With Edge Computing

Residential demand side management (DSM) is a promising technique to improve the stability and reduce the cost of power systems. However, residential DSM is facing challenges under the ongoing paradigm shift of computation, such as edge computing. With the proliferation of smart appliances (e.g., appliances with computing and data analysis capabilities) and high-performance computing devices (e.g., graphics processing units) in the households, we expect surging residential energy consumption caused by computation. Therefore, it is important to schedule edge computing as well as traditional energy consumption in a smart way, especially when the demand for computation and thus for electricity occurs during the peak hours of electricity consumption.

In this paper, we investigate an integrated home energy management system (HEMS) who participates in a DSM program and is equipped with an edge computing server. The HEMS aims to maximize the home owner's expected total reward, defined as the reward from completing edge computing tasks minus the cost of electricity consumption, the cost of computation offloading to the cloud, and the penalty of violating the DSM requirements. The particular DSM program considered in this paper, which is a widely-adopted one, requires the household to reduce certain amount of energy consumption within a specified time window. In contrast to well-studied real-time pricing, such a DSM program results in a long-term temporal interdependency (i.e., of a few hours) and thus high-dimensional state space in our formulated Markov decision processes. To address this challenge, we use deep reinforcement learning, more specifically Deep Deterministic Policy Gradient, to solve the problem. Experiments show that our proposed scheme achieves significant performance gains over reasonable baselines.