Enabling Deep Learning for Traffic Engineering in Software Defined WANs

The wide area network (WAN) is a crucial infrastructure for many cloud service provides as it carries traffic for numerous online applications and services at a global scale. These WANs are typically software defined with a logically centralized controller taking charge. Traffic engineering (TE) is a critical and difficult problem in these SD-WANs that involves assigning traffic with various requirements to paths with different constraints. Recently, deep learning (DL) algorithms are applied to TE, yet they all assume that the network is a black box, limiting them to model-free reinforcement learning (RL) algorithms with many issues.  We therefore ask, why are we stuck with RL for TE? Can we find a way to enable the rapid advance in many other DL models to be adopted for TE? This project presents a holistic effort to address this fundamental question. We show that the SD-WAN network environment can be sufficiently modeled for TE, and design a fully- differentiable network environment, ∂NE, that can be directly integrated into any DL models. With ∂NE, we can differentiate with respect to control parameters, and directly evaluate gradients between actions and states to facilitate gradient descent based training. We show with a proof-of-concept prototype that ∂NE accelerates DL training for TE by 228× and achieves higher scalability compared to existing network simulators. Based on ∂NE, we also propose to develop novel DL-based TE algorithms beyond RL. As a concrete example, we examine a new TE scheme based on LSTM (Long Short-Term Memory), and preliminary experiments on our ∂NE prototype show that it achieves 7.0% higher throughput and 89.9% lower congestion loss than state-of-the-art RL-based TE. Finally, we plan to investigate the explainability of DL-based TE, and better understand how DL helps TE in a principled way.  TE in SD-WANs is an important research area with eminent practical value, and DL promises huge potential for online control problems like TE. Our project opens up the possibility to apply arbitrary DL models to TE beyond RL. To maximize the impact, we plan to implement ∂NE and our DL-based TE algorithms in popular machine learning frameworks like PyTorch and Tensorflow and open source them. We are working with Tencent to experiment ∂NE using their production network and traffic information, and we plan to collaborate with other cloud operators in order to propel the transfer of our research results to production systems.