This thesis investigates the effectiveness of reinforcement learning (RL) approaches for adaptive network traffic control problems in large-scale multi-modal networks. In the context of dynamic optimisation, we develop a model-based and data-driven control framework to address three key challenges: 1) the curse of dimensionality in traffic control problems, 2) traffic congestion and transit service variabilities and 3) demand uncertainties and stochastic network capacity. In particular, the thesis studies adaptive traffic control on the intersection and network levels, aiming to improve road traffic efficiency and bus service reliability. We demonstrate that RL can serve as an effective and unifying solution framework, which integrates traffic flow models and parametric function approximations.

The first work establishes an integrated model-based and data-driven adaptive signal controller to minimise network delay. The model-based component operates based upon a kinematic wave model, and the data-driven component operates based on the parametric approximator with linear regression. The results show that the model-based component facilitates the training of the approximator and the data-driven approximator reduces the computational complexity of the adaptive control problem. Furthermore, we develop a decentralised solution to allow individual intersections to derive their own control policies asynchronously. With the approximator serving as a central coordinator, the decentralised controller can improve and stabilise the performance of the overall control system even under congested conditions.

The second work extends the proposed adaptive signal controller to the transit network in a multi-objective optimisation framework. Specifically, the adaptive signal controller is designed to manage traffic delays and bus service reliability jointly. Different from the previous work, the model-based component relies on a hybrid kinematic wave model that incorporates bus service status, and the data-driven component adopts a multi-layer artificial neural network to capture the more sophisticated dynamics in the hybrid traffic system. The results show that the proposed controller can reduce traffic delays and bus service variabilities subject to stochastic demand with acyclic timing plans that can be derived in a short computational time.

The final work combines the network-level perimeter control and the local adaptive signal control to present a novel hierarchical control framework for stochastic road networks. By regulating the network inflow, the upper-level perimeter control can maintain and improve the operational capacity and reliability of the road network by maximising network throughput. To be more specific, the upper level uses a RL algorithm that learns and responds to the traffic dynamics in the core road network without the need of an underlying system model and macroscopic fundamental diagram. The lower level is a local signal control system that regulates the spatial distribution of traffic flow within the core network. The results show that the hierarchical control framework can improve the network throughput by coordinating the control actions conducted by the two levels.

In conclusion, this thesis contributes to the development of adaptive network traffic control using advanced traffic modelling and RL-based optimisation techniques. The findings of the thesis support further analysis and development in this area.