Real-time Map Construction and Utilization: An End-to-End Framework for Autonomous Driving

Description

In recent years, the convergence of advanced sensor technologies and machine learning algorithms has propelled autonomous driving (AD) closer to large-scale deployment, heralding a safer, more efficient, and accessible era of urban mobility. The architecture of AD, pivoting around critical modules such as perception, prediction, and planning, can have its efficacy significantly augmented with accurate map information. However, this brings forth two main challenges: 1) ensuring the availability of real-time accurate maps for AD modules, and 2) optimizing the utilization of map information across these modules. Concerning availability, map information can be sourced from offline maps provided by map providers or generated online by the perception task. While offline High-Definition (HD) maps offer detailed geometric and semantic insights, their generation and maintenance are expensive and labor-intensive. Conversely, online maps provide real-time solutions but may lack accuracy due to occlusions and perception limitations. The dynamic urban environments further challenge the sole reliance on either map type for achieving the desired accuracy in AD tasks.  Regarding optimal map information usage, existing AD approaches often treat maps as auxiliary inputs across various modules, lacking a unified map-centric representation for different AD tasks. This fragmented approach leads to separated optimization of map information utilization for each AD task, potentially compromising the system-level driving performance. To address the two major challenges, we propose a map-centric framework, embodying an end-to-end approach from encoding offline map data to motion planning. Specifically, we tackle the first challenge by developing advanced machine learning algorithms for a sequence of tasks: 1) encoding the offline map into a graph representation, 2) merging this graph with real-time sensing inputs to generate a real-time scene graph, and 3) further utilizing both graphs to construct an accurate and real-time online map graph. For the second challenge, we propose to develop a unified map-centric graph representation for different AD tasks, ensuring consistent and coherent map information utilization across the perception, prediction, and planning modules. This unification fosters a holistic optimization of system-level driving performance, aiming to significantly enhance each module's accuracy, efficiency, and safety towards achieving safe, efficient, and rule-compliant autonomous vehicles (AVs). In conclusion, our proposal presents a robust framework to address fundamental challenges in AD technology. By fostering seamless interaction and real-time updates among the mapping, perception, prediction, and planning modules, we plan to enhance AVs' accuracy, efficiency, and safety, accelerating the transition towards smart urban mobility and contributing to smart city initiatives.