Adversarial Perspectives on Security in AI-Driven Autonomous Driving

Abstract

Autonomous vehicles (AVs) have rapidly advanced in recent years, shaping the future of transportation. AI-driven models are deeply integrated into autonomous driving (AD) systems across all modules—from perception and prediction to planning—to enhance driving performance. However, real-world complexity introduces a range of security risks, exposing learning-based models to uncertain, dynamic, and potentially adversarial conditions. Critically, these models are vulnerable to malicious attacks that can manipulate perception and decision-making, potentially leading to accidents. Given the high stakes for road safety, it is essential to systematically identify security vulnerabilities and develop robust defenses to ensure AD systems are resilient against adversarial threats.

Significant prior work has examined attacks and defenses targeting the perception modules of AD systems, primarily focusing on object-related tasks such as object detection and traffic sign recognition. However, vulnerabilities in another critical data flow, map perception, remain largely underexplored. More importantly, the impact of perception attacks on downstream prediction and planning modules has received limited attention. This thesis addresses these gaps by investigating security risks in these underexplored areas. This involves demonstrating real-world attacks in map perception and decision-making modules and propose both targeted and general defense mechanisms to secure the AI-driven AD systems against adversarial threats.

To achieve this goal, my thesis begins by examining the online map construction task within the perception module, which initiates the map-related data flow. We identify a novel model-level vulnerability: a bias toward predicting symmetric road structures (e.g., straight roads, intersections) in asymmetric scenes (e.g., forks, merges). To exploit this bias, we propose a physical-world attack that identifies effective configurations for deploying flashlights or adversarial patches along the roadside. These interventions mislead the AV into generating incorrect map structures, ultimately resulting in unsafe planning behavior.

Beyond perception, we identify that downstream prediction and planning modules lack both input validation for upstream perception outputs and dedicated defense mechanisms. Exploiting this vulnerability, we propose the first physical-world attack targeting the prediction module in AD systems. By strategically placing common objects, we manipulate perception outputs, indirectly inducing prediction errors and triggering hazardous planning decisions in the victim AV. This work highlights a critical pipeline vulnerability, where upstream perception attacks can propagate and compromise downstream decision-making.

Finally, we propose both targeted and general defenses to mitigate the identified threats. To address the map perception vulnerability, we introduce asymmetric data fine-tuning to correct training data imbalance. To protect the prediction module, we develop a heading-centric adversarial state detector that validates perception outputs using spatiotemporal consistency across frames. For a general-purpose defense, we incorporate uncertainty into multi-modal fusion, enhancing perception robustness under adverse weather and attacks. These defenses contribute toward building more safe and secure AD systems.

Date of Award	15 Aug 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Jianping WANG (Supervisor)