Adversarial T-Shirt! Evading Person Detectors in a Physical World

Kaidi Xu; Gaoyuan Zhang; Sijia Liu; Quanfu Fan; Mengshu Sun; Hongge Chen; Pin-Yu Chen; Yanzhi Wang; Xue Lin

doi:10.1007/978-3-030-58558-7_39

Adversarial T-Shirt! Evading Person Detectors in a Physical World

Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi Wang, Xue Lin^*

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

254 Citations (Scopus)

Abstract

It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person’s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in the digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously. © 2020, Springer Nature Switzerland AG.

Original language	English
Title of host publication	Computer Vision – ECCV 2020
Subtitle of host publication	16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V
Editors	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Place of Publication	Cham
Publisher	Springer
Pages	665-681
ISBN (Electronic)	978-3-030-58558-7
ISBN (Print)	9783030585570
DOIs	https://doi.org/10.1007/978-3-030-58558-7_39
Publication status	Published - 2020
Externally published	Yes
Event	16th European Conference on Computer Vision (ECCV 2020) - Online, Glasgow, United Kingdom Duration: 23 Aug 2020 → 28 Aug 2020 https://eccv2020.eu/

Publication series

Name	Lecture Notes in Computer Science
Volume	12350
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Conference on Computer Vision (ECCV 2020)
Abbreviated title	ECCV 2020
Place	United Kingdom
City	Glasgow
Period	23/08/20 → 28/08/20
Internet address	https://eccv2020.eu/

Funding

Acknowledgement. This work is partly supported by the National Science Foundation CNS-1932351. We would also like to extend our gratitude to MIT-IBM Watson AI Lab.

Research Keywords

Deep learning
Object detection
Physical adversarial attack

Access Output

10.1007/978-3-030-58558-7_39

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Xu, K., Zhang, G., Liu, S., Fan, Q., Sun, M., Chen, H., Chen, P.-Y., Wang, Y., & Lin, X. (2020). Adversarial T-Shirt! Evading Person Detectors in a Physical World. In A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Eds.), Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V (pp. 665-681). (Lecture Notes in Computer Science; Vol. 12350 ). Springer . https://doi.org/10.1007/978-3-030-58558-7_39

Xu, Kaidi ; Zhang, Gaoyuan ; Liu, Sijia et al. / Adversarial T-Shirt! Evading Person Detectors in a Physical World. Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V. editor / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Cham : Springer , 2020. pp. 665-681 (Lecture Notes in Computer Science).

@inproceedings{fc4a79cced23447ab0eb69f68a814dd1,

title = "Adversarial T-Shirt! Evading Person Detectors in a Physical World",

abstract = "It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person{\textquoteright}s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in the digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously. {\textcopyright} 2020, Springer Nature Switzerland AG.",

keywords = "Deep learning, Object detection, Physical adversarial attack",

author = "Kaidi Xu and Gaoyuan Zhang and Sijia Liu and Quanfu Fan and Mengshu Sun and Hongge Chen and Pin-Yu Chen and Yanzhi Wang and Xue Lin",

year = "2020",

doi = "10.1007/978-3-030-58558-7_39",

language = "English",

isbn = "9783030585570",

series = "Lecture Notes in Computer Science",

publisher = "Springer ",

pages = "665--681",

editor = "Vedaldi, {Andrea } and Bischof, {Horst } and Brox, {Thomas } and Frahm, {Jan-Michael }",

booktitle = "Computer Vision – ECCV 2020",

note = "16th European Conference on Computer Vision (ECCV 2020), ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

url = "https://eccv2020.eu/",

}

Xu, K, Zhang, G, Liu, S, Fan, Q, Sun, M, Chen, H, Chen, P-Y, Wang, Y & Lin, X 2020, Adversarial T-Shirt! Evading Person Detectors in a Physical World. in A Vedaldi, H Bischof, T Brox & J-M Frahm (eds), Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V. Lecture Notes in Computer Science, vol. 12350 , Springer , Cham, pp. 665-681, 16th European Conference on Computer Vision (ECCV 2020), Glasgow, United Kingdom, 23/08/20. https://doi.org/10.1007/978-3-030-58558-7_39

Adversarial T-Shirt! Evading Person Detectors in a Physical World. / Xu, Kaidi; Zhang, Gaoyuan; Liu, Sijia et al.
Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Cham: Springer , 2020. p. 665-681 (Lecture Notes in Computer Science; Vol. 12350 ).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Adversarial T-Shirt! Evading Person Detectors in a Physical World

AU - Xu, Kaidi

AU - Zhang, Gaoyuan

AU - Liu, Sijia

AU - Fan, Quanfu

AU - Sun, Mengshu

AU - Chen, Hongge

AU - Chen, Pin-Yu

AU - Wang, Yanzhi

AU - Lin, Xue

PY - 2020

Y1 - 2020

N2 - It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person’s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in the digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously. © 2020, Springer Nature Switzerland AG.

AB - It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person’s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in the digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously. © 2020, Springer Nature Switzerland AG.

KW - Deep learning

KW - Object detection

KW - Physical adversarial attack

UR - http://www.scopus.com/inward/record.url?scp=85097409608&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85097409608&origin=recordpage

U2 - 10.1007/978-3-030-58558-7_39

DO - 10.1007/978-3-030-58558-7_39

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9783030585570

T3 - Lecture Notes in Computer Science

SP - 665

EP - 681

BT - Computer Vision – ECCV 2020

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer

CY - Cham

T2 - 16th European Conference on Computer Vision (ECCV 2020)

Y2 - 23 August 2020 through 28 August 2020

ER -

Xu K, Zhang G, Liu S, Fan Q, Sun M, Chen H et al. Adversarial T-Shirt! Evading Person Detectors in a Physical World. In Vedaldi A, Bischof H, Brox T, Frahm JM, editors, Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V. Cham: Springer . 2020. p. 665-681. (Lecture Notes in Computer Science). Epub 2020 Oct 29. doi: 10.1007/978-3-030-58558-7_39

Adversarial T-Shirt! Evading Person Detectors in a Physical World

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this