Boosting Adversarial Robustness From The Perspective of Effective Margin Regularization

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationBMVC 2022 - 33rd British Machine Vision Conference Proceedings
PublisherBritish Machine Vision Association, BMVA
Number of pages14
Publication statusPublished - Nov 2022

Publication series

NameBMVC - British Machine Vision Conference Proceedings

Conference

Title33rd British Machine Vision Conference (BMVC 2022)
PlaceUnited Kingdom
CityLondon
Period21 - 24 November 2022

Abstract

The adversarial vulnerability of deep neural networks (DNNs) has been actively investigated in the past several years. This paper investigates the scale-variant property of cross-entropy loss, which is the most commonly used loss function in classification tasks, and its impact on the effective margin and adversarial robustness of deep neural networks. Since the loss function is not invariant to logit scaling, increasing the effective weight norm will make the loss approach zero and its gradient vanish while the effective margin is not adequately maximized. On typical DNNs, we demonstrate that, if not properly regularized, the standard training does not learn large effective margins and leads to adversarial vulnerability. To maximize the effective margins and learn a robust DNN, we propose to regularize the effective weight norm during training. Our empirical study on feedforward DNNs demonstrates that the proposed effective margin regularization (EMR) learns large effective margins and boosts the adversarial robustness in both standard and adversarial training. On large-scale models, we show that EMR outperforms basic adversarial training, TRADES and two regularization baselines with substantial improvement. Moreover, when combined with several strong adversarial defense methods (MART [48] and MAIL [26]), our EMR further boosts the robustness. © 2022. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.

Citation Format(s)

Boosting Adversarial Robustness From The Perspective of Effective Margin Regularization. / Liu, Ziquan; Chan, Antoni B.
BMVC 2022 - 33rd British Machine Vision Conference Proceedings. British Machine Vision Association, BMVA, 2022. (BMVC - British Machine Vision Conference Proceedings).

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review