Human Attention-Guided Explainable AI for Object Detection

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings of the 45th Annual Conference of the Cognitive Science Society
Volume45
Publication statusPublished - Jul 2023

Conference

TitleThe 45th Annual Meeting of the Cognitive Science Society
LocationDarling Harbour
PlaceAustralia
CitySydney
Period26 - 29 July 2023

Abstract

Although object detection AI plays an important role in many critical systems, corresponding Explainable AI (XAI) methods remain very limited. Here we first developed FullGrad-CAM and FullGrad-CAM++ by extending traditional gradient-basedmethods to generate object-specific explanations with higher plausibility. Since human attention may reflect features more interpretable to humans, we explored the possibility to use it as guidance to learn how to combine the explanatory information in the detector model to best present as an XAI saliency map that is interpretable (plausible) to humans. Interestingly, we found that human attention maps had higher faithfulness forex plaining the detector model than existing saliency-based XAI methods. By using trainable activation functions and smoothing kernels to maximize the XAI saliency map similarity to human attention maps, the generated map had higher faithfulness and plausibility than both existing XAI methods and human attention maps. The learned functions were model-specific, well generalizable to other databases.

© 2023 The Author(s).

Research Area(s)

  • Object detection, XAI, Human attention, Deep learning, Saliency map

Bibliographic Note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Citation Format(s)

Human Attention-Guided Explainable AI for Object Detection. / Liu, Guoyang; Zhang, Jindi; Chan, Antoni B. et al.
Proceedings of the 45th Annual Conference of the Cognitive Science Society. Vol. 45 2023.

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review