Enhancing Valid Test Input Generation with Distribution Awareness for Deep Neural Networks
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2024 IEEE 48th Annual Computers, Software, and Applications Conference |
| Editors | Hossain Shahriar, Hiroyuki Ohsaki, Moushumi Sharmin |
| Publisher | Institute of Electrical and Electronics Engineers, Inc. |
| Pages | 1095-1100 |
| ISBN (electronic) | 979-8-3503-7696-8 |
| ISBN (print) | 979-8-3503-7697-5 |
| Publication status | Published - 2024 |
Publication series
| Name | |
|---|---|
| ISSN (Print) | 2836-3787 |
| ISSN (electronic) | 2836-3795 |
Conference
| Title | 48th IEEE International Conference on Computers, Software, and Applications (COMPSAC 2024) |
|---|---|
| Location | Osaka University |
| Place | Japan |
| City | Osaka |
| Period | 2 - 4 July 2024 |
Link(s)
Abstract
Comprehensive testing is important in improving the reliability of Deep Learning (DL)-based systems. Various Test Input Generators (TIGs) have been proposed to generate misbehavior-inducing test inputs. However, the lack of validity checking in TIGs often results in the generation of invalid inputs (i.e., out of the learned distribution), leading to unreliable testing. To save the effort of manually checking the validity and improve test efficiency, it is important to assess the effectiveness and reliability of automated validators.
In this study, we comprehensively assess four automated Input Validators (IV s), Our findings show that the accuracy of IVs ranges from 49% to 77%. Distance-based IVs generally outperform reconstruction-based and density-based IVs for both classification and regression tasks.
Based on the findings, we enhance existing testing frameworks by incorporating distribution awareness through joint optimization. The results demonstrate our framework leads to a 2 % to 10% increase in the number of valid inputs, which establishes our method as an effective technique for valid test input generation.
© 2024 IEEE
In this study, we comprehensively assess four automated Input Validators (IV s), Our findings show that the accuracy of IVs ranges from 49% to 77%. Distance-based IVs generally outperform reconstruction-based and density-based IVs for both classification and regression tasks.
Based on the findings, we enhance existing testing frameworks by incorporating distribution awareness through joint optimization. The results demonstrate our framework leads to a 2 % to 10% increase in the number of valid inputs, which establishes our method as an effective technique for valid test input generation.
© 2024 IEEE
Research Area(s)
- Input Validation, Anomaly Detection, Deep Learning, Software Testing
Bibliographic Note
Since this conference is yet to commence, the information for this record is subject to revision.
Citation Format(s)
Enhancing Valid Test Input Generation with Distribution Awareness for Deep Neural Networks. / Zhang, Jingyu; Keung, Jacky; Ma, Xiaoxue et al.
Proceedings - 2024 IEEE 48th Annual Computers, Software, and Applications Conference. ed. / Hossain Shahriar; Hiroyuki Ohsaki; Moushumi Sharmin. Institute of Electrical and Electronics Engineers, Inc., 2024. p. 1095-1100.
Proceedings - 2024 IEEE 48th Annual Computers, Software, and Applications Conference. ed. / Hossain Shahriar; Hiroyuki Ohsaki; Moushumi Sharmin. Institute of Electrical and Electronics Engineers, Inc., 2024. p. 1095-1100.
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
