Impact of the Distribution Parameter of Data Sampling Approaches on Software Defect Prediction Models

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)

3 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings - 24th Asia-Pacific Software Engineering Conference, APSEC 2017
EditorsJian Lv, He (Jason) Zhang, Mike Hinchey, Xiao Liu
PublisherIEEE
Pages630-635
ISBN (Electronic)978-1-5386-3681-7
Publication statusPublished - Dec 2017

Publication series

NameAsia-Pacific Software Engineering Conference
PublisherIEEE
ISSN (Print)1530-1362

Conference

Title24th Asia-Pacific Software Engineering Conference, APSEC 2017
PlaceChina
CityNanjing, Jiangsu
Period4 - 8 December 2017

Abstract

Sampling methods are known to impact defect prediction performance. These sampling methods have configurable parameters that can significantly affect the prediction performance. It is however, impractical to assess the effect of all the possible different settings in the parameter space for all the several existing sampling methods. A constant and easy to tweak parameter present in all sampling methods is the distribution of the defective and non-defective modules in the dataset known as Pfp (% of fault-prone modules). In this paper, we investigate and assess the performance of defect prediction models where the Pfp parameter of sampling methods are tweaked. An empirical experiment and assessment of seven sampling methods on five prediction models over 20 releases of 10 static metric projects indicate that (1) Area Under the Receiver Operating Characteristics Curve (AUC) performance is not improved after tweaking the Pfp parameter, (2) pf (false alarms) performance degrades as the Pfp is increased. (3) a stable predictor is difficult to achieve across different Pfp rates. Hence, we conclude that the Pfp parameter setting can have a large impact on the performance (except AUC) of defect prediction models. We thus recommend researchers experiment with the Pfp parameter of the sampling method since the distribution of training datasets vary.

Research Area(s)

  • Search based SE, Defect prediction, Sampling methods, Imbalanced Data, Preprocessing, Empirical software engineering

Citation Format(s)

Impact of the Distribution Parameter of Data Sampling Approaches on Software Defect Prediction Models. / Bennin, Kwabena Ebo; Keung, Jacky; Monden, Akito.

Proceedings - 24th Asia-Pacific Software Engineering Conference, APSEC 2017. ed. / Jian Lv; He (Jason) Zhang; Mike Hinchey; Xiao Liu. IEEE, 2017. p. 630-635 (Asia-Pacific Software Engineering Conference).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)