Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review

19 Scopus Citations
View graph of relations

Author(s)

  • Koji Toda
  • Yasutaka Kamei
  • Akito Monden
  • Naoyasu Ubayashi

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages214-221
ISBN (Print)9781509041275
Publication statusPublished - 12 Oct 2016

Conference

Title2nd IEEE International Conference on Software Quality, Reliability and Security, QRS 2016
PlaceAustria
CityVienna
Period1 - 3 August 2016

Abstract

To prioritize quality assurance efforts, various fault prediction models have been proposed. However, the best performing fault prediction model is unknown due to three major drawbacks: (1) comparison of few fault prediction models considering small number of data sets, (2) use of evaluation measures that ignore testing efforts and (3) use of n-fold cross-validation instead of the more practical cross-release validation. To address these concerns, we conducted cross-release evaluation of 11 fault density prediction models using data sets collected from 2 releases of 25 open source software projects with an effort-Aware performance measure known as Norm(Popt). Our result shows that, whilst M5 and K∗ had the best performances, they were greatly influenced by the percentage of faulty modules present and size of data set. Using Norm(Popt) produced an overall average performance of more than 50% across all the selected models clearly indicating the importance of considering testing efforts in building fault-prone prediction models.

Research Area(s)

  • crossversionprediction, Demsar's significance diagram, empirical study, fault-density estimation, open sourceproject

Citation Format(s)

Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models. / Bennin, Kwabena Ebo; Toda, Koji; Kamei, Yasutaka; Keung, Jacky; Monden, Akito; Ubayashi, Naoyasu.

Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 214-221 7589801.

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review