Assessing the Significant Impact of Concept Drift in Software Defect Prediction

Md Alamgir Kabir; Jacky W. Keung; Kwabena E. Bennin; Miao Zhang

doi:10.1109/COMPSAC.2019.00017

Assessing the Significant Impact of Concept Drift in Software Defect Prediction

Md Alamgir Kabir, Jacky W. Keung, Kwabena E. Bennin, Miao Zhang

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

21 Citations (Scopus)

Abstract

Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models.

Original language	English
Title of host publication	2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019
Subtitle of host publication	Proceedings
Editors	Vladimir Getov, Jean-Luc Gaudiot, Nariyoshi Yamai, Stelvio Cimato, Morris Chang, Yuuichi Teranishi, Ji-Jiang Yang, Hong Va Leong, Hossian Shahriar, Michiharu Takemoto, Dave Towey, Hiroki Takakura, Atilla Elci, Susumu Takeuchi, Satish Puri
Publisher	IEEE
Pages	53-58
Number of pages	6
ISBN (Print)	9781728126074
DOIs	https://doi.org/10.1109/COMPSAC.2019.00017
Publication status	Published - Jul 2019
Event	43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019 - Milwaukee, United States Duration: 15 Jul 2019 → 19 Jul 2019 https://ieeecompsac.computer.org/2019/

Publication series

Name	Proceedings - International Computer Software and Applications Conference
ISSN (Print)	0730-3157

Conference

Conference	43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019
Place	United States
City	Milwaukee
Period	15/07/19 → 19/07/19
Internet address	https://ieeecompsac.computer.org/2019/

Funding

ACKNOWLEDGMENT This work is supported in part by the General Research Fund of the Research Grants Council of Hong Kong (No.11208017) and the research funds of City University of Hong Kong (9678149, 7005028, and 9678149), and the Research Support Fund by Intel (9220097).

Research Keywords

Concept drift detection
Defect prediction
Empirical software engineering
Software quality
Streaming data

Access Output

10.1109/COMPSAC.2019.00017

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Kabir, M. A., Keung, J. W., Bennin, K. E., & Zhang, M. (2019). Assessing the Significant Impact of Concept Drift in Software Defect Prediction. In V. Getov, J.-L. Gaudiot, N. Yamai, S. Cimato, M. Chang, Y. Teranishi, J.-J. Yang, H. V. Leong, H. Shahriar, M. Takemoto, D. Towey, H. Takakura, A. Elci, S. Takeuchi, & S. Puri (Eds.), 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019: Proceedings (pp. 53-58). Article 8754363 (Proceedings - International Computer Software and Applications Conference). IEEE. https://doi.org/10.1109/COMPSAC.2019.00017

Kabir, Md Alamgir ; Keung, Jacky W. ; Bennin, Kwabena E. et al. / Assessing the Significant Impact of Concept Drift in Software Defect Prediction. 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019: Proceedings. editor / Vladimir Getov ; Jean-Luc Gaudiot ; Nariyoshi Yamai ; Stelvio Cimato ; Morris Chang ; Yuuichi Teranishi ; Ji-Jiang Yang ; Hong Va Leong ; Hossian Shahriar ; Michiharu Takemoto ; Dave Towey ; Hiroki Takakura ; Atilla Elci ; Susumu Takeuchi ; Satish Puri. IEEE, 2019. pp. 53-58 (Proceedings - International Computer Software and Applications Conference).

@inproceedings{7fe0768865f540669227acbf54ab019d,

title = "Assessing the Significant Impact of Concept Drift in Software Defect Prediction",

abstract = "Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models.",

keywords = "Concept drift detection, Defect prediction, Empirical software engineering, Software quality, Streaming data",

author = "Kabir, {Md Alamgir} and Keung, {Jacky W.} and Bennin, {Kwabena E.} and Miao Zhang",

year = "2019",

month = jul,

doi = "10.1109/COMPSAC.2019.00017",

language = "English",

isbn = "9781728126074",

series = "Proceedings - International Computer Software and Applications Conference",

publisher = "IEEE",

pages = "53--58",

editor = "Vladimir Getov and Jean-Luc Gaudiot and Nariyoshi Yamai and Stelvio Cimato and Morris Chang and Yuuichi Teranishi and Ji-Jiang Yang and Leong, {Hong Va} and Hossian Shahriar and Michiharu Takemoto and Dave Towey and Hiroki Takakura and Atilla Elci and Susumu Takeuchi and Satish Puri",

booktitle = "2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019",

address = "United States",

note = "43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019 ; Conference date: 15-07-2019 Through 19-07-2019",

url = "https://ieeecompsac.computer.org/2019/",

}

Kabir, MA , Keung, JW, Bennin, KE & Zhang, M 2019, Assessing the Significant Impact of Concept Drift in Software Defect Prediction. in V Getov, J-L Gaudiot, N Yamai, S Cimato, M Chang, Y Teranishi, J-J Yang, HV Leong, H Shahriar, M Takemoto, D Towey, H Takakura, A Elci, S Takeuchi & S Puri (eds), 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019: Proceedings., 8754363, Proceedings - International Computer Software and Applications Conference, IEEE, pp. 53-58, 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019, Milwaukee, Wisconsin, United States, 15/07/19. https://doi.org/10.1109/COMPSAC.2019.00017

Assessing the Significant Impact of Concept Drift in Software Defect Prediction. / Kabir, Md Alamgir ; Keung, Jacky W.; Bennin, Kwabena E. et al.
2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019: Proceedings. ed. / Vladimir Getov; Jean-Luc Gaudiot; Nariyoshi Yamai; Stelvio Cimato; Morris Chang; Yuuichi Teranishi; Ji-Jiang Yang; Hong Va Leong; Hossian Shahriar; Michiharu Takemoto; Dave Towey; Hiroki Takakura; Atilla Elci; Susumu Takeuchi; Satish Puri. IEEE, 2019. p. 53-58 8754363 (Proceedings - International Computer Software and Applications Conference).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Assessing the Significant Impact of Concept Drift in Software Defect Prediction

AU - Kabir, Md Alamgir

AU - Keung, Jacky W.

AU - Bennin, Kwabena E.

AU - Zhang, Miao

PY - 2019/7

Y1 - 2019/7

N2 - Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models.

AB - Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models.

KW - Concept drift detection

KW - Defect prediction

KW - Empirical software engineering

KW - Software quality

KW - Streaming data

UR - http://www.scopus.com/inward/record.url?scp=85072713940&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85072713940&origin=recordpage

U2 - 10.1109/COMPSAC.2019.00017

DO - 10.1109/COMPSAC.2019.00017

M3 - RGC 32 - Refereed conference paper (with host publication)

AN - SCOPUS:85072713940

SN - 9781728126074

T3 - Proceedings - International Computer Software and Applications Conference

SP - 53

EP - 58

BT - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019

A2 - Getov, Vladimir

A2 - Gaudiot, Jean-Luc

A2 - Yamai, Nariyoshi

A2 - Cimato, Stelvio

A2 - Chang, Morris

A2 - Teranishi, Yuuichi

A2 - Yang, Ji-Jiang

A2 - Leong, Hong Va

A2 - Shahriar, Hossian

A2 - Takemoto, Michiharu

A2 - Towey, Dave

A2 - Takakura, Hiroki

A2 - Elci, Atilla

A2 - Takeuchi, Susumu

A2 - Puri, Satish

PB - IEEE

T2 - 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019

Y2 - 15 July 2019 through 19 July 2019

ER -

Kabir MA , Keung JW, Bennin KE, Zhang M. Assessing the Significant Impact of Concept Drift in Software Defect Prediction. In Getov V, Gaudiot JL, Yamai N, Cimato S, Chang M, Teranishi Y, Yang JJ, Leong HV, Shahriar H, Takemoto M, Towey D, Takakura H, Elci A, Takeuchi S, Puri S, editors, 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019: Proceedings. IEEE. 2019. p. 53-58. 8754363. (Proceedings - International Computer Software and Applications Conference). doi: 10.1109/COMPSAC.2019.00017

Assessing the Significant Impact of Concept Drift in Software Defect Prediction

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this