Comprehensible Counterfactual Explanation on Kolmogorov-Smirnov Test

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Zicun Cong
  • Lingyang Chu
  • Yu Yang
  • Jian Pei

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)1583-1596
Journal / PublicationProceedings of the VLDB Endowment
Volume14
Issue number9
Online published1 May 2021
Publication statusPublished - May 2021

Conference

Title47th International Conference on Very Large Data Bases, VLDB 2021
LocationHybrid
PlaceDenmark
CityCopenhagen
Period16 - 20 August 2021

Abstract

The Kolmogorov-Smirnov (KS) test is popularly used in many applications, such as anomaly detection, astronomy, database security and AI systems. One challenge remained untouched is how we can obtain an explanation on why a test set fails the KS test. In this paper, we tackle the problem of producing counterfactual explanations for test data failing the KS test. Concept-wise, we propose the notion of most comprehensible counterfactual explanations, which accommodates both the KS test data and the user domain knowledge in producing explanations. Computation-wise, we develop an efficient algorithm MOCHE (for MOst CompreHensible Explanation) that avoids enumerating and checking an exponential number of subsets of the test set failing the KS test. MOCHE not only guarantees to produce the most comprehensible counterfactual explanations, but also is orders of magnitudes faster than the baselines. Experiment-wise, we present a systematic empirical study on a series of benchmark real datasets to verify the effectiveness, efficiency and scalability of most comprehensible counterfactual explanations and MOCHE.

Bibliographic Note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Citation Format(s)

Comprehensible Counterfactual Explanation on Kolmogorov-Smirnov Test. / Cong, Zicun; Chu, Lingyang; Yang, Yu et al.

In: Proceedings of the VLDB Endowment, Vol. 14, No. 9, 05.2021, p. 1583-1596.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review