An empirical evaluation of several test-a-few strategies for testing particular conditions

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

3 Scopus Citations
View graph of relations


Related Research Unit(s)


Original languageEnglish
Pages (from-to)967-994
Journal / PublicationSoftware - Practice and Experience
Issue number8
Publication statusPublished - Aug 2012


Existing specification-based testing techniques often generate comprehensive test suites to cover diverse combinations of test-relevant aspects. Such a test suite can be prohibitively expensive to execute exhaustively because of its large size. A pragmatic strategy often adopted in practice, called test-once strategy, is to identify certain particular conditions from the specification and to test each such condition once only. This strategy is implicitly based on the uniformity assumption that the implementation will process a particular condition uniformly, regardless of other parameters or inputs. As the decision of adopting the test-once strategy is often based on the specification, whether the uniformity assumption actually holds in the implementation needs to be critically assessed, or else the risk of inadequate testing could be non-negligible. As viable alternatives to reduce such a risk, a family of test-a-few strategies for the testing of particular conditions is proposed in this paper. Two rounds of experiments that evaluate the effectiveness of the test-a-few strategies as compared with the test-once strategy are further reported. Our experiments do the following: (1) provide clear evidence that the uniformity assumption often, but not always, holds and that the assumption usually fails to hold when the implementation is faulty; (2) demonstrate that all our proposed test-a-few strategies are statistically more reliable than the test-once strategy in revealing faulty programs; (3) show that random sampling is already substantially more effective than the test-once strategy; and (4) indicate that, compared with other test-a-few strategies under study, choice coverage seems to achieve a better trade-off between test effort and effectiveness. Copyright © 2011 John Wiley & Sons, Ltd. Copyright © 2011 John Wiley & Sons, Ltd.

Research Area(s)

  • particular condition, software testing, software validation, test case selection, test-a-few strategy, test-once strategy