- author = "Giovani Guizzo and Federica Sarro and Jens Krinke and Silvia Regina Vergilio",
- title = "Sentinel: A Hyper-Heuristic for the Generation of Mutant Reduction Strategies",
- journal = "IEEE Transactions on Software Engineering",
- year = "2022",
- volume = "48",
- number = "3",
- pages = "803--818",
- month = mar,
- keywords = "genetic algorithms, genetic programming, Grammatical Evolution, SBSE, Mutation Testing, Mutant Reduction, Software Testing, Hyper-Heuristic, Search Based Software Testing, Search Based Software Engineering",
-
DOI = "
doi:10.1109/TSE.2020.3002496",
- size = "16 pages",
-
abstract = "Mutation testing is an effective approach to evaluate
and strengthen software test suites, but its adoption
is currently limited by the mutants execution
computational cost. Several strategies have been
proposed to reduce this cost (a.k.a. mutation cost
reduction strategies), however none of them has proven
to be effective for all scenarios since they often need
an ad-hoc manual selection and configuration depending
on the software under test (SUT).
we propose a novel multi-objective evolutionary hyper-heuristic approach, dubbed Sentinel, to automate the generation of optimal cost reduction strategies for every new SUT. We evaluate Sentinel by carrying out a thorough empirical study involving 40 releases of 10 open-source real-world software systems and both baseline and state-of-the-art strategies as a benchmark for a total of 4800 experiments, which results are evaluated with both quality indicators and statistical significance tests, following the most recent best practice in the literature.
The results show that strategies generated by Sentinel outperform the baseline strategies in 95percent of the cases always with large effect sizes, and they also obtain statistically significantly better results than state-of-the-art strategies in 88percent of the cases with large effect sizes for 95percent of them. Also, our study reveals that the mutation strategies generated by Sentinel for a given software version can be used without any loss in quality for subsequently developed versions in 95percent of the cases. These results show that Sentinel is able to automatically generate mutation strategies that reduce mutation testing cost without affecting its testing effectiveness (i.e. mutation score), thus taking off from the testers shoulders the burden of manually selecting and configuring strategies for each SUT.",
-
notes = "Presented at Facebook TAV 2020 FaceTAV2020
https://fbtavsymposium2020.bevylabs.com/events/details/facebook-tav-symposium-division-facebook-testing-and-verification-symposium-presents-facebook-testing-and-verification-symposium-2020/#/
Department of Computer Science, University College London, 4919 London, London United Kingdom of Great Britain and Northern Ireland WC1E 6BT",
