Affiliations: [a] Vietnam-Korea Friendship Information Technology College, Vietnam | [b] Faculty of Computer Science and Management, Wroclaw University of Science and Technology, Wroclaw, Poland
Correspondence:
[*]
Corresponding author. Lech Madeyski, Faculty of Computer Science and Management, Wroclaw University of Science and Technology, Wyb.Wyspianskiego 27, 50370 Wroclaw, Poland. Tel.: +48 71 320 2886; Fax: +48 71 321 1018; E-mail: lech.madeyski@pwr.edu.pl.
Abstract: Traditional mutation testing is a powerful technique to evaluate the quality of test suites. Unfortunately, it is not yet widely used due to the problems of a large number of generated mutants, limited realism (mutants not necessarily reflect real software defects), and equivalent mutants problem. Higher order mutation (HOM) testing has been proposed to overcome these limitations of first order mutation testing. We present an empirical evaluation of our approach to higher order mutation testing. We apply different multi-objective optimization algorithms (including one modified by us), as well as our classification of HOMs, proposed objectives and fitness functions. We search for “High Quality and Reasonable HOMs” able to replace all of its constituent FOMs without scarifying test effectiveness and to reflect complex defects requiring more than one change to correct them. Our approach leads to: 1) reduced cost of mutation testing due to reduced number of HOMs, 2) harder to kill mutants (which mimic harder to find defects), 3) reduced cost of mutation testing as it does not waste resources for creating easy-to-kill mutants. Furthermore, we establish a relevant upper bound on mutation order in higher order mutation testing and thus reduce the cost of mutation even further.
Keywords: Mutation testing, higher order mutation testing, higher order mutants, multi-objective optimization algorithm, upper bound order
