Software Testing

Activity: Program mutation (CountWords)

(no submission)

Purpose: Understand and apply mutation testing concept to program source code; get ready to work on homework assignment, and prepare for quiz 5 and the final exam.

You may make a copy of a worksheet and complete this activity or type your answers in any text editor.

You may work alone or with at most two other students in this course.

Consider the following count() method of the muCountWords class.

public class muCountWords 
{
     /**
      * Counts the number of words in a string that end with either "r" or "t"	
      * @param str -- an incoming sentence
      * @return word_count -- the number of words
      */
      
line  1:   public static int count(String str) 
line  2:   {
line  3:      int word_count = 0;    
line  4:      char last = ' ';      
line  5:    
line  6:      for (int i=0; i<str.length(); i++)
line  7:      {
line  8:         if ( !Character.isLetter(str.charAt(i)) && (last == 'r' || last == 't') )
line  9:            word_count++;
line 10:
line 11:         last = str.charAt(i);
line 12:      }
line 13:      
line 14:      if ( last == 'r' || last == 't' )
line 15:         word_count++;
line 16:      
line 17:      return word_count;       
line 18:   }

}

Create at least one mutation operator that is applicable (and meaningful) to the count() method.
- Write down the name of your mutation operator(s)
- For each operator, briefly describe what it does or how it mutates the program source code
Assume we are considering first-order mutation testing; i.e., only one change per mutant.

Apply your mutation operator(s) to the count() method. How many mutants does your operator create? For each of your mutation operator(s), Write down all mutants generated by the operator.

For example, a Conditional Operator Replacement (COR) replaces each occurrence of
each logical operator (&&, ||, falseOp – replace the entire condition by false, 
and trueOp – replace the entire condition by true) by each of the other operators.

Applying COR to this method will generate 9 mutants:
                 
original: line 8: if ( !Character.isLetter(str.charAt(i)) && (last == 'r' || last == 't') )
  mutant# 1: line 8: if ( !Character.isLetter(str.charAt(i)) || (last == 'r' || last == 't') )
  mutant# 2: line 8: if ( false )
  mutant# 3: line 8: if ( true )
  mutant# 4: line 8: if ( !Character.isLetter(str.charAt(i)) && (last == 'r' && last == 't') )
  mutant# 5: line 8: if ( !Character.isLetter(str.charAt(i)) && false )
  mutant# 6: line 8: if ( !Character.isLetter(str.charAt(i)) && true ) 

original: line 14: if ( last == 'r' || last == 't' )
  mutant# 7: line 14: if ( last == 'r' && last == 't' )
  mutant# 8: line 14: if ( false )
  mutant# 9: line 14: if ( true )

Are any of your mutants "equivalent"?
Are any of your mutants redundant?
Design a set of tests to kill the mutants. Try to satisfy Strong Mutation Coverage.
Download the subject under test (muCountWords.java) and create mutants using your operator(s). For simplicity (no additional software installation / configuration), let's manually mutate the program source code (i.e., inject the fault(s)).
Transform your abstract tests into JUnit test code.
Execute your JUnit tests on the original program. Record the results.
Execute your JUnit tests on the mutants. Compare the results of running the tests on the mutants and the results of running the tests on the original program. Record the number of mutants killed by your set of tests.
Compute the mutation score of your set of tests. If your set of tests does not achieve a 100% mutation score, design more tests. If it is impossible to achieve a 100% mutation score, analyze and discuss why not.

For more hands-on experience, you may explore mutation testing tools such as PIT, Major, or a traditional mutation testing tool, muJava. You will need to set up the mutation testing framework, upload the subject under test, apply the pre-defined mutation operators, upload and execute your JUnit tests, and analyze the results/reports.