Software Testing

Activity: Program mutation (Good-Fast-Cheap)

(no submission)

Purpose: Understand and apply mutation testing concept to program source code; get ready to work on homework assignment, and prepare for quiz 5 and the final exam.

You may make a copy of a worksheet and complete this activity or type your answers in any text editor.

You may work alone or with at most two other students in this course.

Consider (again) an implementation of the old engineering joke: Good, Fast, Cheap.

public final class GoodFastCheap 
{
   // boolean variables good, fast, and cheap 
   // and other stuff omitted
    
   public boolean isSatisfactory() 
   {
      if ((good && fast) || (good && cheap) || (fast && cheap)) 
         return true;
      return false;
   }

   public boolean isSatisfactoryRefactored() 
   {
      if (good && fast)     return true;
      if (good && cheap)    return true;
      if (fast && cheap)    return true;
      return false;
   }
}

Consider two mutation operators, one that replaces boolean variables with the constant "true" and one that replaces boolean variables with the constant "false". (Reminder: we are considering first-order mutation testing; i.e., only one change per mutant)

Expand / Collapse all

How many mutants do these operator generates for
- isSatisfactory()?
```
12 mutants 
```
- isSatisfactoryRefactored()?
```
12 mutants 
```

Are any of these mutants "equivalent"? What is it about these mutation operators that makes this analysis easy?

No equivalent mutants.
The operators replace the boolean variable with the constant "true" and "false."
Thus, we can use short circuit to the mutant and 
compare the simplified predicate with the original predicate to determine 
if they are functionally equivalent.

Also, notice that all mutants can be killed by at least one test. 
Thus, no equivalent mutant.

Some of these mutants are bound to be redundant. How many? Why?

isSatisfactory()     -- 3 redundant mutants   
For predicate P = (g & f) | (g & c) | (f & c)                   
    m2: ( F & f) | (g & c) | (f & c)    and     m4: ( g & F) | (g & c) | (f & c)         
    m6: ( g & f) | (F & c) | (f & c)    and     m8: ( g & f) | (g & F) | (f & c)                  
   m10: ( g & f) | (g & c) | (F & c)    and    m12: ( g & f) | (g & c) | (f & F)          
     
isSatisfactoryRefactored()     -- 3 redundant mutants
For predicate P1 = (g & f)      
   m14: ( F & f)    and    m16: ( g & F)
For predicate P2 = (g & c)
   m18: ( F & c)    and    m20: ( g & F)         
For predicate P3 = (f & c)   
   m22: ( F & c)    and    m24: ( f & F)

Consider a pair of corresponding mutants, one in each method. Will these mutants be killed by exactly the same tests? Is this a good thing?
```
Yes 
```

For each mutant, find all the tests that kill that mutant

isSatisfactory()                        killed by 
   m1: ( T & f) | (g & c) | (f & c)            t6   (test case inputs: FTF)
   m2: ( F & f) | (g & c) | (f & c)            t2   (test case inputs: TTF)
   m3: ( g & T) | (g & c) | (f & c)            t4   (test case inputs: TFF)
   m4: ( g & F) | (g & c) | (f & c)            t2   (test case inputs: TTF)
   m5: ( g & f) | (T & c) | (f & c)            t7   (test case inputs: FFT)
   m6: ( g & f) | (F & c) | (f & c)            t3   (test case inputs: TFT)
   m7: ( g & f) | (g & T) | (f & c)            t4   (test case inputs: TFF)
   m8: ( g & f) | (g & F) | (f & c)            t3   (test case inputs: TFT)
   m9: ( g & f) | (g & c) | (T & c)            t7   (test case inputs: FFT)
  m10: ( g & f) | (g & c) | (F & c)            t5   (test case inputs: FTT)
  m11: ( g & f) | (g & c) | (f & T)            t6   (test case inputs: FTF)
  m12: ( g & f) | (g & c) | (f & F)            t5   (test case inputs: FTT)
    
isSatisfactoryRefactored()                   killed by
r(p1) = true
  m13: ( T & f)                                t5, t6   (test case inputs: FTT, FTF)
  m14: ( F & f)                                t2       (test case inputs: TTF)
  m15: ( g & T)                                t3, t4   (test case inputs: TFT, TFF)
  m16: ( g & F)                                t2       (test case inputs: TTF)
r(p2) = r(p1) & !p1, thus f = F  
  m17: ( T & c)                                t7       (test case inputs: FFT)
  m18: ( F & c)                                t3       (test case inputs: TFT)
  m19: ( g & T)                                t4       (test case inputs: TFF)
  m20: ( g & F)                                t3       (test case inputs: TFT)
r(p3) = r(p2) & !p2, thus g = F  
  m21: ( T & c)                                t7       (test case inputs: FFT)
  m22: ( F & c)                                t5       (test case inputs: FTT)
  m23: ( f & T)                                t6       (test case inputs: FTF)
  m24: ( f & F)                                t5       (test case inputs: FTT)

Consider the RACC-adequate tests you designed for the original isSatisfactory() and the refactored version of the isSatisfactory() methods (from Activity: Logic coverage for source code (gfc)).

From Activity: Logic coverage for source code (gfc)

The original isSatisfactory()
RACC-adequate test = { 2, 3, 4, 6 }           // let's call it RACC-test1  

The refactored version of the isSatisfactory()
RACC-adequate test = { 2, 3, 4, 5, 6, 7 }     // let's call it RACC-test2

Compute the mutation score for RACC-test1 on isSatisfactory()

mutation score for RACC-test1 on isSatisfactory() = 8/12

Compute the mutation score for RACC-test2 on isSatisfactoryRefactored()

mutation score for RACC-test2 on isSatisfactoryRefactored() = 12/12

Expand / Collapse all

You might find the following table useful. You should add more columns as needed.

  p = ((g && f) || (g && c) || (f && c))

Row

Original

m10

m11

m12

  p1 = (g && f) 
  p2 = (g && c) 
  p3 = (f && c)

Row

Original

mutants of p1

mutants of p2

mutants of p3

m13

m14

m15

m16

m17

m18

m19

m20

m21

m22

m23

m24