Software Testing

Activity: Coverage-based test design

(no submission)

Purpose:

Understand the concept of coverage-based test design
Practice coverage-based test design
Analyze and evaluate the quality of tests

Instruction:

You may make a copy of a worksheet and complete this activity, or simply type your answers in any text editor.

You may work alone or with at most two other students in this course.

Consider the following code snippet

public static int numZero(int[] x) 
{
   // Effects: if x == null throw NullPointerException
   // else return the number of occurrences of 0 in x
   int count = 0;                          // line 1 
   for (int i=0; i<x.length; i++)          // line 2 
   {
      if (x[i] == 0)                       // line 3 
         count++;                          // line 4
   }
   return count;                           // line 5  
}

Suppose a line coverage criterion (which requires that all lines must be covered) is used. Develop a set of test requirements that satisfies line coverage criterion.
Suppose a branch coverage criterion (which requires that all branches must be covered) is used. Develop a set of test requirements that satisfies branch coverage criterion.
Suppose there exists a test set T = { x=[5], x=[1,2,3], x=[] }
where test case t1 has test input x=[5], t2 has test input x=[1,2,3], and t3 has test input x=[].

Compute the coverage level of test set T on line coverage criterion (question 1).
Suppose there exists a test set T = { x=[5], x=[1,2,3], x=[] }
where test case t1 has test input x=[5], t2 has test input x=[1,2,3], and t3 has test input x=[].

Compute the coverage level of test set T on branch coverage criterion (question 2).
Consider line coverage criterion and branch coverage criterion on the numZero() method, does one subsume another?
[Thought question] Design your own coverage criterion to test the numZero() method. Describe your criterion. Give a name of your coverage criterion. Then, specify what must be covered. For example, "A line coverage criterion requires that all lines must be covered."
[Thought question] Develop a set of test requirements that satisfies your coverage criteria.
[Thought question] Suppose there exists a test set T = { x=[5], x=[1,2,3], x=[] }
where test case t1 has test input x=[5], t2 has test input x=[1,2,3], and t3 has test input x=[].

Compute the coverage level of test set T on your coverage criterion.
[Thought question] Does your coverage criterion subsume a line coverage criterion? Does a line coverage criterion subsume your coverage criterion? How do you know if one subsumes another? What does it mean if one criterion subsumes another criterion? Which criterion is stronger?
[Thought question] Suppose there is a fault in the numZero() method and suppose that Humpty's coverage criterion subsumes a line coverage criterion.

Let T1 be a set of tests that satisfies a line coverage criterion and T2 be a set of tests that satisfies Humpty's coverage criterion.

If T1 reveals the fault, can we conclude that T2 also reveals the fault? Justify.