Software Testing

Activity: IDM and ISP criteria — GenericStack

(no submission)

Purpose:

Understand and practice ISP coverage criteria
Get ready to work on assignments
Prepare for quizzes and the final exam

You may make a copy of a worksheet and complete this activity, or simply type your answers in any text editor.

You may work alone or with another student in this course (max team size = 2).

Refer to the IDM GenericStack activity. Complete the following tasks. Use your IDMs to design tests

Apply All Combination Coverage (ACoC)

Compute the (maximum) number of tests
sample solution
```
#tests = 3 * 2 * 2 * 2 = 24
```

Derive test requirements

sample solution

For simplicity, let's rename the blocks in Task 1, question 3.
   C1: [C11, C12, C13]
   C2: [C21, C22]
   C3: [C31, C32]
   C4: [C41, C42]

TR = { 
   (C11,C21,C31,C41), (C11,C21,C31,C42), (C11,C21,C32,C41), (C11,C21,C32,C42), 
   (C11,C22,C31,C41), (C11,C22,C31,C42), (C11,C22,C32,C41), (C11,C22,C32,C42),

   (C12,C21,C31,C41), (C12,C21,C31,C42), (C12,C21,C32,C41), (C12,C21,C32,C42), 
   (C12,C22,C31,C41), (C12,C22,C31,C42), (C12,C22,C32,C41), (C12,C22,C32,C42),
   
   (C13,C21,C31,C41), (C13,C21,C31,C42), (C13,C21,C32,C41), (C13,C21,C32,C42), 
   (C13,C22,C31,C41), (C13,C22,C31,C42), (C13,C22,C32,C41), (C13,C22,C32,C42)  }

Analyze and remove any infeasible test requirements

   C11: an empty stack, C21: stack contains null  
     -- conflict, thus remove all test requirements that have C11 and C21
      
   C11: an empty stack, C41: X appears in the stack
     -- conflict, thus remove all test requirements that have C11 and C41
    
   C12: stack of size 1, C21: stack contains null, C32: X is not null, C41: X is in stack
     -- cannot have a stack of size 1 that contains a null entry, X in a stack but X is not null,
        thus remove a test requirement (C12, C21, C32, C41)
        
   C12: stack of size 1, C22: stack does not contain null, C31: X is null, C41: X is in stack
     -- cannot have a stack of size 1 that does not contain a null entry, but X is null and is in a stack,
        thus remove a test requirement (C12, C22, C32, C41)
        
   C21: stack contains null, C31: X is null, C42: X is not in stack
     -- if a stack contains a null, X is null, X must appear in a stack
        thus remove all test requirements with C21, C31, C42
     
   C22: stack does not contain null, C31: X is null, C41: X is in stack
     -- cannot have a stack that does not contain null, but X is null and is in a stack
        thus remove all test requirements with C22, C31, C41  
     
TR = { 
   (C11,C21,C31,C41), (C11,C21,C31,C42), (C11,C21,C32,C41), (C11,C21,C32,C42), 
   (C11,C22,C31,C41), (C11,C22,C31,C42), (C11,C22,C32,C41), (C11,C22,C32,C42),

   (C12,C21,C31,C41), (C12,C21,C31,C42), (C12,C21,C32,C41), (C12,C21,C32,C42), 
   (C12,C22,C31,C41), (C12,C22,C31,C42), (C12,C22,C32,C41), (C12,C22,C32,C42),
   
   (C13,C21,C31,C41), (C13,C21,C31,C42), (C13,C21,C32,C41), (C13,C21,C32,C42), 
   (C13,C22,C31,C41), (C13,C22,C31,C42), (C13,C22,C32,C41), (C13,C22,C32,C42)  }

Substitute test input values

sample solution

Use the values defined in Task 1, question 4. Revise and update the values as needed. 
(C11,C22,C31,C42) --> (X=null, stack=[])   
(C11,C22,C32,C42) --> (X="cat", stack=[]) 
(C12,C21,C31,C41) --> (X=null, stack=[null])
(C12,C21,C32,C42) --> (X="cat", stack=[null])
(C12,C22,C31,C42) --> (X=null, stack=["cat"])
(C12,C22,C32,C41) --> (X="cat", stack=["cat"]) 
(C12,C22,C32,C42) --> (X="dog", stack=["cat"])
(C13,C21,C31,C41) --> (X=null, stack=["cat", null])
(C13,C21,C32,C41) --> (X="cat", stack=["cat", null])
(C13,C21,C32,C42) --> (X="dog", stack=["cat", null])
(C13,C22,C31,C42) --> (X=null, stack=["cat", "dog"])
(C13,C22,C32,C41) --> (X="cat", stack=["cat", "dog"])
(C13,C22,C32,C42) --> (X="bird", stack=["cat", "dog"])

Apply Each Choice Coverage (ECC)

Compute the (maximum) number of tests
sample solution
```
#tests = max_#blocks = 3
```

Derive test requirements

sample solution

TR = { (C11,C22,C32,C42), (C12,C21,C31,C41), (C13,C22,C32,C42) }

Substitute test input values

sample solution

(C11,C22,C32,C42) --> (X="cat", stack=[])
(C12,C21,C31,C41) --> (X=null, stack=[null])
(C13,C22,C32,C42) --> (X="bird", stack=["cat", "dog"])

Apply Pair-Wise Coverage (PWC)

Compute the (maximum) number of tests
sample solution
```
#test = 1st_max_#block * 2nd_max_#block = 3 * 2 = 6
```

Derive test requirements

sample solution


TR = { (C11,C21,C31,C41), (C11,C22,C32,C42), (C12,C21,C32,C41), 
       (C12,C22,C31,C42), (C13,C21,C31,C42), (C13,C22,C32,C41) }
       
Analyze and remove any infeasible test requirements 

    C11: an empty stack, C21: stack contains null  
      -- conflict, thus remove all test requirements that have C11 and C21
         
    C12: stack of size 1, C21: stack contains null, C32: X is not null, C41: X is in stack
      -- cannot have a stack of size 1 that contains a null entry, X in a stack but X is not null,
        thus remove a test requirement (C12, C21, C32, C41)     
        
    C21: stack contains null, C31: X is null, C42: X is not in stack
      -- if a stack contains a null, X is null, X must appear in a stack
        thus remove all test requirements with C21, C31, C42    
            
TR = { (C11,C21,C31,C41), (C11,C22,C32,C42), (C12,C21,C32,C41), 
       (C12,C22,C31,C42), (C13,C21,C31,C42), (C13,C22,C32,C41) }

Substitute test input values

sample solution

(C11,C22,C32,C42) --> (X="cat", stack=[]) 
(C12,C22,C31,C42) --> (X=null, stack=["cat"])
(C13,C22,C32,C41) --> (X="cat", stack=["cat", "dog"])

Apply Base Choice Coverage (BCC)

Compute the (maximum) number of tests

sample solution

#tests = 1(for #base) + ( (3-1) + (2-1) + (2-1) + (2-1) ) = 6

Derive test requirements

sample solution

First, pick a base. Suppose a base is (C13,C22,C32,C41)

Derive the remaining test requirements. 
Hold all but one base choice constant, 
use each non-base choice in each other characteristic.

C13,C22,C32,C42
C13,C22,C31,C41
C13,C21,C32,C41
C12,C22,C32,C41
C11,C22,C32,C41

TR = { (C13,C22,C32,C41), (C13,C22,C32,C42), (C13,C22,C31,C41),
             (C13,C21,C32,C41), (C12,C22,C32,C41), (C11,C22,C32,C41) }   

Analyze and remove any infeasible test requirements
             
    C11: an empty stack, C41: X appears in the stack
      -- conflict, thus remove all test requirements that have C11 and C41
      
    C22: stack does not contain null, C31: X is null, C41: X is in stack
      -- cannot have a stack that does not contain null, but X is null and is in a stack
        thus remove all test requirements with C22, C31, C41   
      
TR = { (C13,C22,C32,C41), (C13,C22,C32,C42), (C13,C22,C31,C41),
       (C13,C21,C32,C41), (C12,C22,C32,C41), (C11,C22,C32,C41) }

Substitute test input values

sample solution

(C13,C22,C32,C41) --> (X="cat", stack=["cat", "dog"])
(C13,C22,C32,C42) --> (X="bird", stack=["cat", "dog"]) 
(C13,C21,C32,C41) --> (X="cat", stack=["cat", null])
(C12,C22,C32,C41) --> (X="cat", stack=["cat"])

Apply Multiple Base Choice Coverage (MBCC)

Compute the (maximum) number of tests

sample solution

Suppose there are 2 base tests: (C13,C22,C32,C41) and (C12,C22,C32,C42)
#tests = #base + ( #base*(3-#base_choiceC1) + #base*(2-#base_choiceC2) + 
                   #base*(2-#base_choiceC3) + + #base*(2-#base_choiceC4) )
       = 2 + ( 2*(3-2) + 2*(2-1) + 2*(2-1) + 2*(2-2) ) = 8

Derive test requirements

sample solution

Hold all but one base choice constant for each base test, 
use each non-base choice in each other characteristic.

For base test (C13,C22,C32,C41)
C13,C22,C32,--   Cannot replace C4 characteristic (all block are used as base choice)
C13,C22,C31,C41
C13,C21,C32,C41
C11,C22,C32,C41


For base test (C12,C22,C32,C42)
C12,C22,C32,--   Cannot replace C4 characteristic (all block are used as base choice)
C12,C22,C31,C42
C12,C21,C32,C42
C11,C22,C32,C42


TR = { (C13,C22,C32,C41), (C13,C22,C31,C41), (C13,C21,C32,C41), (C11,C22,C32,C41),
       (C12,C22,C32,C42), (C12,C22,C31,C42), (C12,C21,C32,C42), (C11,C22,C32,C42) }    
       
Analyze and remove any infeasible test requirements

    C22: stack does not contain null, C31: X is null, C41: X is in stack
      -- cannot have a stack that does not contain null, but X is null and is in a stack
        thus remove all test requirements with C22, C31, C41         
        
    C11: an empty stack, C41: X appears in the stack
      -- conflict, thus remove all test requirements that have C11 and C41     
        
TR = { (C13,C22,C32,C41), (C13,C22,C31,C41), (C13,C21,C32,C41), (C11,C22,C32,C41),
       (C12,C22,C32,C42), (C12,C22,C31,C42), (C12,C21,C32,C42), (C11,C22,C32,C42) }

Substitute test input values

sample solution

(C13,C22,C32,C41) --> (X="cat", stack=["cat", "dog"])
(C13,C21,C32,C41) --> (X="cat", stack=["cat", null])
(C12,C22,C32,C42) --> (X="dog", stack=["cat"])
(C12,C22,C31,C42) --> (X=null, stack=["cat"])
(C12,C21,C32,C42) --> (X="cat", stack=[null])
(C11,C22,C32,C42) --> (X="cat", stack=[])