Software Testing

Activity: Input Space Grammar

(no submission)

Purpose: Applying syntax-based testing to input space to create test cases; get ready to work on homework assignment, and prepare for quiz 5 and the final exam

You may make a copy of a worksheet and complete this activity or type your answers in any text editor.

You may work alone or with at most two other students in this course.

Expand / Collapse all

Consider a simple stack class that has five operations:

Init:     Create a new stack
Push:     Add an item to the stack; the item is placed at the top of the stack
Pop:      Pop an element from the top of the stack, remove that item from the stack
Peek:     Obtain the item present at the top of the stack, does not remove it from the stack
Dispose:  Destroy an existing stack

The following BNF grammar constrains how the stack is used.

A legal use is a sequence of stack operations that is in the language defined by the grammar.
For example, the sequence "Init Push Pop Dispose" is legal.

An illegal use is a sequence of stack operations that is not in the language defined by the grammar.
For example, the sequence "Init Peek" is illegal.

DoIt ::=  X | X DoIt
X    ::=  "Init" "Dispose" | "Init" Y "Dispose"
Y    ::=  "Push" | "Push" Y | "Push" Z | "Push" Y Z
Z    ::=  "Peek" | "Pop" | "Peek" "Pop"

Is it legal to Peek a non-empty stack? Justify your answer by showing a derivation in the grammar.

Yes, it is legal. 

DoIt  ::= X DoIt
      ::= Init Y Dispose
      ::= Init Push Z Dispose
      ::= Init Push Peek Dispose

Give a valid input (strings) that satisfy Terminal Symbol Coverage (TSC)

To satisfy TSC, the input must cover all terminal symbols

TR = { Init, Dispose, Push, Peek, Pop } 

There are many possible answers. A sample valid input would be 
   Init Push Peek Pop Dispose

Give a minimal set of test requirements that satisfy Production Coverage (PDC)

PDC requires that all productions must be covered
There are 11 tests.

TR = { DoIt ::= X
       DoIt ::= X DoIT
       X    ::= "Init" "Dispose"
       X    ::= "Init" Y "Dispose"
       Y    ::= "Push"
       Y    ::= "Push" Y
       Y    ::= "Push" Z
       Y    ::= "Push" Y Z
       Z    ::= "Peek"
       Z    ::= "Pop"
       Z    ::= "Peek" "Pop" }

Consider the following regular expression for bank example and answer the questions

  ("deposit" account amount | "debit" account amount)^*

Draw a Finite State Machine (FSM) to represent the grammar
```
  
```

Create 3 possible input strings from the grammar

deposit 12 $10.50 
debit 0987 $5.111 
deposit 306 20.29 

Note: These tests satisfy edge coverage and node coverage on the FSM. 
However, To design test case inputs, we need to make assumption on 
  - what the account looks like and 
  - what the amount looks like.  
Regular expression is not expressive.

Consider the following Backus-Naur Form (BNF) grammar for bank example and answer the questions

  bank     ::=  action^*
  action   ::=  dep  |  deb
  dep      ::=  "deposit" account amount
  deb      ::=  "debit" account amount
  account  ::=  digit⁴
  amount   ::=  "$" digit⁺ "." digit²
  digit    ::=  "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"

Draw a Finite State Machine (FSM) to represent the grammar
```
  
```
Create 3 possible ground (input) strings from the grammar
```
deposit 4444 $100.50
debit 0987 $5.11 
deposit 3066 $20.29 
```

Apply input grammar mutation operators to create mutants

Nonterminal replacement operator replaces every nonterminal symbol in a production
dep ::= "deposit" account amount by other nonterminal symbols

Based on the given BNF, there are 7 non-terminal symbols:
   bank
   action
   dep
   deb
   account
   amount
   digit
   
To apply this mutation operator, each non-terminal symbol in a given production
is replaced with the other non-terminal symbols. This will results in at least 12 mutants
(since some productions can be repeated). 

Sample mutants are
   dep ::= "deposit" amount amount
   dep ::= "deposit" account digit
   
Test case inputs that can be derived from these mutants are 
   deposit $100.50 $2345.86
   deposit 4444 9

Terminal replacement operator replaces every terminal symbol in a production
amount ::= "$" digit+ "." digit² by other terminal symbols

Based on the given BNF, there are 14 terminal symbols:
   "deposit"
   "debit"
   "$"
   "."
   "0"  "1"  "2"  "3"  "4"  "5"  "6"  "7"  "8"  "9"
   
To apply this mutation operator, each terminal symbol in a given production
is replaced with the other terminal symbols. This will results in 26 mutants.

Sample mutants are
   amount ::= "." digit+ "." digit2
   amount ::= "$" digit+ "$" digit2
   amount ::= "$" digit+ "1" digit2

Test case inputs that can be derived from these mutants are 
   deposit 4444 .2345.00
   deposit 4444 $2345$00
   deposit 4444 $2345100

Terminal and nonterminal deletion operator deletes every terminal and nonterminal symbol in a production
dep ::= "deposit" account amount

Applying this mutation operator results in 3 mutants

Remove every terminal symbol 
   dep ::= account amount
Test case input that can be derived from this mutant is 
   4444 $2345.00   
   
Remove every non-terminal symbol
   dep ::= "deposit" amount   
   dep ::= "deposit" account
Test case input that can be derived from these mutants are
   deposit $2345.00
   deposit 4444    
   
Note: we do first-order mutation -- remove one symbol at a time. 
Each deletion results in a mutant.

Terminal and nonterminal duplication operator duplicates every terminal and nonterminal symbol in a production
dep ::= "deposit" account amount

Applying this mutation operator results in 3 mutants: 

Duplicate every terminal symbol 
   dep ::= "deposit" "deposit" account amount
Test case input that can be derived from this mutant is 
   deposit deposit 4444 $2345.00   
   
Duplicate every non-terminal symbol
   dep ::= "deposit" account account amount  
   dep ::= "deposit" account amount amount
Test case input that can be derived from these mutants are
   deposit 4444 4444 $2345.00
   deposit 4444 $2345.00 $2345.00
   
Note: we do first-order mutation -- duplicate one symbol at a time. 
Each duplication results in a mutant.

Expand / Collapse all