Software Testing

Activity: Candy Tasting Testing

(no submission)

Purpose:

Practice coverage-based test design
Analyze, evaluate, and improve the quality of tests
Analyze and identify infeasible tests
Compare coverage criteria and understand criteria subsumption

For this activity, we will stop periodically, and do group work one step at a time with discussion between steps. You have ~25 minutes to complete this activity.

Form groups of 8-10. To get started, use the two-breath rule, introduce yourself. Get to know everyone in your development and testing team. Work with your team and complete the following tasks.

You may make a copy of a worksheet and complete this activity, or write your answer on paper(s) or your computer.

Task 1: Test Design

You have 10 minutes to complete this task
Form a team of 8-10, each team will get two bags of candies.
Examine bag #1
Imagine you are conducting a "candy testing" — Yes, imagine, don't eat yet .. You will execute your tests later

Discuss in your team, use the worksheet

Define one coverage criterion to test the artifacts (Candies!)
```
example:    C = taste one candy of each texture
```

Develop a set of test requirements that satisfies your criterion

example:   TR = {hard, soft}, where tr1 = hard, tr2 = soft

[Thought question] How were test requirements derived?

Develop a set of test cases that satisfies your test requirements

example:    T = {two sweet tarts, one sour patch}, where t1 = two sweet tarts, t2 = one sour patch, 
                 assuming two sweet tarts are consumed at once. What if one is consumed at a time?

[Thought question] How were test cases designed?

(note: you will trade your answers in task 1 with another team later)

Task 2: Coverage level of your tests

You have ~5 minutes to complete this task
Execute your tests against bag #1
- You will now transform yourself into a "human-PUT"
- For each test case, the "human-PUT"
  - Takes input (candy)
  - Performs a "consume" operation
    - Due to the "no eating in class policy," please imagine the "consume" operation while you are in class — you will do the actual "consume" operation outside of class.
- Let's agree that the Expected output is a normal behavior; i.e., "human-PUT" does not crash

Evaluate your tests. Given your test requirements and your set of test cases,

Record which test requirements are satisfied by your test set

Analyze and compute the coverage level of your test set.

Record the number of test cases that passed

Record the number of failed test cases

Record the number of test cases that are infeasible (make a note, why they are infeasible)

What is the coverage level of your test set?

Task 3: Coverage level of another team's tests

You have ~5 minutes to complete this task
Trade your test design (from task 1) with another team
- If you use an electronic version of the worksheet, you may use this sheet to help you trade your test design with another team
Execute another team's test cases against bag #2
- You will now transform yourself into a "human-PUT"
- For each test case, the "human-PUT"
  - Takes input (candy)
  - Performs a "consume" operation
    - Due to the "no eating in class policy," please imagine the "consume" operation while you are in class — you will do the actual "consume" operation outside of class.
- Let's agree that the Expected output is a normal behavior; i.e., "human-PUT" does not crash

Evaluate your tests. Given your test requirements and another team's set of test cases,

Record which test requirements are satisfied by the test set

Analyze and compute the coverage level of the test set.

Record the number of test cases that passed

Record the number of failed test cases

Record the number of test cases that are infeasible (make a note, why they are infeasible)

What is the coverage level of the test set?

Task 4: Wrap-up questions

You have 5 minutes to complete this task

Is your criterion appropriate? Justify, why?
Is there redundancy in your test set?
Coverage level: Given your criterion, consider the coverage levels of your test set and another team's test set. Are they different? What does it mean if one is higher than another?
Comparing criteria: Compare your coverage criterion and another team’s criterion. Does one subsume another? (Is one a subset of another? Is it a many-to-one or one-to-many mapping? Is it a one-to-one mapping?)