Fall 2021 — Syllabus

The true subject matter of the tester is not testing, but the design of test cases —Jeff Offutt

Contact information

Instructor: Upsorn Praphamontripong
Office hours: Wednesdays 2:30pm-4:30pm, Thursdays 3:30pm-4:30pm (my office hours will be held online, using Zoom)

Teaching Assistants: Natalie Zhang, Jane Weng, MaryJeanine Seaman, Kelly Schaefer, Samarth Saxena, Brian Tran, Sammy Lahrime
All office locations and hours

We have fantastic teaching assistants for this course. Get to know them!

Our TAs are students too. Let them be students when they are not on the clocks as TAs.
Please only contact TAs via Piazza or in person (or via Zoom) during their scheduled office hours, 
do not contact them via email.
There will also be specific TAs serving as points-of-contact for various aspects of the course. 
If you have a question about any of these aspects, please be sure to contact the right point-of-contact. 
This will facilitate getting your question answered in a more efficient way.  

Class hours

Tuesday and Thursday 2:00pm−3:15pm EST, Room Olsson Hall 120 (This course will be held in-person)

You are the main driver of your learning success. 
Your engagement and participation is the most important aspect of your learning experience. 
You are encouraged to do all the activities, try all concepts, experience everything 
introduced in this course, ... and do beyond the course's minimum requirements. 
The best way to learn software testing is to do it, and do it often. To help you practice  
and maximize hands-on experience, we will do lab-style activities almost every class meeting. 
My current plan is to hold and record synchronous lectures during the official meeting 
times (and you will be able to catch up by viewing the recording). However, you will have 
the best learning experience in the course if you are able to attend the meeting.

Class recordings

My current plan is to record synchronous lectures during the official meeting times and you will be able to catch up by viewing the recording. The recordings will be available under Class Recordings in UVa Collab, accessible by Fall 2021 CS 3250 students and course staff. By default, the recordings will be made available within 24-48 hours after class. If the recordings are unavailable due to technical difficulty or any unavoidable reasons, I will happily work with you to make sure that you do not miss anything.

Per UVA policy on recording of class sessions, students are not allowed to record class sessions / meetings / discussions, regardless of purpose or modality.


Masking

Many of you have raised concern about the risk of being exposed to COVID-19 while attending an in-person class. I, too, am worried. Since my child is not yet old enough to be vaccinated, my comfort level about in-person classes might be different from other instructors'. Therefore, I will practice social distancing and wear a mask in class.

Fall 2021 CS 3250 requires masks to be worn in class for the duration of the class meeting. No eating, no drinking in the classroom. I would deeply appreciate you keeping a physical distance from me. If you have any questions or concerns (before or after class) that are sensitive or personal, the best way to communicate with me is via email.

If you feel uncomfortable, unwell, or have signs of any kinds of illness, please stay home and rest. The most importance is your health and everyone's health. I will happily work with you to make sure that there is no impact on your grade.

Note: During the semester, our plan may change depending on the University's policies. I will formally announce if changes or updates must be made. When possible, we will discuss and decide on a contingency plan; we will get through this together.

If you are mask-free in class for the duration of the class meeting, I will ask you to put on a mask. If you refuse to wear a mask, I must report the violation through the UVA Compliance Reporting portal (refer to SEC-045: COVID-19 Health & Safety Requirements).


References:

Course overview

Software is everywhere. Software defines behavior of services such as transportation systems, medical services, and embedded applications such as spaceships, household appliances, and mobile phones. Society's reliance on software applications places a high demand on their reliability; knowledge of software testing is becoming necessary for all software developers and software engineers. Although many factors affect the engineering of reliable software, including careful design and sound process management, testing is the primary way industry evaluates software during development.

Testing is the most time consuming and expensive part of software development. Not testing is even more expensive. Inappropriate or inadequate testing can cause unexpected behavior, catastrophic software failure, monetary loss, and loss of life.

Have you ever wondered how we should test software? How do we know if we have tested enough? How many tests do we need? When should we stop testing? We write tests to test software, but how do we test our tests?

This course has two closely related themes:

  • First, more than half the effort in software development is devoted to activities related to testing, including test design, execution and evaluation. In this course, you will learn quantitative, technical, practical methods that software engineers and developers can use to test their software.
  • Second, more than half of software development effort is not new development, but maintenance activities such as adding new features, correcting problems, migrating to new platforms, and integrating third-party components into new projects.

These two themes are intertwined because much of the effort during maintenance is testing the changes, and much of the effort in testing is about evaluating changes. This course covers both themes quantitatively, with a solid basis in theory and with practical applications. Along the way, you will (i) learn when, where, and how to test software appropriately and effectively, (ii) have hands-on experience with test generation and test automation, and (iii) see how you can easily apply the concepts to any software testing situation.

Everyone can be successful in this course and my hope is that developing an understanding of the field of software testing will help you build confidence, get excited about exploring and developing new testing approaches, and have a positive impact on your software-related profession.

image showing course goals

Learning objectives

  • Be able to explain the quantitative, technical, practical methods that software engineers and developers can use to test their software
  • Apply testing techniques and criteria to all phases of software development — unit (developer) testing, integration testing, system testing, etc.
  • Be able to analyze and assess the quality of software, and apply theoretical and practical knowledge of test criteria to improve the quality of software
  • Recognize good quantitative programming and design practices to ensure software can be efficiently and effectively modified and tested
  • Realize that maintainability and testability are more important than efficiency for almost all modern software projects
  • Develop research and inquiry-based skills that can be used to learn, connect, and develop software testing phenomena
  • Communicate ideas with others within a group learning environment
image showing course goals

Prerequisite

You must have passed CS 2150 (Program and Data Representation) or CS 2501 topic DSA2 with a grade of C- or better, or COVID CR grade.

Please note: the CS student services staff will monitor and remove students not meeting prerequisites in courses in which they are enrolled.

We will generate tests from mathematical models of the software. You will need knowledge of discrete math (sets, graphs, logic, and grammars), programming, data representation, and general knowledge of software engineering. You will need an understanding of the syntax and semantics of multiple programming languages. Most examples will be in Java and some assignments will require automated tests. Programming skills are required and you are expected to know Java or learn on your own. You are expected to have ability to pace yourself on a semester-long project and be able to attend class regularly.


Textbook

[Recommended]  Introduction to Software Testing (2nd edition), Paul Ammann and Jeff Offutt, Cambridge University Press, 2016. Book website (solution manual)

Where appropriate, additional references that can be relevant for further reading will be provided.

You are expected to read the relevant material (from the text, various sources on the web, and slides and supplements that will be made available on the website). You will understand the class discussion much better if you read the material before class. The class meetings may not cover everything in the readings and will often include material not found in the readings (based on class discussion).


Learning activities

What you will do to succeed in learning?   To facilitate your learning process, the course structure consists of six main activities: hands-on activities, practice-of-the-days, homework assignments, showcase, quizzes, and final exam.

1. Hands-on activities

Five main purposes of this activity are to help you to (i) understand the underlying concepts and apply them to test software, (ii) get started with specific concepts, technologies, tools and frameworks, (iii) get ready to work on homework assignments and showcase, (iv) prepare for quizzes and final exam, and (v) practice skills test engineers need (such as technical skills, software evaluation, collaboration, and communication).

You are encouraged to work in small groups, allowing peer learning and imitating industrial software development and quality assurance scenarios.

Each activity is designed to get you to think about some specific aspect of the material we are learning that day. It helps you to develop competency in basic and advanced concepts. Throughout the course, you will have multiple opportunities to explore test generation tools and test automation frameworks and share your opinion through discussion and writing.

We will do hands-on activities almost every class meeting. They will not be graded.

You are the main driver of your learning success. Your engagement and participation is the most important aspect of your learning experience. Although no submission is required, you are encouraged to do all the activities. If you miss class, you should do the posted activity on your own to learn the material.

Hands-on activities will form the basis for POTDs, homework assignments, quizzes, and final exam.

2. Practice-of-the-days (POTDs)

Five main purposes of this activity are to help you to (i) refine your thoughts and reinforce the concepts, (ii) assess your understanding, (iii) get ready to work on homework assignments and showcase, (iv) prepare for quizzes and final exam, and (v) practice skills test engineers need (such as technical skills, software evaluation, collaboration, and communication).

POTDs help you develop competency in basic and advanced concepts. This activity helps you to reinforce and assess your understanding of the concepts in recent classes. It also gives me an opportunity to work with you on any particular topics or skills that can be improved.

We will do POTDs on most Thursdays and some Tuesdays. Most will be done in small groups and some will be individual exercises. You will submit your POTDs electronically to UVa Collab. They will be graded on a completion basis. Full credit can only be received if it is complete and submitted by 2:00pm EST the next day. Any submission received after this time will be considered to be late. If you miss class, you should do the posted POTD on your own to learn the material.

POTDs will form the basis for homework assignments, quizzes, and final exam.

3. Homework assignments

Homework assignments are the main mechanism to help you learn the fundamentals, apply the concepts, and experience the software testing process. Some homework assignments will be written and some will require programming. Most will allow collaboration with one partner to help you practice team working skills.

Assignments will be posted on the class website and discussed in class. You will submit your homework assignments to UVa Collab. More details on submission guidelines will be provided on the assignment pages.

Because each homework helps you develop competency in basic and advanced concepts, it is important that you manage your time wisely. All homework assignments must be submitted before class on the day they are due − this will also help you focus on the class discussion instead of being worried about the homework. Any submission received after this time will be considered to be late.

You are expected to have a backup disk (or storage somewhere) for every assignment you turn in. In this way, if there is any problem with the copy that is handed in, the backup can be used for grading purposes.

The UVA Honor Code is in effect. Please refer to the Collaboration and Professionalism sections of the course policies.

Homework assignments will form the basis for quizzes and final exam.

4. Showcase

A showcase will provide you the opportunity to explore and experience software testing in practice. There are three options, please choose only one option:
  1. Test generation project — to provide you the opportunity to design and automate tests for a software project
  2. Position paper — to expose you to advanced topics and challenges in software testing and give you the opportunity to learn more about an area of your interest
  3. Digital media — to explore aspects of software testing that interest you and then produce a 5-7 minute movie, photo story, or animation that creatively teaches something about software testing

You may work on the showcase individually or with one partner. Descriptions will be provided on the showcase page.

5. Quizzes

We will have five quizzes, approximately every two weeks, and no midterm exam. The goals of the quizzes are to reinforce the concepts in recent classes and assess your understanding of recent topics / skills. This also gives me an opportunity to work with you on any particular topics or skills that can be improved.

Quizzes will generally cover material from previous classes. Quiz guidelines will be posted on the Schedule page.

You will take quizzes through UVa Collab, during the first 15 minutes of class on Tuesdays (and some Thursdays) and no make-up or late quizzes will be given. This is to ensure that the quizzes are available to everyone at the same period of time. If you attend class in person, please be sure to have your laptop ready. If you concern about the risk of being exposed to COVID-19, you can take the quizzes through Collab during the scheduled quiz time without being physically present in the classroom.

Quizzes are open book / notes / class recordings, no collaboration. You should study before the scheduled quiz time. Fifteen minutes are not a lot of time; you should focus on solving problems and answering questions instead of opening or reviewing materials.

To encourage you to relearn what you have not yet mastered, you may submit a reflection and correction. My hope is that you will review the materials and / or seek help if needed to ensure that you understand, and can appropriately apply the testing concepts and analyze the quality of your tests. You will have an opportunity to reflect on and correct each quiz to receive up to 50% of the points that you lose on the quiz. The quiz reflection and correction must be submitted within three days of the graded quiz being returned to you. The due date will be announced on the Schedule page.

Quizzes will form the basis for the final exam.

6. Final Exam

We will have a take-home final exam. The exam is comprehensive, open book / notes / web / IDE, no collaboration. Details will be provided on the final exam guide.

If you know in advance that you are unable to make an exam for a valid and unavoidable reason (such as a scheduled surgery, etc), you must notify me at least one week before the scheduled exam date to make arrangements for a make-up. Appropriate documentation (as determined by instructor) and requisite permissions (approved by the Dean's Office, in particular) are required for make-up exam requests. If you miss an exam due to a university-accepted excused absence, you must notify me within 24 hours of your absence to make arrangements for a make-up, and submit approved documentation. Failure to follow either of these policies will result in a zero grade on the exam.

 1. Set aside time and space for your studies. 
    Once you have selected that time and space, try to stick with your plan. 
    Pick a comfortable spot, as much as possible, don’t use that time or place for anything else.
    
 2. Make friends. Build our Software Testing community. 
    When possible, form a study/work group for the class.
       
 3. Attend class meeting. 
    Attending class meeting will help to establish your routine, put you in the learning mindset, 
    and remind you that you are learning in a community.
    
 4. Everyone brings diverse experiences, backgrounds, and opinions.  
    Let's learn, grow, and achieve our goals together.

Help yourself learn

Please feel free to share your ideas / opinions / suggestions on things that we should do or try to help you learn software testing, using the Help yourself learn form.


Assessment

How will your learning be evaluated?   The grade you will earn for this course will be based on how well you demonstrate your learning and understanding.

Assessment and evaluation

  • Practice-of-the-days: 15%
  • Homework assignments: 30%
  • Showcase: 20% (choose one of the three options)
  • Quizzes: 15%
  • Final exam: 20% (comprehensive)

Your overall score will be mapped to a letter grade as follows:

You get If you score
A+ >= 98
A   >= 93
A-  >= 90
B+ >= 87
B   >= 83
B-  >= 80
C+ >= 77
C   >= 73
C-  >= 70
D+ >= 67
D   >= 63
D-  >= 60
F    < 60

Students enrolled in pass/fail require a 65.0 or higher to pass.

By default, grades will not be rounded or curved in this course

Cheating / Plagiarism / Professionalism Penalty — up to 100% penalty
Excessive missed classes, rude behavior toward course staff (instructor and TAs) or classmates, unauthorized homework assistance, contacting TAs when they are not on the clock, etc., can be held against a student when final grades are calculated.


Discussion board

We will use Piazza for a discussion board. In addition to in-person assistance, this is the place where you can seek help, offer help, share your thoughts and discoveries, or discuss technical difficulties and potential troubleshooting. Information for accessing our class Piazza site will be provided on the first day of class. Ask all technical questions about the materials, activities, POTDs, homework assignments, or showcase on Piazza.

Posting program source code publicly will usually be considered an honor code violation. If you need to include a "fragment" of your code in your post, please make your post "private" and tag instructor.

Updates to the course web page will generally be noted on the Piazza page.

You may post any reasonable class-related topics.
  • Real-life software failures: Start a discussion about a real-life software failure. Tell us what happened and how it affected users.
  • Real-life troubleshooting: Start a discussion about your experience, tutorial, suggestion, and troubleshooting that is related to our class
  • Application: Describe how one of the ideas from our testing class could have found a real-life software failure before the software was released.
  • Start a thread about an error in the handouts or slides
  • Start a thread about a topic that goes beyond what we discuss in class.
  • Post questions to an appropriate thread and they will be answered by instructor, TAs, or classmates.
  • Give an answer to a classmate's question.

Academic integrity

The School of Engineering and Applied Science relies upon and cherishes its community of trust. We firmly endorse, uphold, and embrace the University’s Honor principle that students will not lie, cheat, or steal, nor shall they tolerate those who do. We recognize that even one honor infraction can destroy an exemplary reputation that has taken years to build. Acting in a manner consistent with the principles of honor will benefit every member of the community both while enrolled in the Engineering School and in the future.

Students are expected to be familiar with the university honor code, including the section on academic fraud. Each assignment will describe allowed collaborations, and deviations from these will be considered Honor violations. If you are in doubt regarding the requirements, please consult with me before you complete any requirement of this course. Unless otherwise noted, exams, quizzes, and individual assignments will be considered pledged that you have neither given nor received help. Among other things, this means that you are not allowed to describe problems on an exam (or a quiz) to a student who is currently taking or has not taken it yet. You are not allowed to show exam (or quiz) papers (or copies) to another student or view another student’s exam (or quiz) papers while working on an exam (or a quiz). Sending, receiving or otherwise copying electronic files that are part of course assignments are not allowed collaborations (except for those explicitly allowed in assignment instructions).

Assignments or exams where honor infractions or prohibited collaborations occur will receive a zero grade for that entire assignment or exam, as well as a full letter-grade penalty on the course grade. Such infractions will also be submitted to the Honor Committee if that is appropriate. Students who have had prohibited collaborations may not be allowed to work with partners on remaining homeworks.


SDAC and other special circumstances

The University of Virginia strives to provide accessibility to all students. If you require an accommodation to fully access this course, please contact the Student Disability Access Center (SDAC) at 434-243-5180 or sdac@virginia.edu. If you are unsure if you require an accommodation, or to learn more about their services, you may contact the SDAC at the number above or by visiting their website at https://www.studenthealth.virginia.edu/sdac.

If you have been identified as an SDAC student, please let the Center know you are taking this class. If you suspect you should be an SDAC student, please schedule an appointment with them for an evaluation. Students who need academic accommodations should see me and contact the SDAC. All academic accommodations must be arranged through the SDAC.

If you need counseling services, please contact CAPS.

If you have other special circumstances (athletics, other university-related activities, etc.), please contact instructor and/or TA as soon as you know these may affect you in class.


Religious accommodations

It is the University's long-standing policy and practice to reasonably accommodate students so that they do not experience an adverse academic consequence when sincerely held religious beliefs or observances conflict with academic requirements.

Students who wish to request academic accommodation for a religious observance should submit their request in writing directly to me by email as far in advance as possible. Students and instructors who have questions or concerns about academic accommodations for religious observance or religious beliefs may contact the University's Office for Equal Opportunity and Civil Rights (EOCR) at UVAEOCR@virginia.edu or 434-924-3200.


Safe environment

The University of Virginia is dedicated to providing a safe and equitable learning environment for all students. To that end, it is vital that you know two values that I and the University hold as critically important:

  1. Power-based personal violence will not be tolerated.
  2. Everyone has a responsibility to do their part to maintain a safe community on Grounds.

If you or someone you know has been affected by power-based personal violence, more information can be found on the UVA Sexual Violence website that describes reporting options and resources available - https://eocr.virginia.edu/.

As your instructor and as a person, know that I care about you and your well-being and stand ready to provide support and resources as I can. As a faculty member, I am a responsible employee, which means that I am required by University policy and federal law to report what you tell me to the University's Title IX Coordinator. The Title IX Coordinator's job is to ensure that the reporting student receives the resources and support that they need, while also reviewing the information presented to determine whether further action is necessary to ensure survivor safety and the safety of the University community. If you would rather keep this information confidential, there are Confidential Employees you can talk to on Grounds (See https://justreportit.virginia.edu/). The worst possible situation would be for you or your friend to remain silent when there are so many here willing and able to help.


Acknowledgments:
Thanks to Professor Jeff Offutt and Professor Paul Ammann (GMU); and Mr. Corey Vaudo and Mr. Conrad Rybka (Mastercard) for the input and material provided for this course. Thanks to Adriana Streifer and Brian Helmke for suggestions on course redesign and improvement.

Copyright © 2021 Upsorn Praphamontripong

Released under the Creative Commons License CC-BY-NC-SA 4.0 license.

Last updated 2021-08-06 15:26
  Top