CS 4501/6501: Software Testing
Syllabus — Fall 2017
The true subject matter of the tester is not testing,
but the design of test cases —Jeff Offutt
Syllabus Schedule Resources

Upsorn Praphamontripong
Office: Rice Hall 206
Email: upsorn
URL: http://www.cs.virginia.edu/~up3f/swtesting/syllabus.html
Office Hours:    Tuesday 3:30pm-4:30pm, Wednesday 10am-12pm, Thursday 3:30pm-4:30pm
TA contact
Weicheng Chao, Monday 10am-12pm, Rice 504
Class hours Tuesday, Thursday 11:00AM - 12:15PM, Rice Hall 340
Prerequisite Knowledge of discrete math, programming, data structures and general knowledge of software engineering. You will need an understanding of the syntax and semantics of multiple programming languages. Most examples will be in Java and some assignments will require automated tests. Programming skills are required and students are expected to know Java or learn on their own.
Text Required: Introduction to Software Testing (2nd edition), Paul Ammann and Jeff Offutt, Cambridge University Press, 2016. Book website

Where appropriate, additional references that can be relevant for further reading will be provided.
Overview This course presents the concepts and techniques for testing software and assuring its quality. Topics cover software testing at the unit, module, subsystem, and system levels, automatic and manual techniques for generating and validating test data, the testing process, static vs. dynamic analysis, functional testing, inspections, web application testing, and reliability assessment.

Over the semester, students will learn when, where, and how to test software appropriately and effectively.

CS 4501 vs CS 6501: Homework assignments and exams in this course will be given as problem solving questions drawn from the Introduction to Software Testing textbook (by Ammann and Offutt) and other assigned reading materials. Each problem is defined by its level of difficulty - fundamental, intermediate, and advanced. Graduate students will be required to solve advanced problems. Extra credit opportunities will be offered to those who choose to complete a short research paper critique presentation. Undergraduates will be expected to solve intermediate problems and those who also choose to solve advanced problems will be given extra credit opportunities.
Learning outcomes
  • Knowledge of quantitative, technical, practical methods that software engineers and developers can use to test their software
  • Testing techniques and criteria for all phases of software development - unit (developer) testing, integration testing, system testing, etc.
  • Theoretical and practical knowledge of how to apply test criteria to improve the quality of software
  • Understanding of best quantitative programming and design practices for ensuring software can be efficiently and effectively modified and tested
  • Understanding that maintainability and testability are more important than efficiency for almost all modern software projects
Content This course has two closely related themes. First, more than half the effort in software development is devoted to activities related to testing, including test design, execution and evaluation. This course will teach quantitative, technical, practical methods that software engineers and developers can use to test their software, both during and at the end of development. Second, more than half of software development effort is not new development, but maintenance activities such as adding new features, correcting problems, migrating to new platforms, and integrating third-party components into new projects. These two themes are intertwined because much of the effort during maintenance is testing the changes, and much of the effort in testing is about evaluating changes.

This course covers these two themes quantitatively, with a solid basis in theory and with practical applications. These topics will be useful to strong programmers in the Computer Science program, as well as engineers, physical scientists, and mathematicians who regularly integrate software components as part of their work. The topic of this course is of interest to and accessible to students in a wide variety of specializations.
Reading Students are expected to read the relevant material (from the text, various sources on the web, and transparencies that will be made available on the website). You will understand the lectures much better if you read the material before the lectures. The lectures may not cover everything in the readings and will often include material not found in the readings.
Grading policies
  • Homework assignments: 15%
  • Weekly quizzes: 40% (lowest 3 will be dropped, no makeups)
  • Participation and discussion board: 5%
  • In-class exercises: 10%
  • Final exam: 30%
Your overall score will be mapped to a letter grade as follows:
A+ >= 98
A   >= 93
A-  >= 90
B+ >= 87
B   >= 83
B-  >= 80
C+ >= 77
C   >= 73
C-  >= 70
D+ >= 67
D   >= 63
D-  >= 60
F    < 60

Rounding: By default, grades will not be rounded in this course
Homework assignments will be made available on the course website. Some homework assignments will be submitted on paper in class. Others must be submitted via UVa Collab. More details on submission guidelines will be provided on the course website. Be sure that you are on the class mailing list, as refinements and hints for the assignments will be sent through email or posted on the discussion boards.

Homeworks must be submitted before class on the day they are due. Any submission received after this time will be considered to be late. Assignments will be checked immediately after the due date; if you finish an assignment late, you must inform us (instructor and TAs) by email when it is ready to be graded. Changing an assignment after the due date without prior permission will be treated as a late submission.

Students are expected to have a backup disk (or storage somewhere) for every assignment they turn in. In this way, if there is any problem with the copy that is handed in, the backup can be used for grading purposes.
Quizzes Instead of a midterm exam, we will have weekly quizzes. Quizzes will be given during the first 10 to 15 minutes of class on Tuesdays and no makeups or late quizzes will be given. Please do not ask for an exception. For overall grade calculations, the lowest three quiz scores will be dropped. The remaining quizzes with the highest scores will be used to calculate the final "quiz" part of the grade. This essentially means that you can miss up to three quizzes without penalty.
This course will use Piazza for a discussion board. Information for accessing our class will be provided on the first day of class. Participation on the discussion threads counts towards your grade. You can earn credit in several ways.
  • Real-life software failures: Start a discussion about a real-life software failure. Tell us what happened and how it affected users.
  • Describe how one of the ideas from our testing class could have found a real-life software failure before the software was released.
  • Start a thread about an error in the handouts or slides, or about a topic that goes beyond what we discuss in class.
  • Post questions to an appropriate thread and they will be answered by instructo or classmate. Basic questions are encouraged, but only "interesting" questions will earn credit. Correct answers will also earn credit.
Participation must occur during the semester, not after final exams start.

Updates to the course web page will generally be noted on the Piazza page.
In-class exercises We will often have in-class exercises. They will not be scheduled or announced ahead of time, and most will be graded on a "done / not done" basis. Some exercises will be graded via brief demonstrations to the instructor or TAs. Students are encouraged to work in small groups.

Each exercise is designed to get you to think about some specific aspect of the material we are learning that day. The exercises will also be used to monitor attendance.
Five-minute papers At the end of some classes we will have a "five-minute paper." You will be asked to respond in written form to a simple question related to the day's lecture. This will be graded on a "done / not done" basis and count towards in-class exercises.
Final Exam Final exam is closed book/notes/web. The only exception is a single sheet of 8 1/2 x 11 paper (double sided) with hand-written or typed notes by the person taking the exam. The final is in-class and comprehensive unless specified otherwise by instructor. Appropriate documentation (as determined by instructor) and requisite permissions are required for make-up exam requests. Picture IDs are required to take all exams.

If you know in advance that you are unable to make an exam for a valid and unavoidable reason (such as a scheduled surgery, etc), you must notify me at least one week before the scheduled exam date to make arrangements for a make-up, and bring documentation with you when you take the make-up. If you miss an exam due to a university-accepted excused absence (such as an illness or car accident the day and time of the exam), you must notify me within 24 hours of your absence to make arrangements for a makeup, and bring approved documentation with you when you take the make-up exam. Failure to follow either of these policies will result in a zero grade on the exam.
Collaboration You are allowed to work on homeworks individually or collaboratively using one of two models:
  • Collaborate while analyzing the problem and developing an answer or solution, then develop the final solution independently. In this model, each person will turn in a separate document. The submissions must include a section that lists everybody you worked with and what each person contributed. You can work with as many classmates as you like with this model, but only other students in this course this semester.
  • Collaborate from start to finish with at most two other students in this course. You must submit one solution and each person will get the same grade. The submission must outline what each person contributed.
Note: You are NOT ALLOWED to include "guest names." Every person listed as a collaborator must contribute. If someone is listed as a collaborator but did not contribute, all will be reported to the university honor committee.
Late submission Unless arrangements are worked out in advance, 50% penalties apply to late assignment submissions. Assignments are not accepted after two days past the deadline.

I place a high emphasis on timely completion of homework. The reason is that I want to immediately use the fact that you have completed an exercise (typically with a quiz). For this reason, late homeworks are heavily penalized.
Regrading Policy If you are dissatisfied with a grade on a homework assignment, quiz, or exam, you should consult with the instructor directly within a week of the day the graded work was returned to you. No regrade requests will be considered after this week period.
In-class computers
and communication
Phone calls, text messages, instant messages, emails, and general web surfing are highly disruptive to other students and hence not allowed during class time. Technology devices may only be used for the class purposes (e.g. following slides or working on in-class exercises.) Violators will be asked to leave the room. Repeat offenders will lose credit on the "Participation" grade.
Email I occasionally send important announcements to your UVA email account, so it is imperative that you read it regularly.

Students must use the UVA email system for all correspondence with the instructor and TAs (using the following email subject format: CS 4501: insert your subject or CS 6501: insert your subject); if not, we may not notice it.

General assignments or class questions should not be sent through email — in such events, your email will be returned and you will be asked to post your question to the discussion board. In general, the turnaround time for responses to questions will be much quicker on the discussion board. Students should also avoid emailing their code to the instructor unless permission has been obtained (under no circumstances should code be posted publicly to the discussion board).

If you need to contact me via emails, please be aware that I typically answer emails during work hours Monday - Friday, 8:00am - 5:00pm.
Office hours Office hours are times that I commit to being in my office, door open, first come, first served. You do not need an appointment, and no appointments are made. If you cannot make my office hours, then we can try to set up an appointment. I will inform you in class or on the discussion board if I have to miss office hours.
Academic Integrity The School of Engineering and Applied Science relies upon and cherishes its community of trust. We firmly endorse, uphold, and embrace the Universityís Honor principle that students will not lie, cheat, or steal, nor shall they tolerate those who do. We recognize that even one honor infraction can destroy an exemplary reputation that has taken years to build. Acting in a manner consistent with the principles of honor will benefit every member of the community both while enrolled in the Engineering School and in the future.

Students are expected to be familiar with the university honor code, including the section on academic fraud. Each assignment will describe allowed collaborations, and deviations from these will be considered Honor violations. If you are in doubt regarding the requirements, please consult with me before you complete any requirement of this course. Unless otherwise noted, exams and individual assignments will be considered pledged that you have neither given nor received help. (Among other things, this means that you are not allowed to describe problems on an exam to a student who has not taken it yet. You are not allowed to show exam papers to another student or view another studentís exam papers while working on an exam.) Send, receiving or otherwise copying electronic files that are part of course assignments are not allowed collaborations (except for those explicitly allowed in assignment instructions). Assignments or exams where honor infractions or prohibited collaborations occur will receive a zero grade for that entire assignment or exam. Such infractions will also be submitted to the Honor Committee if that is appropriate. Students who have had prohibited collaborations may not be allowed to work with partners on remaining homeworks.
SDAC and Other Special Circumstances If you have been identified as an SDAC student, please let the Center know you are taking this class. If you suspect you should be an SDAC student, please schedule an appointment with them for an evaluation. Students who need academic accommodations should see me and contact the SDAC. All academic accommodations must be arranged through the SDAC. If you have other special circumstances (athletics, other university-related activities, etc.) please contact instructor and/or TA as soon as you know these may affect you in class.

Thanks to Professor Jeff Offutt and Professor Paul Ammann for the input and material provided for this course.