CS305 Group Project: Interface Design and Evaluation

(Version 1.0, Sept. 26)

Note: You can see some sample projects at this site by students at Ga Tech. Our project is modeled on that course's, though there are some differences. (E.g. in the past they have emphasized HTA while we put more emphasis on use cases and scenarios.)  We may also put some links up to past UVa projects.  (Watch this space.)

Outline

Quick access to the sections of this document:

• Project Report Book
• Part 0 Define teams. Due 3 pm, Monday, Oct. 6.
• Part 1 Investigate problem domain.   Due 3 pm, Mon., Oct. 20. 
• Part 2 Design alternatives.  Probably Due 3 pm, Mon., Nov. 3.  
• Part 3 Prototype iteration and evaluation plan.  Probably Due Mon., Nov. 24

Project Overview

This semester you will undertake a group project (teams of 4, self-selected) to evaluate some computing-related task/problem, to develop interface design alternatives for the task/problem, to implement a prototype of your design, and to plan and execute the evaluation of your design. This project should provide you with hands-on experience with the tasks that interface designers face every day.

Ideally, the topic of the project will be a problem that matters to some "real-life" people. You can pretend this is true <grin>, or if you wish you can find real stakeholders with whom you must communicate with and learn from. This will certainly provide a more realistic experience if you do real user-centered design for this project; however, this can be time-consuming and difficult in the time we have for this project. So you are allowed to build something for yourselves or for a presumed set of stakeholders, but at various points you will have to have "outsiders" evaluate it.  Part of your grade will be based on how well you simulate or really carry out user-centered design.

Each of the three parts of the project will be assigned a grade, but team members will be given the opportunity to give input about the level and quality of work done by all members of the team. These results will be combined and evaluated by the instructor, and an individual's score for the complete project will include a "contribution" element, which may be worth as much as one-third of the total grade (roughly a letter-grade in the course).

If you treat your team members professionally and responsibly (exactly as you would hope to be treated), everyone will earn full credit for participation. No "penalties" will be given without providing justification about what went wrong, and a student will have the right to discuss this matter with the instructor. We all recognize that each person may not contribute equally on each assignment, so a degree of reasonable expectations will be taken into consideration by teams and by the instructor. We also recognize that problems or issues sometimes arise that require someone to be out of town unexpectedly, etc. These will be taken into account as long as the student informs the team and the instructor as early as possible.

Project Report Book

Each part of the project will include a deliverable report. This report will be placed on the Web or the Wiki and should be created with HTML, Word, or PDF.. Each team should have a "home" page which includes: 1) a brief (paragraph) description of the problem/task; 2) the team members; 3) Links to the reports for project parts 1-3 (no report is needed for part 0). The format of the reports for the individual parts is up to you, but it should be professionally prepared, expressive, grammatically sound, illustrative of your efforts and process, and easy to view and understand. A good design effort can easily be hampered by a poor communication of what was done. If you need Web space to store this material, let the instructor know and we will help you get space for your pages.

Part 0 - Identifying Team and Topic

This first part of the project is relatively simple. You must list the members of your team and identify the problem that you will be working on. (If you have more than one possible project that you can't determine, define both and alert the instructor and TA that you need help deciding. But you must do this 24 hours before the deadline above!  Send email to horton@cs.virginia.edu and ross.gore@gmail.com with "CS305 Group Project" in the Subject line.)

We would like you to put all your info on the Wiki, in the area for your team. If you don't, you'll have to set up your web project space that lists your team name, members, and provides links to all of the project deliverables. 

Part 1 - Understanding the Problem

The key goal of this first substantive part of the project is to deeply understand that problem that you are addressing, its set of pertinent users and stakeholders, and the issues and constraints that are involved in the problem. If an existing system or systems exist for this problem, you should include an assessment of one existing system currently or commonly used to accomplish these tasks. Our emphasis is to identify important characteristics of the problem that will influence your subsequent design of the interface and interaction characteristics of the system.

Your grade will in part be based on how well you use appropriate techniques covered in class for acquiring this kind of information. Feel free to utilize the techniques that you feel are most appropriate to the particular task you are examining. Keep in mind that this analysis has the purpose of defining a set of constraints for your subsequent design, including what criteria should be used to judge if a particular design is a good solution or not.

Your report and deliverable for this part should deeply examine the problem of study. Who are the potential users? What tasks do they seek to perform? What functionality should the system provide? Basically, you are setting up a set of constraints for your subsequent design. What criteria should be used to judge if your design is a success or not?

Specifically you should develop the following items in this part, and you should communicate them through your report:

• an initial problem statement - an overview of what they system will do and why it is needed;
• a description of the important characteristics of the users of the system;
• a task analysis, consisting of:
  • a description of the important characteristics of the tasks performed by users;
  • a description of important characteristics of the task environment;
  • a simple structured task analysis of the problem. You may use any of forms described in class or our text. But you should provide a high-level overview or organization of the set of tasks, and then some brief documentation of all the subtasks (e.g. a use-case name and short description). For a small number of important or complex subtasks, give a more detailed description (e.g. a fully documented use-case).
• (if appropriate) a description of the larger system , the technical or social organization, in which your product will participate;
• (if appropriate) an analysis of the existing system, automated or manual, including its advantages and deficiencies;
• an initial list of usability criteria, or principles, that should be used in the eventual evaluation of your design; and
• a description and justification of how the above information was gathered, including references to existing literature and on-line material that was instrumental in helping you complete this stage of the project.
• Important: A discussion of the implications of what you learned above.

The last item in the list above is critical. Don't just describe the target users, tasks, environment, etc. You must also tell us how these attributes should/will influence your design. Are there any implications to be made from the user profiles and other data you learned? We will be very careful to look for this information in your report.

There may be a required presentation of your results by some of the groups at this stage.

Part 2 - Design Alternatives

The key goal of part 2 of the project is to use the knowledge gained in part 1, as well as that from class, to develop a set of design alternatives for your problem.. The purpose of these design alternatives is for you to explore and illustrate the potential design space. Based on your experiences creating these designs, you should iterate on the requirements and usability criteria for your product.

In this part of the project you will develop mock-ups, scenarios, storyboards, and sketches of your interface designs. That is, you should provide pencil-and-paper or electronic images of the interface at various stages. You will not build a working prototype at this stage. However, your design sketches should be sufficiently detailed for a potential user to provide useful feedback about the design. Along with your design mock-ups, you should provide a brief narrative walk-through of how the system will work. It's also very important that you include your justifications for why design decisions were made, and what you consider to be the relative strengths and weaknesses of your different designs.

The key in this part of the project is to come up with a number of different design ideas (three is the minimum) that are not just a small set of variations derived from some basic design. Note: You will probably have the best results if you do not simply have each team member to plan and create a prototype completely on his or her own. A good creative design process involves multiple ideas and discussion. The group should start with a brainstorming session with all members present to talk about what to prototype, why, and what the three different ideas might look like. After this discussion, the group can break into subgroups (possibly one person each) to flesh out each of the three possibilities. With your three approaches for this design, you should seek to create and explore some fundamentally different design ideas from the many possibilities in the design space for the problem you have chosen.

It is important here to understand and distinguish between interaction/paradigm designs and software UI designs. You might wonder if you should try to find differents paradigm designs or just have several different UI designs from the same paradigm. For some topics, doing some paradigm design is appropriate and does make sense. You need not submit paradigms that just will be very unlikely to be selected in Part 3, however. In Part 3 you focus on one particular design that you think will be good. Our goal here in this part is to find out something useful that will help us choose a good design to work with in Part 3.

Your project report should include all the explanatory material mentioned above as well as all the design sketches, drafts, storyboards, etc., that you generated. If some of your sketches are on paper, that's fine. (If you want to scan them for an electronic copy, contact me if you need access to a scanner.) Make sure that your report adequately describes the design process that your group undertook.

More specifically, you should develop the following items in this part, and you should communicate them through your report:

• At least three interface designs (prototypes) illustrating some portion of your product. With each design you should include:
  1. A rationale for this design choice. What do you think might be good about it? Why is it worth evaluating?
  2. Illustrations of the design (sketches, storyboards ...). You do not have to create the kind of paper-prototype that can be "executed" in an evaluation. But drawings of screens, windows, menus, pop-ups, etc. are a good way to describe
  3. Use at least one scenario from an end-user's perspective and describe (words) or show (drawing etc.) how it would be carried out for this design.
  4. An assessment of this design. This assessment should include feedback from a small number potential users. These could be people outside your team, or other team members who have not worked much on this particular design prototype. (This assessment does not need to be as formal as your earlier assignment.) It should talk about strengths and weaknesses, using terms and principles studied in our usability course.
• An explanation for why you chose this set of designs to explore the potential designs. This section will talk about what you were trying to learn at a high-level about choosing an interface or conceptual model for your proposed product. (This may repeat material covered in part 1 for each of the three designs.)
• A summary of modifications to your project's functional requirements and usability criteria that came out of this process. It is likely that you will learn about some needed changes during this process.

Note: It is possible that we will utilize one full class day for all groups (or some groups selected randomly) to give a presentation (perhaps an overview) of their results.You may receive input from the class or the instructor that will help you in determining how to narrow the design space for Part 3.

Part 3 - System Prototype and Evaluation Plan

In part 3 of the project, your group will implement a detailed prototype of your product. You can use any software that you would like to assist this process (e.g., VB, Flash, Java, FLUID/FLTK, etc.). You should be able to get a complete "chunk" of the interface functionality working, but clearly you will not be able to implement all back-end application functionality. You may also limit yourself to some subset of the total functionality your requirements described, but this should be based on the usability questions that are most important for this system.
Note: Talk early about what you want to prototype, and talk to me if you have questions or concerns!

If your project involves a new hardware device, say new handheld device, then produce a second prototype that emphasizes the physical form factor of all artifacts to be used in your system. This could be a drawing, a physical mock-up, etc.

Additionally, you must provide a set of initial usability specifications for your system and a plan for an evaluation of it. To develop usability specifications, consider the objectives of your design. For example, if you are working on a calendar manager, you might specify time limits in which you expect a user to be able to schedule or modify an appointment, or a maximum number of errors that you expect to occur. Basically, you should list a set of criteria by which your interface can be evaluated. I suggest you choose a few focused areas of usability that are appropriate for your application and concentrate on those. (Do not try to be exhaustive. Prioritize.)

Further, this part of the project should include an initial evaluation plan for the system. (A plan only -- you don't have to execute it!) What kinds of benchmark tasks would you have users perform to help evaluate the interface? What kind of subjective questionnaire would you deploy to have a user critique the interface? If you had access to a usability lab and software like Morae, would it be appropriate to use these facilities to find out something (what?) about your system? The key here is not to do some exhaustive description of the complete, ultimate usability evaluation plan, but to motivate why the particular plan you propose is appropriate for this interface for your system.

Note that developing an initial evaluation plan is also a good way to figure out how much of the interface you need to develop. You should be able to build and connect to enough of the application functionality to be able to conduct an initial usability evaluation with the benchmark tasks as you are proposing here.

Again, note that you are only responsible for creating the plan, not carrying out the evaluation according to the plan!

More specifically, you should develop the following items in this part, and you should communicate them through your report:

• An overall description of your final design, including a justification about how you arrived at this. This justification should include an explanation of what you learned from the three prototypes you did in the previous part of the project. The key component to include in your project report is a justification of why you settled on the design that you chose. What's special about this particular design with respect your problem?
• One or two prototypes illustrating various portions of your final design:
  • You should provide a functional software prototype of some limited portion of your system. Ideally, this will run on an intended platform described above, but if that is not possible, you should attempt to provide a software prototype that mimics the intended platform as best as you can. The important point here is that you address only a small portio of overall function with this prototype, so choose some critical piece of functionality and implement it.
  • (Not applicable to all projects.) A complete description of the physical form factor of all relevant artifacts to be used. You should provide sufficient visual material to convey the essence of your intended physical form factor. If you are using available hardware, be sure to provide pictures. If you anticipate the need for novel hardware, provide detailed sketches. You should even consider building a physical model that you can show to potential users.
• Appropriate usability and specfications for your system, as described above. These should tie in with what you have prototyped, and should be the basis for the evaluation plan described next.
• A detailed evaluation plan. You should indicate multiple evaluation exercises that will be performed on one or all of the prototype representations. Each evaluation exercise must highlight the usability criteria to be targeted, and provide enough description of the required input for the evaluation exercise so that an experienced HCI evaluator could perform the evaluation on the appropriate prototype representation.