Plan for Monitoring CS Degree Program Effectiveness
Computer Science Program, University of Virginia

Maintainer of this document: Tom Horton (horton@virginia.edu)
Date: May 21, 2003 (version 4)

This document describes how the Computer Science Department will monitor the effectiveness of its Bachelor of Computer Science degree program. This plan represents a continuous improvement process that will allow the department to collect information on program effectiveness, make changes to address issues that have been identified, and evaluate how well they improved the program.

Roles within the Department:

The department has several people and committees that play a role in the process of monitoring the quality of our undergraduate CS degree.

Stakeholders:

Several groups have an interest in the quality of the undergraduate computer science program, including its goals and objectives, how the department seeks to accomplish these goals, and how both these activities and goals can be improved over time.

There are others who may also be stakeholders to a lesser degree: parents, other departments in SEAS or the University, etc. When appropriate we will seek input from these groups, but we wil focus our efforts to collect feedback from the groups listed above.

Inputs Gathered Regarding Program Effectiveness:

  1. The department will use any standard mechanism used by the School of Engineering and Applied Science (SEAS) to capture feedback from classroom evaluation of students that determines how well that course meets its objectives.
  2. SEAS requires each faculty member who teaches an undergraduate course to submit (as part of their annual report) an end-of-course memo, which is a short summary of the assessment carried out, an interpretation of the results, proposed changes, evaluation of previous changes, etc. These reports (for all courses integral to the CS program, whether taught within the department or not) will be collected together by the Assessment Coordinator, through the department Chair and the Dean's office. They will be reviewed by the CS Undergraduate Committee at least once a year.
  3. Each fall, the Accreditation Coordinator arranges a meeting with current students to get their input on the degree program's objectives, on the curriculum, facilities, and any other issues the students wish to raise. This meeting will normally be scheduled through the student chapter of the ACM, with the department chair present to answer questions and hear comments.
  4. Normally once a year, the Department arranges a day when members of the Corporate Sponsors Program come to the university to meet with the department. (In 2001, meetings were held in February and in November.) The Accreditation Coordinator is responsible for arranging for information to be distributed and/or feedback collected at this meeting, if that is deemed appropriate by the Accreditation Committee.
  5. SEAS administers the "EBI Survey" to its students each year, with some questions addressing questions related to ABET standard outcomes A-K and other issues relative to program effectiveness. Results of this survey (including comparison to national norms and a set of peer institutions) are returned in the fall to each department. The Accreditation Coordinator is responsible for summarizing these results and identifying how they apply to the CS program, and will report these findings to the Accreditation Committee.
  6. In Fall 2002, SEAS also administered a survey sent to students who have graduated, with an accompanying survey to be given to the alumna's supervisor. The standard SEAS survey will address common issues (including ABET outcomes A-K). In addition, a degree-program specific form (or set of questions) will also be sent out. When these results are returned, the Accreditation Coordinator is responsible for summarizing these results and identifying how they apply to the CS program, and will report these findings to the Accreditation Committee.

Process:

  1. Each semester: Each CS faculty member teaching an undergraduate course will follow the SEAS standard process of defining course objectives for each course taught and document this in the beginning-of-course memo. The instructor will attempt to assess as effectively as possible how well the students performed in achieving the course objectives.
  2. At the end of each semester Each CS faculty member will submit an end-of-course memo for each course they taught. Per SEAS policy, this memo will include a short summary of the course's objectives, the assessment carried out, an interpretation of the results, proposed changes, evaluation of previous changes, etc.
  3. Start of fall term: The Accreditation Coordinator obtains SEAS survey data, faculty course reports, and other inputs collected during the previous academic year. S/he may carry out an initial analyis of this information, if required, before presenting it in the Fall Accreditation Committee meeting.
  4. Fall term: The Accreditation Committee holds an accreditation meeting with the following goals. One outcome of this meeting is the preparation of the required annual fall report to SEAS.
    1. To evaluate any changes made in the degree program in the last year, noting the benefits/problems that resulted from these changes.
    2. To review inputs from stakeholders, survey information, course reports, collected course materials, etc.
    3. From this data, to identify major issues to be addressed during this academic year.
    4. To begin the process of making specific changes in the curriculum or how it's delivered out in order to meet specific needs. (Such changes may require time to develop, to get approval from the department and SEAS, etc.)
    5. Decide on what to report to CS department faculty, and on how best to get input or participation as needed during the current academic year.
    6. Review the timeline and work-plan for the next upcoming ABET visit.
  5. Fall Term: After the meeting described above, the Department Chair, the Associate Department Chair, or the Accreditation Coordinator prepares the annual self-study report to the Dean's office.
  6. Later in fall term: Department chair and faculty meet with current students to get their inputs on the program.
  7. Spring and/of Fall term: The department may hold a meeting with its Corporate Sponsors Program, and issues related to the quality of the CS undergraduate degree may be presented, with feedback sought.
  8. Spring term: The Accreditation Committee holds another accreditation meeting with the following goals:
    1. To review and possibly modify the objectives and outcomes of the CS undergraduate degree program.
    2. To finalize any changes in the curriculum or how it is carried out.
    3. Evaluate any faculty feedback or special activities relevant to program improvement.
    4. Discuss what results, issues, etc. should be included in annual report to be submitted to Dean's office.
    5. Review the timeline and work-plan for the next upcoming ABET visit.

Documents Produced:

All documents will be made available on the department's ABET Web site. If deemed appropriate, some documents may be password-protected.