Taking CS4102 Algorithms Online

We have taught Algorithms pretty consistently over the past few years, adjusting the model to scale. The course relies heavily on and encourages student collaboration on homework assignments to better understand concepts and derive algorithms for new problems. The current version of the course focuses more heavily on describing algorithms and proving their runtime and correctness rather than their implementation or algorithms by example. To that end, the course has 10 homeworks, 7 type-written using LaTeX and 3 coding assignments. The written assignments require describing multiple algorithms (in English or pseudocode) and proving their runtime and correctness. The coding assignments require students to define, implement, and test their algorithms in either Java or Python, but we do not provide an autograding system or integrated testing environment. Our goal has been to entice students to design before implementation and to avoid a trial-and-error approach.

In-person course management

Since the course has approached and now exceeded 300 students, we have put systems in place to handle the larger course management. Before taking the course online, we utilized the following tools:

  • Student-facing tools
    • Collab, UVA’s course management system, to handle annoucement emails, posting and submission of assignments, and gradebook maintenance
    • Piazza as a discussion board system for students and course staff to interact, usually for quick homework and course questions
    • Panopto to house lecture videos, which we recorded live every class session
  • Instructor-only tools
    • Github repository to handle shared editing and posting of the course website, including lecture notes, schedule, and contact information – integrated into Collab
    • Dropbox for sharing lecture content, a private Github repo for all other course content
    • An Ubuntu virtual machine testing environment to run and grade student coding assignments using command-line PHP
    • A plagiarism detection tool capable of testing .tex, .java, and .py submissions
  • TA and grading management tools
    • TPEGS, a UVA paper-exam grading software, to grade the in-class midterm
    • A PDF grading software I developed to handle dividing the grading between TAs more easily than using Collab’s native interface
    • Slack for communication among course staff

At the beginning of the semester, we walk through a “Homework Help” algorithm that encourages students to not only seek help on Piazza, but provide any help they can while waiting. We also attempt to have extensive office hours, where TAs are available most times throughout the day for questions in a group discussion and white-board session format.

Moving online only

With so many tools in place already to handle assignments, course delivery, and management, we did not anticipate a large shift in the course to bring it online. We had two distinct concerns in the transition:

  1. dividing up and presenting the lecture material in an organized format, and
  2. continuing to foster the collaborative nature of the course in homework assignments.

To mitigate those concerns, we reformatted the course homepage to present information about the transition. Before Spring Break, our schedule was broken down by in-person lecture schedule; afterwards we reformatted the page to reflect upcoming topic areas. For each topic area, we grouped lecture videos and notes, a live Q&A session, and a watch-by date. We also provided a list of tools, such as online video chat, whiteboard, and note sharing sites to help students continue their collaboration. To foster collaboration and continue to engage with students virtually, we added the following new tools to our already robust set:

  • Zoom video conferencing software for in-person Q&A during former in-person lecture timeslots and for instructor office hours
  • Discord for TA office hours to provide a system for students to interact and TAs to share screens and address the concerns of multiple students at once

Results

Student engagement on Piazza followed roughly the same trajectory as before the break. We did notice a more sustained interaction with Piazza after the break, since at least one question was asked each day, as seen in Figure 1.

Figure 1: Questions asked per day
Figure 1: Questions asked per day

Overall user engagement followed roughly the same trajectory as before the break, as seen in Figure 2, however there are slightly more sustained users on the site.

Figure 2: Active users per day
Figure 2: Active users per day

Interestingly, the overall number of questions continued to increase linearly (Figure 3).

Figure 3: Questions asked and answered over time
Figure 3: Questions asked and answered over time

Collaboration Policy

We encourage collaboration in Algorithms. It has changed over the semesters, and we’re currently studying how students use this policy. Starting off the spring, we encouraged collaboration in groups of up to 5 students. Our goal was for students to share problem solving techniques and to “white board” their ideas with classmates. Since we have had concerns about “oversharing” of information, we requested that students not share any written notes or code, including online document platforms. The full collaboration policy is shown below:

You are encouraged to collaborate with up to 4 other students, but all work submitted must be your own independently written solution. List the computing ids of all of your collaborators in the collabs command at the top of the tex file. Do not share written notes, documents (including Google docs, Overleaf docs, discussion notes, PDFs), or code. Do not seek published or online solutions for any assignments. If you use any published or online resources (which may not include solutions) when completing this assignment, be sure to cite them. Do not submit a solution that you are unable to explain orally to a member of the course staff. Any solutions that share similar text/code will be considered in breach of this policy. Please refer to the syllabus for a complete description of the collaboration policy.

On the transition to virtual delivery, we wanted to continue to encourage collaboration. In order to facilitate collaboration, we needed to loosen our policy and provide mechanisms for students to continue meeting with their study groups. We revised the policy as follows:

You are encouraged to collaborate with up to 4 other students, but all work submitted must be your own independently written solution. List the computing ids of all of your collaborators in the comments at the top of the tex file. You are permitted to collaborate through online tools such as Google Docs, interactive whiteboards, Google Meet, Google Hangouts, Zoom, Skype, etc, however you must limit written/typed details to high-level algorithm design. Each person is responsible for taking those ideas and turning them into pesudocode and a writeup. Do NOT copy and paste from shared documents, which includes re-typing verbatim or trying to disguise text that you are essentially copying. Over-collaboration of that form is fairly easy to detect with plagiarism tools. Do not seek published or online solutions, including pseudocode, for this assignment. If you use any published or online resources (which may not include solutions) when completing this assignment, be sure to cite them. Do not submit a solution that you are unable to explain orally to a member of the course staff. Any solutions that share similar text/code will be considered in breach of this policy. Please refer to the syllabus for a complete description of the collaboration policy.

Student Reactions

Positive Reactions

Negative Reactions


Next

Helpful Tools

I have found the following tools very helpful. Some I have created, but all are free for UVA instructors. Most are free to everyone.