Title: Problem Set 2 - The Good Auld Shell Date: 2014-01-27 Category: PS Tags: Problem Sets, Shell, Good Ole Song, gash Author: David Evans and Weilin Xu Slug: ps2

Due: 11:59pm on Sunday, 9 February (16 Feburary for open students).

That good auld shell of Wah-who-wah,
We'll bash it o'er and o'er.

It touch es our head and greps our dirs To split them tee and tar.
test -d Virginia ; echo $?
where more is Bourne-Again
less su join xargs |& cron aspell
for ls -t | tail -1 > vi.
Wha-who-wha
!!

traceroute 128.143.22.36
id -p ray
!!
fsck; !mount -u VA!

(All of the highlighted lyrics are syntactically valid bash commands, but some may be damaging if you try executing them without knowing what they mean. If you want to try executing the good auld shell song, we recommend doing it inside a virtualized OS with no access to any valuable data.)

In addition to the on-line submission, you will also schedule a demo (more details coming later). ## Purpose The goals of this assignment are to: - Learn about process abstractions provided by MULTICS and its successors, and how they are used in Linux and its derivatives. - Understand how to create and manage processes by implementing a simple shell. **Collaboration Policy.** For this problem set, you are expected to work with one other student in the class. You should select your own partner (but will be required to work with someone else on the next assignment). If you don't already have a partner, use the [teammates forum](|filename|../../teammates.md). You and your partner should work together in a way that is efficient and collaborative, and ensures that both of you understand everything in the code you submit. Both team members will be expected to answer questions at the demo, and you will be asked to rate your teammate (and your teammate asked to rate you) after the assignment. You could choose to use [pair programing](http://en.wikipedia.org/wiki/Pair_programming) if you want, but it is also fine to discuss together a plan for the assignment, work independently on different parts of the project, then explain and do code reviews on each others work. Please note that only one of you need to create the private repository for this problem set, the other member should work in the same repository as a collaborator. In addition to working directly with your partner, you should feel free to discuss the problems, provide coding help, and ask for help with any students in the class (or anyone else in the world, for that matter), so long as you don't to it in a way that is detrimental to your own or anyone else's learning. You can do this in person, using the discussion (at the bottom of this page), using the `#cs4414` and `#rust` IRC channels, or any other communication medium you find most effective. # Getting Started Some of the Rust programming constructs you will use for this assignment are covered in the tutorial, [Part 3](http://aml3.github.io/RustTutorial/html/03.html) and [Part 4](http://aml3.github.io/RustTutorial/html/04.html). You do not need to submit anything from the tutorial for PS2, but we expect you will find it helpful to go through the tutorial (including the exercises). Before continuing with this assignment, you should find a teammate and **one of you** should: 1. Set up the private repository named 'cs4414-ps2'. 2. Add your teammate and 'cs4414uva' as the collaborators. 3. Clone the empty private repository to your working environment. Instead of _mygithubname_ below, use your github username. ```text git clone https://github.com/mygithubname/cs4414-ps2.git ``` 4. Get the starting code for ps2. ```text git remote add course https://github.com/cs4414/ps2.git git pull course master git push --tags origin master ``` After finishing these steps, everyone in the team should have access to your own `cs4414-ps2` repository that contains starting code for ps2. ## Background A _shell_ is a program that provides an interface for users of an operating system to access the services of a kernel and launch other programs. In some documents, operating system shells may include both command-line shell (e.g., bash and what you will build for this assignment) and graphical shell (e.g. Windows Shell). However, you are just required to design and implement a simple interactive shell that provides a command-line interface (CLI) to the operating system. A simple interactive shell can just accept commands in interactive mode. Batch mode is not required even though it has become a common feature of modern OS shells. The most popular text-based shell today is Bourne-Again shell (bash), which can be found on most GNU/Linux and Mac OS X systems (this is the default shell that runs when you open a Terminal on a Mac). ## Exploring Processes Start a bash shell on your computer (if you are running Mac OS X or some other Unix variant natively, you should be able to do this directly; if you are using Windows, you should do this on Ubuntu inside VirtualBox; if you are familiar with other tools to examine processes running on Windows, you may also want to use those tools to explore the processes running on your host OS).

Exploration 1. Use top and ps or other tools to examine how the resources are being used on your computer. Try running some programs like a web browser, playing a video, and running a large system build (for example, you try building Rust from source code from https://github.com/mozilla/rust.git). While you do, watch which processes are getting to run and which are sleeping. Is your operating system doing a good job allocating resources? Describe something interesting you observed or learned in [the comments here](|filename|./ex1.md).

## The Good Auld Shell For this assignment, we have provided starting code for a simple interactive shell in `gash.rs`. It is enough to start processes running, but not much else. The only internal command is 'exit', and it can only run external programs in foreground mode. For this assignment, you will modify gash to include some common and useful shell features, similar to those provided by bash. Compile and run the shell. You should see the gash prompt on your screen: `gash> ` Now try launching some programs using gash. For example, try running `uname -a` to view your system information. You should see something like: :::text gash> uname -a Linux xuweilin-530U3BI-530U4BI-530U4BH 3.2.0-52-generic #78-Ubuntu SMP Fri Jul 26 16:21:44 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux ## Internal Commands Before working on process management, warm up by adding some basic internal commands for gash. At this point, the only internal command of gash is `exit`, which is really not sufficient for an OS shell. For the next problem, you are required to implement two internal commands on gash: - `cd`: change the working directory of gash - `history`: list the commands you typed on gash If you are not familiar with these commands, try them in the bash shell to get a sense of what they do. (The bash versions of these commands can take options that allow them to do more complex things, but it is not expected for you to implement any of those options.)

Problem 1. (modify gash.rs)
Modify the gash code so it supports the internal `cd` and `history` commands.

## Running programs in the background For the next problem, your goal is to enable the program launched by gash to run in background mode. This feature is very necessary in a multi-programming operating system. For example, you can use gash to run a _zhttpto_ server in the background, and continue to use the shell to run other programs. This is very cool! Without background processes, you could only have one program running at a time in a shell, and would need to run many shells to run many programs. The syntax for running a program in the background is to add an ampersand (&) at the end of the command. For example, :::shell gash> ./zhttpto & starts a zhttpto server running in the background. Note that you get the gash prompt back after this and can run another command, but the zhttpto server is still running. (Try this in the bash shell to see the desired behavior that you will implement in your shell.)

Problem 2. (modify gash.rs)
Modify the gash code so it supports running programs in the background. **Warning/hint:** This is a tough problem and you should think carefully about how to do it before starting to write code. You should avoid using libc functions such as `fork()` and `execv()`, and aim for a solution that does not need and `unsafe` constructs.

**Challenging ("optional") question:** Suppose you are running a zhttpto server on a machine that is open to the rest of the Internet. Is there anyway for someone with only an external connection to your host (that is, they can just send requests over the Internet to your zhttpto server) to determine if your server is running as a foreground or background process?

## I/O Redirection and Pipes One useful feature most shells provide is input/output (I/O) redirection which allows you to connect two programs (and iteratively, connect any number of programs). For example, if you ask someone to run your simple zhttpto as a service, she might use this command to save all of the output as a log file while running it in the background: :::shell $ ./zhttpto > ./zhttpto.log 2>&1 & The command redirects standard output (stdout) to ./zhttpto.log and redirects the standard error (stderr) stream to standard output. Thus, all of the output that you can see on the screen in foreground mode will be redirected to ./zhttpto.log. You are not expected to spend much time on parsing arguments, so your gash does not have to handle complex arguments like `2>&1`, but I/O redirection using the basic `<` and `>` operators should work.

Problem 3. Modify your gash code to support I/O redirection using the `<` and `>` operators.

A similar feature is a pipe, which, like the I/O redirects is useful for connecting multiple programs, allows us to do it without needing to go through files. Unix is designed to allow users to perform many complex operations by combining lots of simple programs using pipes. For example, we used `ps aux | wc -l` to count the number of processes running. This connects the output produced by `ps aux` to the input to `wc -l`. You can use pipes to connect multiple programs. For example, `./zhttpto | grep User-Agent | uniq | wc -l` should count the number of unique user agent strings seen by your server. (Aside: if you are wondering why you might care, see [https://panopticlick.eff.org/](https://panopticlick.eff.org/) to get a sense of how many browsers share your User Agent string.)

Problem 4. Modify the gash code so it supports piping using `|`.

Exploration 2. Now that you have both pipes and redirects working, you should be able to make some creative shell commands. Come up with an interesting way to combine small programs to accomplish something useful or interesting, and [post it here](|filename|./ex2.md).

## Signal Handling At this point, gash can now run your zhttpto program well in either the foreground or background. However, when you try to close zhttpto by pressing `Ctrl-C` or `Ctrl-Z`, you will find that not only zhttpto exits as expected, gash also exits! In order to solve this problem, you need to add signal handling in gash code. Figure out what happens (in the bash shell) when you press `Ctrl-C` and `Ctrl-Z`. What's the difference between these two signals? For all signals in Unix, which is the most powerful one to stop process? And how can you do that?

Problem 5. Modify the gash code to support signal handling for `Ctrl-C` to have the behavior that it kills the current process, but does not exit the gash shell. If this is done correctly, you should be able to start a zhttpto server in the background, then run a command like `tail -f gash.rs` (which never completes). When you use `Ctrl-C` to kill the tail process, the zhttpto server running in the background should continue running, and you should get a new gash prompt and be able to run more commands. Note: See the comments below about this. It is not possible to do this with Rust 0.9 (without using mostly libc functions), so is okay if `Ctrl-C` kills all child processes created by your shell, instead of just killing the foreground process.

## Extra Features

Problem 6. Popular shells like bash provide many other features (e.g., command completion, but not very well), but there are also many features I would like in a shell that bash doesn't provide. For this question, its up to you to decide what you would like to add to your shell and to add it. It could be something that other shells already provide, or something more creative and interesting. The most obvious thing would be to add support for `Ctrl-Z` behavior and other commands to make it useful, but I'm hoping most teams will come up with more interesting ideas than that!

### Submission for Open Students **Open students should submit PS2 using [this form](http://goo.gl/BpDZr6).** If you run into problems with the autograder, don't get stressed about it, just let us know in the submission. We are still working on making the autograder more robust. If you have successfully completed PS2, you are eligible to join a team for PS3, and can indicate if you wish to do this on your submission form. ### Submission and Demos (These are the directions for for-credit students. The deadline for submission has passed.) Steps 1-3 should be done as a team (each team submits one response together). Step 4 is done individually. 1. **[Submit your repository to the autograder](http://128.143.136.170:9080/).** You will need to login using Persona (this can be done with any email address, but for-credit students should use their abc2d@virginia.edu address for it). You may submit your repo to the autograder as many times as you want; the **only submission that counts for grading is the last submission you make before the deadline**. 2. Once you are satisfied with the results from the autograder, you should add a tag on your code repository with a version number, and check the corresponding URL still passes the autograder. Then, **[submit the submission form](https://docs.google.com/forms/d/12H7JXGtZV-ia1AXdX0O5Vf9dGztr9M9Ar8HOVlhvObI/viewform)**. 3. [**Schedule a demo**](https://docs.google.com/spreadsheet/ccc?key=0AvXGI69lkg_edGE4dk0taUFlcmVueGg0UExIT1RZSXc&usp=sharing#gid=0) at which you will present your (hopefully working!) gash shell to one of the course staff and answer questions about how you did it. You should be ready at the scheduled demo time with your code loaded in your favorite editor and your shell ready to run. Both team members are expected to be able to answer questions about how you implemented your shell at the demos. 4. Within 24 hours of finishing your demo, each team member shoud invidually submit the [**PS2 Assessment Form**](https://docs.google.com/forms/d/1Q3CVeaVj1T9HDAgakWSRVw5rKY6hJeQNmai2a91r9Gg/viewform). Everyone should have submitted this form by Saturday, 15 February, but you should submit it shortly after your demo while things are fresh in your mind. ### Teaser We will use both web server and shell in Problem Set 3 to build the Zhtta server (which, among other things, will include functionality similar to Apache server-side-includes that can run shell commands).

comments powered by Disqus

This assignment was originally created by Weilin Xu, Purnam Jantrania, and David Evans for University of Virginia cs4414 Fall 2013, and revised by Weilin Xu for Spring 2014.