Problem Set 2 - The Good Auld Shell
We'll bash it o'er and o'er.
It touch es our head and greps our dirs To split them tee and tar.
test -d Virginia ; echo $?
where more is Bourne-Again
less su join xargs |& cron aspell
for ls -t | tail -1 > vi.
id -p ray !! fsck; !mount -u VA!
In addition to the on-line submission, you will also schedule a demo (more details coming later).
The goals of this assignment are to:
Learn about process abstractions provided by MULTICS and its successors, and how they are used in Linux and its derivatives.
Understand how to create and manage processes by implementing a simple shell.
Collaboration Policy. For this problem set, you are expected to work with one other student in the class. You should select your own partner (but will be required to work with someone else on the next assignment). If you don't already have a partner, use the teammates forum.
You and your partner should work together in a way that is efficient and collaborative, and ensures that both of you understand everything in the code you submit. Both team members will be expected to answer questions at the demo, and you will be asked to rate your teammate (and your teammate asked to rate you) after the assignment.
You could choose to use pair programing if you want, but it is also fine to discuss together a plan for the assignment, work independently on different parts of the project, then explain and do code reviews on each others work.
Please note that only one of you need to create the private repository for this problem set, the other member should work in the same repository as a collaborator.
In addition to working directly with your partner, you should feel free
to discuss the problems, provide coding help, and ask for help with any
students in the class (or anyone else in the world, for that matter), so
long as you don't to it in a way that is detrimental to your own or
anyone else's learning. You can do this in person, using the discussion
(at the bottom of this page), using the
channels, or any other communication medium you find most effective.
Some of the Rust programming constructs you will use for this assignment are covered in the tutorial, Part 3 and Part 4. You do not need to submit anything from the tutorial for PS2, but we expect you will find it helpful to go through the tutorial (including the exercises).
Before continuing with this assignment, you should find a teammate and one of you should:
- Set up the private repository named 'cs4414-ps2'.
- Add your teammate and 'cs4414uva' as the collaborators.
- Clone the empty private repository to your working environment. Instead of mygithubname below, use your github username.
git clone https://github.com/mygithubname/cs4414-ps2.git
- Get the starting code for ps2.
git remote add course https://github.com/cs4414/ps2.git git pull course master git push --tags origin master
After finishing these steps, everyone in the team should have access to your own
repository that contains starting code for ps2.
A shell is a program that provides an interface for users of an operating system to access the services of a kernel and launch other programs.
In some documents, operating system shells may include both command-line shell (e.g., bash and what you will build for this assignment) and graphical shell (e.g. Windows Shell). However, you are just required to design and implement a simple interactive shell that provides a command-line interface (CLI) to the operating system.
A simple interactive shell can just accept commands in interactive mode. Batch mode is not required even though it has become a common feature of modern OS shells.
The most popular text-based shell today is Bourne-Again shell (bash), which can be found on most GNU/Linux and Mac OS X systems (this is the default shell that runs when you open a Terminal on a Mac).
Start a bash shell on your computer (if you are running Mac OS X or some other Unix variant natively, you should be able to do this directly; if you are using Windows, you should do this on Ubuntu inside VirtualBox; if you are familiar with other tools to examine processes running on Windows, you may also want to use those tools to explore the processes running on your host OS).
Is your operating system doing a good job allocating resources? Describe something interesting you observed or learned in the comments here.
The Good Auld Shell
For this assignment, we have provided starting code for a simple
interactive shell in
gash.rs. It is enough to start processes
running, but not much else. The only internal command is 'exit', and it
can only run external programs in foreground mode. For this assignment,
you will modify gash to include some common and useful shell features,
similar to those provided by bash.
Compile and run the shell. You should see the gash prompt on your screen:
Now try launching some programs using gash. For example, try running
uname -a to
view your system information. You should see something like:
gash> uname -a Linux xuweilin-530U3BI-530U4BI-530U4BH 3.2.0-52-generic #78-Ubuntu SMP Fri Jul 26 16:21:44 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Before working on process management, warm up by adding some basic
internal commands for gash. At this point, the only internal command of
exit, which is really not sufficient for an OS shell.
For the next problem, you are required to implement two internal commands on gash:
cd: change the working directory of gash
history: list the commands you typed on gash
If you are not familiar with these commands, try them in the bash shell to get a sense of what they do. (The bash versions of these commands can take options that allow them to do more complex things, but it is not expected for you to implement any of those options.)
Problem 1. (modify gash.rs)
Modify the gash code so it supports the internal
Running programs in the background
For the next problem, your goal is to enable the program launched by gash to run in background mode. This feature is very necessary in a multi-programming operating system. For example, you can use gash to run a zhttpto server in the background, and continue to use the shell to run other programs. This is very cool! Without background processes, you could only have one program running at a time in a shell, and would need to run many shells to run many programs.
The syntax for running a program in the background is to add an ampersand (&) at the end of the command. For example,
gash> ./zhttpto &
starts a zhttpto server running in the background. Note that you get the gash prompt back after this and can run another command, but the zhttpto server is still running. (Try this in the bash shell to see the desired behavior that you will implement in your shell.)
Problem 2. (modify gash.rs)
Modify the gash code so it supports running programs in the background.
Warning/hint: This is a tough problem and you should think carefully
about how to do it before starting to write code. You should avoid
using libc functions such as
execv(), and aim for a
solution that does not need and
Challenging ("optional") question: Suppose you are running a zhttpto server on a machine that is open to the rest of the Internet. Is there anyway for someone with only an external connection to your host (that is, they can just send requests over the Internet to your zhttpto server) to determine if your server is running as a foreground or background process?
I/O Redirection and Pipes
One useful feature most shells provide is input/output (I/O) redirection which allows you to connect two programs (and iteratively, connect any number of programs). For example, if you ask someone to run your simple zhttpto as a service, she might use this command to save all of the output as a log file while running it in the background:
$ ./zhttpto > ./zhttpto.log 2>&1 &
The command redirects standard output (stdout) to ./zhttpto.log and redirects the standard error (stderr) stream to standard output. Thus, all of the output that you can see on the screen in foreground mode will be redirected to ./zhttpto.log.
You are not expected to spend much time on parsing arguments, so your
gash does not have to handle complex arguments like
2>&1, but I/O
redirection using the basic
> operators should work.
A similar feature is a pipe, which, like the I/O redirects is useful for connecting multiple programs, allows us to do it without needing to go through files.
Unix is designed to allow users to perform many complex operations by
combining lots of simple programs using pipes. For example, we used
aux | wc -l to count the number of processes running. This connects
the output produced by
ps aux to the input to
You can use pipes to connect multiple programs. For example,
./zhttpto | grep User-Agent | uniq | wc -l should count the number of
unique user agent strings seen by your server. (Aside: if you are
wondering why you might care, see
https://panopticlick.eff.org/ to get a
sense of how many browsers share your User Agent string.)
At this point, gash can now run your zhttpto program well in either the
foreground or background. However, when you try to close zhttpto by
Ctrl-Z, you will find that not only zhttpto exits
as expected, gash also exits! In order to solve this problem, you need
to add signal handling in gash code.
Figure out what happens (in the bash shell) when you press
Ctrl-Z. What's the difference between these two signals? For all
signals in Unix, which is the most powerful one to stop process? And how
can you do that?
Ctrl-Cto have the behavior that it kills the current process, but does not exit the gash shell. If this is done correctly, you should be able to start a zhttpto server in the background, then run a command like
tail -f gash.rs(which never completes). When you use
Ctrl-Cto kill the tail process, the zhttpto server running in the background should continue running, and you should get a new gash prompt and be able to run more commands.
Note: See the comments below about this. It is not possible to
do this with Rust 0.9 (without using mostly libc functions), so is okay
Ctrl-C kills all child processes created by your shell, instead of
just killing the foreground process.
Ctrl-Zbehavior and other commands to make it useful, but I'm hoping most teams will come up with more interesting ideas than that!
Submission for Open Students
Open students should submit PS2 using this form. If you run into problems with the autograder, don't get stressed about it, just let us know in the submission. We are still working on making the autograder more robust.
If you have successfully completed PS2, you are eligible to join a team for PS3, and can indicate if you wish to do this on your submission form.
Submission and Demos
(These are the directions for for-credit students. The deadline for submission has passed.)
Steps 1-3 should be done as a team (each team submits one response together). Step 4 is done individually.
Submit your repository to the autograder. You will need to login using Persona (this can be done with any email address, but for-credit students should use their email@example.com address for it). You may submit your repo to the autograder as many times as you want; the only submission that counts for grading is the last submission you make before the deadline.
Once you are satisfied with the results from the autograder, you should add a tag on your code repository with a version number, and check the corresponding URL still passes the autograder. Then, submit the submission form.
Schedule a demo at which you will present your (hopefully working!) gash shell to one of the course staff and answer questions about how you did it. You should be ready at the scheduled demo time with your code loaded in your favorite editor and your shell ready to run. Both team members are expected to be able to answer questions about how you implemented your shell at the demos.
Within 24 hours of finishing your demo, each team member shoud invidually submit the PS2 Assessment Form. Everyone should have submitted this form by Saturday, 15 February, but you should submit it shortly after your demo while things are fresh in your mind.
We will use both web server and shell in Problem Set 3 to build the Zhtta server (which, among other things, will include functionality similar to Apache server-side-includes that can run shell commands).