Project 3: The Shell

This project was written by instructors of CS354 at Purdue University, and is taken from their web site.

Introduction

The goal of this project is to build a shell interpreter like csh. The project has been divided in several parts. Some sources are being provided so you don't need to start from scratch.

Using the Debugger

It is important that you learn how to use a debugger to debug your C and C++ programs.  If you spend a few hours learning how to use gdb, it will save you a lot of hours of development in this lab.

To start gdb type "gdb program". For example, to debug your shell type:

csh> gdb shell
Then type
(gdb) break main
This will make the debugger stop your program before main is called. In general, to set a breakpoint in a given function type "break <function-name>"

To start running your program type:

(gdb)run
Your program will start running and then will stop at main.

Use "step"or "next" to execute the following line in your program. "step" will execute the following line and if it is a function, it will step into it. "next" will execute the following line and if it a function it will execute the function.

(gdb) next    - Executes following line. If it is a function it will execute the function and return.
or
(gdb) step    - Executes following line. If it is a function it will step into it.
An empty line in gdb will rerun the previous gdb command.

Other useful commands are:

print var   - Prints a variable
where      - Prints the stack trace
quit       - Exits gdb
For more complete tutorials on gdb see:

GDB Tutorial 1
GDB Tutorial 2
GDB Tutorial 3
GDB Tutorial 4

First part: Lex and Yacc

In this part you will build the scanner and parser for your shell.
The deadline of this part of the project is Wednesday, April 3rd, at 11:59 P.M. Follow these instructions to turnin your part one.

1. Login to CSSUN.

2. cd to lab3-src and type "make clean"

3. Type "make" to make sure that your shell is build correctly.

4. Type "make clean" again.

5. cd one directory above lab3-src

6. Create a tar file named <user_name>.tar, where <user_name> is your CSSUN login, by typing

    tar -cf <user_name>.tar lab3-src

7. Gzip the tar file by typing

    gzip <user_name>.tar

8. Since this timestamp will be used to verify whether the work was completed on time or not, you should set the permissions on the file you submitted to make sure that the file timestamp is not changed. So this by typing:

    chmod a-w <user_name>.tar.gz

9. Mail the gzipped tar file to clay at cs dot georgetown dot edu as an attachment.
 

Here are the man pages for Lex and Yacc .

Second part: Process Creation, Execution, File Redirection, Pipes, and Background

Starting from the command table produced in Part 1, in this part you will execute the simple commands, do the file redirection, piping and if necessary wait for the commands to end.
  1. For every simple command create a new process using fork() and call execvp() to execute the corresponding executable. If the _bakground flag in the Command struct is not set then your shell has to wait for the last simple command to finish using waitpid(). Check the manual pages of fork(), execvp(), and waitpid(). Also there is an example file that executes processes and does redirection in cat_grep.cc. After this part is done you have to be able to execute commands like:
    2.              ls -al

             ls -al /etc &
  2. Now do the file redirection. If any of the input/output/error is different than 0 in the Command struct, then create the files, and use dup2() to redirect file descriptors 0, 1, or 2 to the new files. See the example cat_grep.cc to see how to do redirection. After this part you have to be able to execute commands like:
    3.              ls -al > out

             cat out

             ls /tttt >& err

             cat err

             cat < out

             cat < out > out2

             cat out2
                 ls /tt >>& out2
    ">& file" redirects both stdout and stderr to file. ">>& file" append both stdout and stderr to file.
    ">> file" appends stdout to file.
  3. Now do the pipes. Use the call pipe() to create pipes that will interconnect the output of one simple command to the input of the next simple command. use dup2() to do the redirection. See the example cat_grep.cc to see how to construct pipes and do redirection. After this part you have to be able to execute commands like:
    4.              ls -al | grep command

             ls -al | grep command | grep command.o

             ls -al | grep command

             ls -al | grep command | grep command.o > out

             cat out
    The deadline of this part of the project is Thursday April 18, 2002 at 11:59pm.

    Follow these instructions to turnin your part two.

    1. Login to CSSUN.

    2. cd to lab3-src and type "make clean"

    3. Type "make" to make sure that your shell is build correctly.

    4. Type "make clean" again.

    5. cd one directory above lab3-src

    6. Create a tar file named <user_name>.tar, where <user_name> is your CSSUN login, by typing

        tar -cf <user_name>.tar lab3-src

    7. Gzip the tar file by typing

        gzip <user_name>.tar

    8. Since this timestamp will be used to verify whether the work was completed on time or not, you should set the permissions on the file you submitted to make sure that the file timestamp is not changed. So this by typing:

        chmod a-w <user_name>.tar.gz

    9. Mail the gzipped tar file to clay at cs dot georgetown dot edu as an attachment.
     
     

Testing your Shell:

Your shell will be graded using automatic testing, so make sure that the tests given to you run.

Your grade for this project will depend on the number of tests that pass. The tests given are for part 2 and 3 of the project. Tests used for grading: test-shell-grading/  and test-shell-grading.tar.Z

See the  file lab3-src/README for an explanation on how to run the tests. The tests will also give you an estimated grade. This grade is just an approximation. Other tests not given to you will be used as well during grading.

Third part: Control-C, Wild Cards, Elimination of Zombie processes, etc.

This is the final part of your shell. You will include some more features to make your shell more useful.
  1. Your shell has to ignore ctrl-c. When ctrl-c is typed, a signal SIGINT is generated that kills the program. There is an example program in ctrl-c.cc that tells you how to ignore SIGINT. Check the man pages for sigset.
  2. You will also have to implement also an internal command called exit that will exit the shell when you type it. Remember that the exit command has to be executed by the shell itself without forking another process.
    3.              myshell> exit

               Good bye!!

             csh>
  3. You will implement an internal command printenv to print the environment variables of the shell. The environment variables of a process are stored in the variable environ. Environ is a null terminated array of strings.
    4.             char **environ;
    Check the man pages of environ.
     
  4. Now do the wildcarding. The wildcarding will work in the same way that it works in shells like csh. The "*" character matches 0 or more nonspace characters. The "." character matches one nonspace character. The shell will expand the wildcards to the file names that match the wildcard where each matched file name will be an argument.
    5.              echo *          // Prints all the files in the current directory

             echo *.cc       // Prints all the files in the current

                             // director that end with cc

             echo c*.cc

             echo M*f*

             echo /tmp/*     // Prints all the files in the tmp directory

             echo /*t*/* 

             echo /dev/*
    You can try this wildcards in csh to see the results. The way you will implement wild carding is the following. First do the wild carding only in the current directory. Before you insert a new argument in the current simple command, check if the argument has wild card (* or ?). If it does, then insert the file names that match the wildcard including their absolute paths. Use opendir and readdir to get all the entries of the current directory (check the man pages). Use the functions compile and advance to find the entries that match the wildcard. Check the example file provided in regular.cc to see how to do this. Notice that the wildcards and the regular expressions used in the library are different and you will have to convert from the wildcard to the regular expression. The "*" wildcard matches 0 or more non-blank characters, except "." if it is the first character in the file name. The "?" wildcar matches one non-blank character, except "." if it is the first character in the file name. Once that wildcarding works for the current directory, make it work for absolute paths.
     
  5. You will notice that in your shell the processes that are created in the background become zombie processes. That is, they no longer run but wait for the parent to acknowledge them that they have finished. Try doing in your shell:
    6.              ls &

             ls &

             ls &

             ls &

             /bin/ps -u <your-login> | grep defunct
    The zombie processes appear as "defunct" in the output of the "ps -u <your-login>" command.

    To cleanup these processes you will have to setup a signal handler, like the one you used for ctrl-c, to catch the SIGCHLD signals that are sent to the parent when a child process exits. The signal handler will then call wait3() to cleanup the zombie child. Check the man pages for wait3 and sigset. The shell should print the process ID of the child when a process in the background exits in the form "[PID] exited."
     

  6. Implement the builtin command setenv A B this command sets the environment variable A to be B. Check man pages for function setenv().

    7.  
  7. Implement the builtin command unsetenv A. This command removes environment variable A from the environment.

    8.  
  8. Implement the cd [ dir ] command. This command changes the current directory to dir. When dir is not specified, the current directory is changed to the home directory. Check "man 2 chdir".
  9. Extend lex to support any character in the arguments that is not a special character such as "&", ">", "<", "|" etc. Also, your shell should allow no spaces between "|", ">" etc. For example, "ls|grep a" without spaces after "ls" and before "grep" should work.

    10.  
  10. Allow quotes in your shell. It should be possible to pass arguments with spaces if they are sorrounded between quotes. E.g.
    11.                 ls "command.cc Makefile"

                command.cc Makefile not found

"command.cc Makefile" is only one argument.
    Remove the quotes before inserting argument. No wild-card expansion is expected inside quotes.
     
  11. Allow the escape character. Any character can be part of an argument if it comes immediately after \. E.g.

    12.   echo \"Hello between quotes\"
    "Hello between quotes"
    echo this is an ampersand \&
    this is an ampersand &
  12. You will implement environment variable expansion. When a string of the form ${var} appears in an argument, it will be expanded to the value that corresponds to the variable var in the environment table. E.g.

    13.  

     
     
     
     
     
     
     
     
     
     
     

    setenv A hello
    setenv B world
    echo ${A} {$B}
    Hello World

    setenv C ap
    setenv D le
    echo I like ${C}p${D}
    I like apple
     

  13. You will implement subshells of the form `subshell`  (Notice that the character "`" used here is a back-tick. This character is usually in the same key as "~". ). A subshell may appear anywhere in a command and when executed the output of the subshell will become input of the shell itself. For example, the following command will copy all files that end with "a" to the directory "mydir".

    14.            ls *a > file-list
          mkdir mydir
          cp `cat file-list` mydir
  14. Tilde expansion: When the character "~" appears by itself or before "/" it will be expanded to the home directory. If "~" appears before a word, the characters after the "~" up to the first "/" will be expanded to the home directory of the user with that login. For example:

    15.            ls ~           -- List the current directory
          ls ~george     -- List george's current directory
          ls ~george/dir -- List subdirectory "dir" in george's directory
  15. When your shell uses a file as standard input your shell should not print a prompt. This is important because your shell will be graded by redirecting small scripts into your shell and comparing the output. Use the function isatty() to find out if the input comes from a file or from a terminal.

Deadline

The deadline of this part of the project is Tuesday, April 30th, at 11:59pm,

Add a README file to the lab3-src/ directory with the following:
 

  1. Features specified in the handout that work
  2. Features specified in the handout that do not work
  3. Extra features implemented
Make sure that your shell can be built by typing "make".
 
Follow these instructions to turnin your project

1. Login to CSSUN.

2. cd to lab3-src and type "make clean"

3. Type "make" to make sure that your shell is build correctly.

4. Type "make clean" again.

5. cd one directory above lab3-src

6. Create a tar file named <user_name>.tar, where <user_name> is your CSSUN login, by typing

    tar -cf <user_name>.tar lab3-src

7. Gzip the tar file by typing

    gzip <user_name>.tar

8. Since this timestamp will be used to verify whether the work was completed on time or not, you should set the permissions on the file you submitted to make sure that the file timestamp is not changed. So this by typing:

    chmod a-w <user_name>.tar.gz

9. Mail the gzipped tar file to clay at cs dot georgetown dot edu as an attachment.