COSC-3450 Project 4

COSC-3450: Artificial Intelligence

Project 4
Fall 2024

Due: F 4/12 @ 5 P.M.
10 points

Implement Flach's version of the perceptron as Perceptron. Implement Zurada's multi-layer feed-forward neural network trained with the backpropagation algorithm as BP.

I put some starter code on cs-class (aka class-1.cs.georgetown.edu). To get started, log on to cs-class and copy over the following zip file:

cs-class-1% cd
cs-class-1% cp ~maloofm/cosc3450/p4.zip ./
cs-class-1% unzip p4.zip

In the p4 directory, you will find encoded data sets and partial class definitions for DataSet, Classifier, BP, Perceptron, and FailedToConvergeException.

The data files contain comments and encoded examples. Lines that start with the percent sign (%) are comments. You must not change the format of the data files.

For these two learning methods, the train methods should terminate if they have not converged after 50,000 epochs (i.e., one complete pass through all of the training examples) by throwing FailedToConvergeException, which I derived from RuntimeException. For Perceptron and BP, use the learning rate of 0.9.

For BP, initialize the weights to small random values. Make sure BP uses a different random seed each time it is constructed so there are random restarts. Use a minimum error of 0.1 and the method setJ to set the number of units in the hidden layer including the bias unit.

Perceptron.main and BP.main should process command-line arguments and handle three use cases. The command-line argument -t specifies the name of the training file. The command-line argument -T specifies the name of the testing file. The command-line argument -p specifies the portion of a data set that is to be used for training, with the remaining examples used for testing. Finally, for BP.main, the command-line argument -J, which is required, specifies the number of hidden units including the bias unit. You do not need to worry about missing or conflicting command-line arguments.

There are three use cases. In all cases, the main method should print the model and the model's accuracy on the testing examples.

If the user provides only the name of a training file using -t, then the learner should train on the examples in the file and classify them.
If the user provides the names of training and testing files, then the learner should train on the examples in the training file and classify the examples in the testing file.
If the user provides the name of a training file and a proportion, then the learner should perform an evaluation using the hold-out method.

As an example, if the user types

cs-class-1% java BP -t monks1.tr.dta -J 4 -T monks1.te.dta

BP.main will construct a neural network with four hidden units (including the bias unit) and train it on the examples in the file monks1.tr.dta. It will then test the model on the examples in monks1.te.dta. My implementation produces the following output:

cs-class-1% java BP -t monks1.tr.dta -J 4 -T monks1.te.dta
Backprop: 
V = [[ 3.665, -4.077, -3.537, 3.760, -.134, .298, .026, .176, .027, 1.671, -.012, 2.493 ]
     [ 1.355, 1.377, 1.811, 1.597, -.560, .162, .025, -4.106, -4.559, 1.751, .439, .262 ]
     [ -4.215, 2.872, 4.099, -3.400, .027, .646, .249, -.015, -.536, 2.278, .659, 1.893 ]]
W = [[ 7.208, -7.414, 7.098, 3.293 ]
     [ -7.209, 7.415, -7.099, -3.293 ]]

Accuracy: 1.0

Keep in mind that training a neural network is a stochastic process. It is unlikely that you will obtain the same connection weights. You should, however, obtain the same or a similar accuracy.

As we have discussed in class, training a neural network can be computationally expensive, and it may not converge. Develop your implementation by training and testing on the small data sets, such xor, until you are confident that everything seems to be working. Use the hold-out method for larger data sets, such as votes and sonar.

If you're using cs-class-1 for experimentation, please be mindful of other users on the system. If you want to kick off a big training job in the background and go to the Tombs, please be nice and use nice. For example:

cs-class-1$ nice java BP -J 256 -t cats-and-dogs.dta < /dev/null >| output &

This command runs BP with a nice priority. The fancy redirects prevent ssh from hanging when you log out and write the output of BP to the file named output. The final ampersand runs the job in the background, where it will run for a long time. At this point, you can log out and head over the Tombs.

When you reconnect to cs-class-1, you can check to see if the job is still running by looking for the name of your executable —in this case, java BP—in the list of active processes:

cs-class-1$ ps -ef | grep BP
maloofm  16205     1 98 15:37 ?        00:00:13 java BP -J 256 -t cats-and-dogs.dta
maloofm  17920 16238  0 15:45 pts/5    00:00:00 grep BP

You can examine the contents of the output file by typing:

cs-class-1$ more output

If for some reason your implementation of BP seems like it will never terminate, please do not leave it running. To kill a job, look in the list of active processes for the job's ID:

cs-class-1$ ps -ef | grep BP
maloofm  16205     1 98 15:37 ?        00:00:13 java BP -J 256 -t cats-and-dogs.dta
maloofm  17920 16238  0 15:45 pts/5    00:00:00 grep BP

In this case, it is 16205. Use the kill command to kill the process. It should no longer appear in the process list.

cs-class-1$ kill 16205
cs-class-1$ ps -ef | grep BP
maloofm  19129 16238  0 15:55 pts/5    00:00:00 grep BP

Instructions for Submission

In a file named HONOR, please include the statement:

In accordance with the class policies and Georgetown's Honor System,
I certify that, with the exceptions of the class resources and those
items noted below, I have neither given nor received any assistance
on this project.

Name
NetID

Include this file in your zip file submit.zip.

Submit p4 exactly like you submitted p3. Make sure you remove all debugging output before submitting. You need to plan for the potentially long running times of BP on Autolab. Unlike with previous projects, it could several minutes for a run to complete. Plan accordingly.

Plan B

If Autolab is down, upload your zip file to Canvas.