COSC-575: Machine Learning

Spring 2017

Announcements
Where, When, Who
Description
Learning Goals
Policies
Schedule
Materials: Readings, Videos, and Links
Assignments and Grading
Other Interesting Links

Announcements

4/27/17: Posted room for final exam: REI 281.

4/12/17: Posted p5.
3/27/17: Posted p4.
3/26/17: Posted a chapter on backpropagation from Rojas (1996).
2/26/17: Changed date of midterm exam to 3/15.
2/17/17: Posted p3.
1/30/17: Posted p2.
1/22/17: Posted the JASON report on AI.
1/10/17: Posted p1.
11/9/16: Created this Web page.

Where, When, Who

Class Time: MW 9:30–10:45 AM

Classroom: REI 264

Instructor: Mark Maloof

Office: 325 St. Mary's Hall

Mailbox: 329A St. Mary's Hall

Office Hours: In-person (325 STM): TR 11:00 AM–12:00 PM; online: M 10:30–11:30 AM and W 3:00–4:00 PM; or by appointment. Send me an email to get the Zoom link for online office hours.

Description

This graduate lecture surveys the major research areas of machine learning. Through traditional lectures and programming projects, students learn (1) to understand the foundations of machine learning, (2) to design and implement methods of machine learning, (3) to evaluate methods of machine learning, and (4) to conduct empirical evaluations of multiple methods of machine learning. The course compares and contrasts machine learning with related endeavors, such as statistical learning, pattern classification, data mining, and information retrieval. Topics include Bayesian decision theory, instance-based approaches, Bayesian methods, decision trees, rule induction, density estimation, linear classifiers, neural networks, support vector machines, ensemble methods, learning theory, evaluation, and applications. Time permitting additional topics include genetic algorithms, unsupervised learning, semi-supervised learning, outlier detection, sequence learning, and reinforcement learning. Students complete programming projects using Java.

Prerequisites: Students should have taken undergraduate courses in computer science through data structures; at the very least, students must be able to implement trees and graphs in a high-level object-oriented programming language. Students should have also taken undergraduate courses in mathematics, such as calculus, linear algebra, and probability and statistics.

Primary Text:

Machine Learning: The Art and Science of Algorithms that Make Sense of Data, by Peter Flach [ WWW | CUP | Amazon ]

Learning Goals

By the end of the semester, students will be able to:

explain the main foundations of machine learning
understand and design object-oriented systems for machine learning
implement methods of machine learning using a high-level programming language
conduct performance evaluations of methods of machine learning
design and conduct empirical studies

Policies

Schedule

Introduction: Definitions, Areas, History, Paradigms
Bayesian Decision Theory
Instance-based learning: k-NN, kd-trees
Probabilistic learning: MLE, Bayes' Theorem, MAP, naive Bayes, Bayesian naive Bayes
Density estimation: Parametric, Non-parametric, Bayesian
Evaluation: Train/Test Methodologies, Measures, ROC Analysis
Decision Trees: ID3, C4.5, Stumps, VFDT
Rule Learning: Ripper, OneR
Midterm Exam
Neural Networks: Linear classifiers, Perceptron
Neural Networks: Multilayer networks, Back-propagation
Support Vector Machines: Perceptron, Dual representation
Support Vector Machines: Margins, Kernels, Training, SMO
Ensemble Methods: Bagging, Boosting
Ensemble Methods: Random Forests, Voting, Weighting
Hidden Variables: k-means, Expectation-Maximization

Materials: Readings, Videos, and Links

Autolab, for submitting projects
Blackboard, for online discussions, document distribution, and submitting projects if something goes wrong with Autolab

Perez-Hernandez, D. (28 March 2014). Taking notes by hand benefits recall, researchers find. The Chronicle of Higher Education. (Read if interested.)
Mueller, P. A. and Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6):1159–1168. (Read if interested.)
Flach, P. (2012). Machine learning: The art and science of algorithms that make sense of data. Cambridge University Press, Cambridge.
Mitchell, T. M. (1997). Machine learning. McGraw-Hill, New York, NY.
Murphy, K. P. (2012). Machine learning: A probabilistic perspective [electronic resource]. MIT Press, Cambridge, MA.
Gomes, L. (20 Oct 2014). Machine-learning maestro Michael Jordan on the delusions of Big Data and other huge engineering efforts. IEEE Spectrum.
Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM 55(10): 78–87.
Duda, R. O., and Hart, P. E. and Stork, D. G. (2000). Pattern classification. John Wiley & Sons, New York, NY.
Slides: Bayesian Decision Theory.
JASON (2017). Perspectives on Research in Artificial Intelligence and Artificial General Intelligence Relevant to DoD. Technical Report JSR-16-Task-003. The MITRE Corporation, 7515 Colshire Drive, McLean, VA 2102-7508.
Provost, F., Fawcett, T., and Kohavi, R. (1998). The case against accuracy estimation for comparing induction algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, 445–453. Morgan Kaufmann, San Francisco, CA.
Fawcett, R. (2006). An introduction to ROC analysis. Pattern Recognition Letters 27(8): 859–928.
Goodfellow, I., Bengio, Y. and Courville, A. (2017). Deep feedforward networks. In Deep Learning. MIT Press, Cambridge, MA.
Rojas, R. (1996). The backpropagation algorithm. In Neural Networks. Springer, Berlin-Heidelberg.
Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, 1097–1105. Curran Associates, Inc., Red Hook, NY.
Montavon, G., Orr, G. B., and Müller, K.-R. (2012). Neural networks: Tricks of the Trade, 2nd Edition. Lecture Notes in Computer Science, Volume 7700. Springer, Berlin-Heidelberg.
LeCun, Y.A., Bottou, L., Orr, G. B., and Müller, K.-R. (2012). Efficient BackProp In Neural networks: Tricks of the Trade, 9–48. Lecture Notes in Computer Science, Volume 7700. Springer, Berlin-Heidelberg.
Hearst, M.A., et al. (1998). Support vector machines. IEEE Intelligent Systems and their Applications 13(4): 19–28.
Müller, K.-R., et al. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks 12(2): 181–201.

Assignments and Grading

Programming Projects, 50%

Project 1, assigned W 1/11, due M 1/30 @ 5 PM, 10 points
Project 2, assigned M 1/30, due F 2/17 @ 5 PM, 10 points
Project 3, assigned F 2/17 @ 5 PM, due M 3/27 @ 5 PM, 10 points
Project 4, assigned M 3/27 @ 5 PM, due W 4/12 @ 5 PM, 10 points
Project 5, assigned W 4/12, due M 5/1 @ 11:59 PM, 10 points

Midterm Exam, W ~~3/1~~ 3/15, 20%
Final Exam, T 5/9 12:30–2:30 PM, REI 281, 30%

String Grades::getLetterGrade()
{
  if (grade >= 94)
    return "A";
  else if (grade >= 90)
    return "A-";
  else if (grade >= 87)
    return "B+";
  else if (grade >= 84)
    return "B";
  else if (grade >= 80)
    return "B-";
  else if (grade >= 67)
    return "C";
  else
    return "F";
} // Grades::getLetterGrade

Class Time:	MW 9:30–10:45 AM
Classroom:	REI 264

Instructor:	Mark Maloof
Office:	325 St. Mary's Hall
Mailbox:	329A St. Mary's Hall
Office Hours:	In-person (325 STM): TR 11:00 AM–12:00 PM; online: M 10:30–11:30 AM and W 3:00–4:00 PM; or by appointment. Send me an email to get the Zoom link for online office hours.