Spring 2017

- Announcements
- Where, When, Who
- Description
- Learning Goals
- Policies
- Schedule
- Materials: Readings, Videos, and Links
- Assignments and Grading
- Other Interesting Links

- 4/27/17: Posted room for final exam: REI 281.
- 4/12/17: Posted p5.
- 3/27/17: Posted p4.
- 3/26/17: Posted a chapter on backpropagation from Rojas (1996).
- 2/26/17: Changed date of midterm exam to 3/15.
- 2/17/17: Posted p3.
- 1/30/17: Posted p2.
- 1/22/17: Posted the JASON report on AI.
- 1/10/17: Posted p1.
- 11/9/16: Created this Web page.

Class Time: | MW 9:30–10:45 AM |

Classroom: | REI 264 |

Instructor: | Mark Maloof |

Office: | 325 St. Mary's Hall |

Mailbox: | 329A St. Mary's Hall |

Office Hours: | In-person (325 STM): TR 11:00 AM–12:00 PM; online: M 10:30–11:30 AM and W 3:00–4:00 PM; or by appointment. Send me an email to get the Zoom link for online office hours. |

This graduate lecture surveys the major research areas of machine learning. Through traditional lectures and programming projects, students learn (1) to understand the foundations of machine learning, (2) to design and implement methods of machine learning, (3) to evaluate methods of machine learning, and (4) to conduct empirical evaluations of multiple methods of machine learning. The course compares and contrasts machine learning with related endeavors, such as statistical learning, pattern classification, data mining, and information retrieval. Topics include Bayesian decision theory, instance-based approaches, Bayesian methods, decision trees, rule induction, density estimation, linear classifiers, neural networks, support vector machines, ensemble methods, learning theory, evaluation, and applications. Time permitting additional topics include genetic algorithms, unsupervised learning, semi-supervised learning, outlier detection, sequence learning, and reinforcement learning. Students complete programming projects using Java.

Prerequisites: Students should have taken undergraduate courses in computer science through data structures; at the very least, students must be able to implement trees and graphs in a high-level object-oriented programming language. Students should have also taken undergraduate courses in mathematics, such as calculus, linear algebra, and probability and statistics.

Primary Text:

*Machine Learning: The Art and Science of Algorithms that Make Sense of Data*, by Peter Flach [ WWW | CUP | Amazon ]

By the end of the semester, students will be able to:

- explain the main foundations of machine learning
- understand and design object-oriented systems for machine learning
- implement methods of machine learning using a high-level programming language
- conduct performance evaluations of methods of machine learning
- design and conduct empirical studies

- Introduction: Definitions, Areas, History, Paradigms
- Bayesian Decision Theory
- Instance-based learning:
*k*-NN,*kd*-trees - Probabilistic learning: MLE, Bayes' Theorem, MAP, naive Bayes, Bayesian naive Bayes
- Density estimation: Parametric, Non-parametric, Bayesian
- Evaluation: Train/Test Methodologies, Measures, ROC Analysis
- Decision Trees: ID3, C4.5, Stumps, VFDT
- Rule Learning: Ripper, OneR
- Midterm Exam
- Neural Networks: Linear classifiers, Perceptron
- Neural Networks: Multilayer networks, Back-propagation
- Support Vector Machines: Perceptron, Dual representation
- Support Vector Machines: Margins, Kernels, Training, SMO
- Ensemble Methods: Bagging, Boosting
- Ensemble Methods: Random Forests, Voting, Weighting
- Hidden Variables:
*k*-means, Expectation-Maximization

- Autolab, for submitting projects
- Blackboard, for online discussions, document distribution, and submitting projects if something goes wrong with Autolab
- Perez-Hernandez, D. (28 March 2014).
Taking notes by hand benefits recall,
researchers find.
*The Chronicle of Higher Education*. (Read if interested.) - Mueller, P. A. and Oppenheimer, D. M. (2014).
The pen is mightier
than the keyboard: Advantages of longhand over laptop note taking.
*Psychological Science*, 25(6):1159–1168. (Read if interested.) - Flach, P. (2012).
*Machine learning: The art and science of algorithms that make sense of data*. Cambridge University Press, Cambridge. - Mitchell, T. M. (1997).
*Machine learning*. McGraw-Hill, New York, NY. - Murphy, K. P. (2012).
*Machine learning: A probabilistic perspective*[electronic resource]. MIT Press, Cambridge, MA. - Gomes, L. (20 Oct 2014).
Machine-learning maestro Michael Jordan on the delusions of
Big Data and other huge engineering efforts.
*IEEE Spectrum*. - Domingos, P. (2012).
A few useful things to know about machine learning.
*Communications of the ACM*55(10): 78–87. - Duda, R. O., and Hart, P. E. and Stork, D. G. (2000).
*Pattern classification*. John Wiley & Sons, New York, NY. - Slides: Bayesian Decision Theory.
- JASON (2017). Perspectives on Research in Artificial Intelligence and Artificial General Intelligence Relevant to DoD. Technical Report JSR-16-Task-003. The MITRE Corporation, 7515 Colshire Drive, McLean, VA 2102-7508.
- Provost, F., Fawcett, T., and Kohavi, R. (1998).
The case against accuracy estimation for comparing
induction algorithms.
In
*Proceedings of the Fifteenth International Conference on Machine Learning*, 445–453. Morgan Kaufmann, San Francisco, CA. - Fawcett, R. (2006).
An introduction to ROC analysis.
*Pattern Recognition Letters*27(8): 859–928. - Goodfellow, I., Bengio, Y. and Courville, A. (2017).
Deep
feedforward networks.
In
*Deep Learning*. MIT Press, Cambridge, MA. - Rojas, R. (1996).
The
backpropagation algorithm.
In
*Neural Networks*. Springer, Berlin-Heidelberg. - Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012).
ImageNet classification with deep convolutional neural networks.
In
*Advances in Neural Information Processing Systems 25*, 1097–1105. Curran Associates, Inc., Red Hook, NY. - Montavon, G., Orr, G. B., and Müller, K.-R. (2012). Neural networks: Tricks of the Trade, 2nd Edition. Lecture Notes in Computer Science, Volume 7700. Springer, Berlin-Heidelberg.
- LeCun, Y.A., Bottou, L., Orr, G. B., and Müller, K.-R. (2012).
Efficient BackProp
In
*Neural networks: Tricks of the Trade*, 9–48. Lecture Notes in Computer Science, Volume 7700. Springer, Berlin-Heidelberg. - Hearst, M.A., et al. (1998).
Support vector machines.
*IEEE Intelligent Systems and their Applications*13(4): 19–28. - Müller, K.-R., et al. (2001).
An introduction to kernel-based learning algorithms.
*IEEE Transactions on Neural Networks*12(2): 181–201.

- Programming Projects, 50%
- Project 1, assigned W 1/11, due M 1/30 @ 5 PM, 10 points
- Project 2, assigned M 1/30, due F 2/17 @ 5 PM, 10 points
- Project 3, assigned F 2/17 @ 5 PM, due M 3/27 @ 5 PM, 10 points
- Project 4, assigned M 3/27 @ 5 PM, due W 4/12 @ 5 PM, 10 points
- Project 5, assigned W 4/12, due M 5/1 @ 11:59 PM, 10 points
- Midterm Exam, W
~~3/1~~3/15, 20% - Final Exam, T 5/9 12:30–2:30 PM, REI 281, 30%

String Grades::getLetterGrade() { if (grade >= 94) return "A"; else if (grade >= 90) return "A-"; else if (grade >= 87) return "B+"; else if (grade >= 84) return "B"; else if (grade >= 80) return "B-"; else if (grade >= 67) return "C"; else return "F"; } // Grades::getLetterGrade

- Google Scholar
- Article: The Great AI Awakening
- Article: Meet Cepheus, the virtually unbeatable poker-playing computer
- Article: Heads-up limit hold'em poker is solved
- Survey: Research Leaders on Data Mining, Data Science, and Big Data key trends, top papers

*
Copyright © 2019 Mark Maloof. All Rights Reserved.
This material may not be published, broadcast, rewritten,
or redistributed.
*