CSCI 416/516 Fall 2024: Introduction to Machine Learning

Overview

Machine learning (ML) is a set of techniques that allow computers to learn from data and experience, rather than requiring humans to specify the desired behaviour by hand. ML has become increasingly central both in AI as an academic field, and in industry. This course provides a broad introduction to some of the most commonly used ML algorithms. It also serves to introduce key algorithmic principles which will serve as a foundation for more advanced courses, such as Deep Learning.

Syllabus

The syllabus for both CSCI 416 and CSCI 516 are available.

Useful Links

Important Dates

Agenda Item Due Date Time Location
Project Proposal (See Instructions) 10/04 23:59 Blackboard
Midterm Exam (See Practice Midterm) 10/16 14:00-15:30 McGlothlin-Street 020
Project Report (See Instructions) 12/06 23:59 Blackboard
Final Exam 12/12 14:00-17:00 McGlothlin-Street 020

Announcements

Where and When

Class will be held synchronously every week, including a combination of lectures and office hours. Students are encouraged to attend both the lectures and office hours each week. There will be two mandatory tests held. Students are also encouraged to reach out to the instructor(s) with any questions or concerns.

Instructor Lecture Time Lecture Location Office Hours Office Location Email
Ashley Gao M/W: 14:00-15:20 McGlothlin-Street Hall 020 Tu/Th: 12:00-1:30 McGlothlin-Street Hall 004 ygao18@wm.edu
Jiacheng Shi N/A N/A Tu: 9:00-12:00 McGlothlin-Street Hall 139 jshi12@wm.edu

Homeworks

This class will have 4 homeworks. Please come back to this page once it is annouced during the class that a homework is posted. The homeworks are collected using Blackboard.

Homeworks will be accepted up to 3 days late, but 10% will be deducted for each day late, rounded up to the nearest day. No credit will be given for assignments submitted after 3 days. Extensions will be granted only in special situations with valid proof (e.g. Doctor’s note).

In each homework, there will be additional 1-2 question(s) required for graduate students but optional to undergraduate students. If an undergraduate student answers correctly to such questions, they will receive extra credits, in addition to the (at maximum) 10 pts they could obtain on a homework.

# Out Due Materials
1 9/13 10/4 [Homework #1] with LaTeX
2 10/4 10/2511/01 [Homework #2] with LaTeX
3 10/2510/27 11/0811/15 [Homework #3] with LaTeX
4 11/8 11/2712/4 [Homework #4] with LaTeX

Grading

Schedule

Suggested readings are optional; they are resources we recommend to help you understand the course material. All of the textbooks listed below are freely available online.

Note that this schedule is tentative and will be updated once a topic is covered in the lecture(s).

Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
MC = Multivariate Calculus, on Youtube.

# Dates Topic Materials Instructor Notes
1 08/28, 09/04 Lecture: Introduction, K-Nearest Neighbors
Tutorial: Curse of Dimensionality 		Lecture: [Lecture #1], [Lecture #2]
Tutorial: [Tutorial]

ESL: 1, 2.1-2.3, 2.5
Domingos, 2012. A few useful things to know about machine learning.
Breiman, 2001. Statistical Modeling: The Two Cultures
2 09/09, 09/11, 09/16 Lecture: Linear Regression, Optimization
Tutorial: Anaconda Programming, Linear Regressor with SGD 		Lecture: [Lecture]
Tutorial: [Tutorial #1], [Tutorial #2]
Notes: [Notes #1], [Notes #2], [Notes #3]
Bishop: 3.1
ESL: 3.1 - 3.2
3 09/18, 09/23, 09/25, 09/30 Lecture: Logistic Regression, Multiclass Classification
Tutorial: Logistic Regression
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
Bishop: 4.1, 4.3
ESL: 4.1-4.2, 4.4, 11
MC: Partial Derivatives - Multivariable Calculus
4 10/02, 10/07, 10/09 Lecture: Support Vector Machines and Kernels
Tutorial: SVM
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
Ng: Support Vector Machines
Sklearn: Support Vector Regression
5 10/14, 10/16 Lecture: Midterm Review & Midterm Exam
Tutorial: Practice Midterm
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]

6 10/21, 10/23 Lecture: Decision Trees
Tutorial: Decision Trees
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
Bishop: 3.2
ESL: 2.9, 9.2
7 10/28 Lecture: Boosting & Ensemble Learning
Tutorial: AdaBoost
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
ESL: 8.7, 10.1-10.5
8 10/30, 11/4, 11/11 Lecture: Multilayer Perceptrons
Tutorial: MLP
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
Bishop: 5.1-5.3
9 11/13, 11/18, 11/20 Lecture: Convolutional Neural Networks
Tutorial: CNN
Lecture: [Lecture]
Tutorial: [Tutorial]
Notes: [Notes]
Ng: CNN for Computer Vision
10 11/25 Lecture: Attention & Transformers
Tutorial: Vision Transformers
Lecture: [Lecture]
Tutorial: [Tutorial]
Phi: Illustrated Guide to Transformers
Vaswani: Attention Is All You Need
11 12/02, 12/04 Lecture: Philosophy of AI/ML in Literary Traditions
Tutorial: Final Exam Practice
Lecture: [Lecture]
Tutorial: [Tutorial]
Gao, 2019. Personal Statement

Practice Exams

The Practice Final, as well as the comphrensive Final Study Guide with LaTeX, are posted.

We will go over the Practice Final in class. Note: The Final is cumulative!

The Practice Midterm, as well as the comphrensive Midterm Study Guide with LaTeX, are posted.

We will go over the Practice Midterm in class.

Final Project

20% of your total mark is allocated to a final project, which will require you to apply several algorithms to a challenging problem and to write a short report analyzing the results. You are allowed to collaborate with at most 2 classmates on the final project. In other words, the maximal number of people a group can have is 3.

Instructions on the final project is avaiable.