S&DS 265: Introductory Machine Learning

The course

This course is designed for undergraduates and graduate students who want to understand how modern machine learning works, and to be able to build and evaluate these methods themselves. It is an introductory course: the goal is to introduce the basic ideas of machine learning, and to develop each of them far enough that you can implement it, apply it, and see where it fails.

The course covers linear regression and classification, optimization, neural networks, unsupervised learning and latent-variable models, generative models including diffusion, reinforcement learning, and language models.

The course is intended for students in Statistics & Data Science, Computer Science, Engineering, and the quantitative sciences, and is useful to graduate students in Economics, SOM, and the Sciences who expect to use these methods in their own research. It is suitable for undergraduates with the appropriate prerequisites, which are linear algebra, multivariate calculus, probability, and programming experience in Python.

The Canvas site has the syllabus, announcements, grades, and assignment submission. This page is the calendar: slides, lecture notes, runnable demos, and the reading for each meeting.

Calendar

Date	Topic	Demos	Slides	Lecture Notes	Reading
Sep 2 Wed	Course logistics; the machine learning workflow	ml-workflow	Slides	Notes	ISLP §2.1–2.2
Sep 4 Fri	Mathematics for machine learning	math-for-ml	Slides	Notes	ISLP §2.1–2.2 (review); D2L Ch. 2
Sep 9 Wed	Linear regression	linear-regression-demo	Slides		ISLP §3.1–3.3
Sep 14 Mon	Regularized regression and model selection	regularized-regression-demo	Slides		ISLP §5.1, §6.2
Sep 16 Wed	Gradient descent and SGD	gradient-sgd-momentum-demo	Slides		ISLP §10.7; D2L §12.1–12.6
Sep 21 Mon	Nonlinear regression	nonlinear-regression-demo	Slides		ISLP Ch. 7
Sep 23 Wed	Generative classification		Slides		ISLP §4.4–4.5
Sep 28 Mon	Logistic classification		Slides		ISLP §4.1–4.3, §4.6
Sep 30 Wed	Margin classification		Slides		ISLP Ch. 9
Oct 5 Mon	Trees and ensembles		Slides		ISLP Ch. 8
Oct 7 Wed	Multilayer perceptrons and training		Slides		ISLP §10.1–10.2, §10.7; D2L Ch. 5
Oct 12 Mon	Optimization for neural networks		Slides		D2L §12.7–12.10
Oct 14 Wed	Midterm examination, in class
Oct 19 Mon	Convolutional networks		Slides		ISLP §10.3; D2L Ch. 7
Oct 21–25	October recess — no class
Oct 26 Mon	Principal components and low-rank structure		Slides		ISLP §12.1–12.2
Oct 28 Wed	Clustering		Slides		ISLP §12.4
Nov 2 Mon	Latent variables and EM		Slides		DL Ch. 13, §19.2
Nov 4 Wed	Autoencoders, VAEs, and VQ-VAE		Slides		DL Ch. 14; Kingma & Welling (2019)
Nov 9 Mon	Diffusion models		Slides		Ho, Jain & Abbeel (2020)
Nov 11 Wed	Latent diffusion and flow matching		Slides		Rombach et al. (2022); Lipman et al. (2023)
Nov 16 Mon	Reinforcement learning		Slides		Sutton & Barto Ch. 3–4
Nov 18 Wed	Policy optimization		Slides		Sutton & Barto Ch. 13; Schulman et al. (2017)
Nov 21–29	November recess — no class
Nov 30 Mon	Language models and transformers		Slides		D2L Ch. 11; Vaswani et al. (2017)
Dec 2 Wed	Posttraining language models		Slides		Ouyang et al. (2022); Rafailov et al. (2023)
Dec 7 Mon	Language model agents		Slides
Dec 9 Wed	Review and course wrap-up

Readings & Textbooks

The lecture slides and notebooks are self-contained. Three textbooks are recommended for a second explanation and additional depth, and all three are free online. The Reading column of the calendar names the chapter to read alongside each meeting, using these tags; where ISLP covers the topic it is the reading, and the slides and notebooks are self-contained in any case.

• [ISLP] An Introduction to Statistical Learning with Applications in Python — James, Witten, Hastie, Tibshirani, and Taylor. statlearning.com
• [D2L] Dive into Deep Learning — Zhang, Lipton, Li, and Smola. The reference for the computational material. d2l.ai
• [DL] Deep Learning — Goodfellow, Bengio, and Courville. Background and depth for the neural-network half. deeplearningbook.org

Materials

Slides, notebooks, and data can be found in the following GitHub repo: ZhuoranYang/sds265-fall26.