Teaching

S&DS 431/631: Optimization and Computation (Fall 2024, Fall 2023, Fall 2022)

Course Description:

This course is designed for undergraduates and graduate students in Statistics & Data Science who need to know about optimization and the essentials of numerical algorithm design and analysis. It is an introduction to more advanced courses in optimization. The overarching goal of the course is to teach students how to design optimization algorithms for Machine Learning and Data Analysis (in their own research, as applies to graduate students). The course is useful for graduate students in programs in Economics, SOM, and the Sciences. It is also suitable for undergraduates with the appropriate prerequisites, which are knowledge of linear algebra, multivariate calculus, and probability.

Topics Covered in Fall 2024 Version:

In the 2024 version, we cover materials on the optimization & computation aspects of artificial intelligence. Concretely, this course covers the following topics:

  • Backpropagation and Automatic Differentiation
  • Background on Convex Optimization: Convex Sets, Convex Functions, Convex Optimization Problems
  • Gradient Descent and First-Order Methods for Convex Optimization and Deep Learning
  • Diffusion Models and Latent Diffusion
  • Transformer and a Primer on Mechanistic Interpretability
  • Newton and Interior Point Methods

S&DS 432/632: S&DS 432 01 (SP24): Advanced Optimization Techniques (Spring 2024)

Course Description:

This course delves deep into the fundamental theory and algorithms in optimization with a special emphasis on convex optimization and additional advanced topics. We will explore in depth several optimization methods that are suitable for large-scale problems arising in various applications. These algorithms include gradient methods, proximal methods, mirror descent, Nesterov’s accelerated methods, ADMM, quasi-Newton methods, stochastic optimization, variance reduction, extragradient methods, as well as some methods developed for nonconvex settings.

Topics Covered:

  • Review of Basic Convex Analysis
  • Duality Theory
  • Newton Methods and Interior Point Methods
  • First-Order Methods: Gradient Descent, Subgradient Descent, Proximal Methods and Mirror Descent
  • Deep Learning: Transformer Models and Implicit Layers

S&DS 685: Theory of Reinforcement Learning (Spring 2023)

Course Description:

There has been a surge of research interest in reinforcement learning recently, fueled by exciting applications of reinforcement learning techniques to various challenging decision-making problems in artificial intelligence, robotics, and natural sciences. Many of these advances were made possible by a combination of innovative use of flexible neural network architectures, modern optimization techniques, and new and classical RL algorithms. However, a systematic understanding of when, why, and to what extent these algorithms work remains active in ongoing research. This course aims to introduce the theoretical foundations of reinforcement learning, with the goal of equipping students with the necessary tools for conducting research.

This graduate-level course focuses on the theoretical and algorithmic foundations of Reinforcement Learning. Specifically, there are four main themes of the course:

(a) fundamentals of RL (Markov decision process, planning algorithms, Q-learning and temporal difference learning, policy gradient)

(b) online RL (bandit algorithms, online learning, exploration)

(c) offline RL (off-policy evaluation, offline policy learning)

(d) further topics (multi-agent RL, partial observability).