Teaching

S&DS 431/631: Optimization and Computation (Fall 2023, Fall 2022)

Course Description:

This course is designed for undergraduates and graduate students in Statistics & Data Science who need to know about optimization and the essentials of numerical algorithm design and analysis. It is an introduction to more advanced courses in optimization. The overarching goal of the course is to teach students how to design optimization algorithms for Machine Learning and Data Analysis (in their own research, as applies to graduate students). The course is useful for graduate students in programs in Economics, SOM, and the Sciences. It is also suitable for undergraduates with the appropriate prerequisites, which are knowledge of linear algebra, multivariate calculus, and probability.

This course is not for students who have taken Optimization Techniques (S&DS 430 / ENAS 530 / EENG 437 / ECON 413). Students who have the time to take many courses are encouraged to instead take a combination of Algorithm Design (CPSC 365/366), Numerical Computation (CPSC 440), Applied Numerical Methods I (ENAS 440), Topics in Numerical Computation (CPSC 640), and some more advanced courses in optimization.

S&DS 685: Theory of Reinforcement Learning (Spring 2023)

Course Description:

There has been a surge of research interest in reinforcement learning recently, fueled by exciting applications of reinforcement learning techniques to various challenging decision-making problems in artificial intelligence, robotics, and natural sciences. Many of these advances were made possible by a combination of innovative use of flexible neural network architectures, modern optimization techniques, and new and classical RL algorithms. However, a systematic understanding of when, why, and to what extent these algorithms work remains active in ongoing research. This course aims to introduce the theoretical foundations of reinforcement learning, with the goal of equipping students with the necessary tools for conducting research.

This graduate-level course focuses on the theoretical and algorithmic foundations of Reinforcement Learning. Specifically, there are four main themes of the course:

(a) fundamentals of RL (Markov decision process, planning algorithms, Q-learning and temporal difference learning, policy gradient)

(b) online RL (bandit algorithms, online learning, exploration)

(c) offline RL (off-policy evaluation, offline policy learning)

(d) further topics (multi-agent RL, partial observability).