Lectures

You can download the lecture slides and find the suggested readings here. We will upload lecture slides prior to their corresponding classes.

• 

  Introduction
  tl;dr: Course introduction and policy.
  [Course policy] [Intro2RL]
  

  Suggested Readings:

  • Chapter 1: Introduction
  • An Intuitive Overview of RL
  • What is reinforcement learning?
• 

  Basic Concepts in Reinforcement Learning
  tl;dr: Basics in RL
  [slides]
  

  Suggested Readings:

  • Reinforcement Learning: A Survey
  • Deep Reinforcement Learning: A Brief Survey
  • A Tutorial on Reinforcement Learning I
  • A Tutorial on Reinforcement Learning II
• 

  Multi-Armed Bandits
  tl;dr: Bandits.
  [slides]
  

  Suggested Readings:

  • Chapter 2: Multi-armed Bandits
  • Recent Advances in Multiarmed Bandits for Sequential Decision Making
  • The Contextual Bandits Problem: A New, Fast, and Simple Algorithm
  • Bandit algorithms
• 

  Markov Decision Process
  tl;dr: MDP
  [slides]
  

  Suggested Readings:-

  • Chapter 3: Finite Markov Decision Process
  • Dynamic Programming and Markov Processes
  • David Silver’s Markov Decision Process Lecture
  • A Tutorial On Partially Observable Markov Decision Processes
• 

  Dynamic Programming
  tl;dr: DP
  [slides]
  

  Suggested Readings:

  • Chapter 4: Dynamic Programming
  • A Markovian Decision Process
  • Modified Policy Iteration Algorithms for Discounted Markov Decision Problems
  • David Silver’s Planning by Dynamic Programming Lecture
• 

  Monte Carlo Methods
  tl;dr: Sampling
  [slides]
  

  Suggested Readings:-

  • Chapter 5: Monte Carlo Methods
  • Monte Carlo Simulation
  • A Survey Of Monte Carlo Tree Search Methods
  • Stanford CS234: Monte Carlo Tree Search
• 

  Temporal-Difference Learning
  tl;dr: TD
  [slides]
  

  Suggested Readings:-

  • Chapter 6: Temporal-Difference Learning
  • Richard Sutton’s talk on The Long-term of AI & Temporal-Difference Learning
  • Learning from Delayed Rewards
  • Human-Level Control Through Deep Reinforcement Learning
• 

  Policy Gradient Methods
  tl;dr: policy gradient
  [slides]
  

  Suggested Readings:-

  • Chapter 13: Policy Gradient Methods
  • David Silver’s Policy Gradient Methods
  • Policy Gradient Methods For Reinforcement Learning With Function Approximation
  • Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
• 

  Approximation Methods
  tl;dr: approximation
  [slides]
  

  Suggested Readings:-

  • Chapter 9: On-policy Prediction with Approximation
  • Chapter 10: On-policy Control with Approximation
  • David Silver’s Value Function Approximation
  • Policy gradient methods for reinforcement learning with function approximation
  • Human-level control through deep reinforcement learning
• 

  Deep Reinforcement Learning
  tl;dr: deep RL
  [slides]
  

  Suggested Readings:-

  • Chapter 9: On-policy Prediction with Approximation
  • Chapter 10: On-policy Control with Approximation
  • David Silver’s Value Function Approximation
  • Playing Atari with Deep Reinforcement Learning
  • Human-level control through deep reinforcement learning
• 

  Offline Reinforcement Learning
  tl;dr: offline RL
  
  

  Suggested Readings:

  • Chapter 11: Off-policy Methods with Approximation
  • NeurIPS 2020 Tutorial on Offline RL
  • Offline reinforcement learning: Tutorial, review, and perspectives on open problems
  • Doubly robust policy evaluation and learning
  • Offline Reinforcement Learning as One Big Sequence Modeling Problem
  • Is Pessimism Provably Efficient for Offline RL?