Welcome to the Winter 2026 edition of CME 241: Foundations of Reinforcement Learning with Applications in Finance.

Table of contents

  1. Overview of the Course
  2. Learning Material
  3. Course Project
  4. Grading
  5. Access and Accommodations

Overview of the Course

  • Theory of Markov Decision Processes (MDPs)
  • Dynamic Programming (DP) Algorithms
  • Backward Induction (BI) and Approximate DP (ADP) Algorithms
  • Reinforcement Learning (RL) Algorithms
  • Plenty of Python implementations of models and algorithms
  • We apply these algorithms to 5 Financial/Trading problems:
    • (Dynamic) Asset-Allocation to maximize Utility of Consumption
    • Pricing and Hedging of Derivatives in an Incomplete Market
    • Optimal Exercise/Stopping of Path-dependent American Options
    • Optimal Trade Order Execution (managing Price Impact)
    • Optimal Market-Making (Bid/Ask managing Inventory Risk)
  • By treating each of the problems as MDPs (i.e., Stochastic Control)
  • We will go over classical/analytical solutions to these problems
  • Then we will introduce real-world considerations, and tackle with RL (or DP)
  • The course blends Theory/Mathematics, Programming/Algorithms and Real-World Financial Nuances

Learning Material

Course Project

The course project is meant to be configured within the realm of a broad topic that is of high practical relevance, has traditionally been challenging to solve, and is well suited for appropriate formulation and solution using the mathematical and computational techniques taught in this course. Each project group can specialize the project scope within this broad topic.

The broad topic is Personal Finance Optimization. At various stages of our lives, we are trying to grow our wealth while simultaneously making payments on our obligations and tactically paying for optional/selective products and services that give us specific forms of satisfaction or enjoyment. We typically grow our wealth by receiving salary, bonuses and equity while doing jobs and by making appropriate choices on investments made with our savings (eg: stocks, real-estate, gold etc.) Our payments obligations are typically our rent or mortgage, vehicle payments, utility bills etc. Our tactical payments for optional/selective products/services giving us satisfaction/enjoyment can be wide-ranging - a vacation, a luxury car, a birthday celebration, a gift for a loved one etc. The objective is to maximize the aggregated Utility of Consumption (something that you will learn about during the course). The problem is to make a sequence of optimal decisions on the optional/selective products/services, amidst significant uncertainty around how our lives will transpire. See details here.

Grading

  • 45%: Assignments (15% per assignment, 3 total)
  • 50%: Course Project
  • 5%: Attendance (18/20 minimum lecture attendance)

Access and Accommodations

Stanford is committed to providing equal educational opportunities for disabled students. Disabled students are a valued and essential part of the Stanford community. We welcome you to our class.

If you experience disability, please register with the Office of Accessible Education (OAE). Professional staff will evaluate your needs, support appropriate and reasonable accommodations, and prepare an Academic Accommodation Letter for faculty. To get started, or to re-initiate services, please visit oae.stanford.edu

If you already have an Academic Accommodation Letter, we invite you to share your letter with us. Academic Accommodation Letters should be shared at the earliest possible opportunity so we may partner with you and OAE to identify any barriers to access and inclusion that might be encountered in your experience of this course.