Calendar
Week 1
- Jan 8
- Lecture Slides
- Course Overview
- Readings
- Chapter 1 of RLForFinanceBook
- Jan 9
- Optional
- Python Session (Time/Location TBD)
- Jan 10
- Readings
- Chapter 3 of RLForFinanceBook
Week 2
- Jan 15
- Readings
- Chapter 4 of RLForFinanceBook (Section on POMDP not exam-able)
- Jan 17
- Lecture Slides
- Dynamic Programming Algorithms
- Readings
- Chapter 5 of RLForFinanceBook
- Assignment
- Assignment 1: Due Jan 21 11:59pm
Week 3
- Jan 22
- Readings
- Chapter 6 of RLForFinanceBook
- Optional
- Appendix F of RLForFinanceBook
- Jan 24
- Lecture Slides
- Understanding Risk-Aversion through Utility Theory (as a pre-req for Finance Applications)
- Readings
- Chapter 7 of RLForFinanceBook
- Optional
- Appendix A and C of RLForFinanceBook
Week 4
- Jan 29
- Readings
- Chapter 8 of RLForFinanceBook
- Optional
- Optional: Appendix B, C, and D of RLForFinanceBook; Some (rough) pointers on Discrete versus Continuous MDPs, and solution techniques
- Jan 31
- Lecture Slides
- Application Problems 2 and 3 - Optimal Exercise of American Options and Optimal Hedging of Derivatives in Incomplete Markets
- Readings
- Intro to Derivatives section in Chapter 9 of RLForFinanceBook
- Optional
- Appendix C and E of RLForFinanceBook; Derivatives Pricing Theory in Chapter 9 of RLForFinanceBook (Derivatives Pricing Theory is not exam-able); Relevant sections in Chapter 9 of RLForFinanceBook for Optimal Exercise and Optimal Hedging in Incomplete Markets; Foundations of Arbitrage-Free and Complete Markets
- Assignment
- Assignment 2: Due Feb 4 11:59pm
Week 5
- Feb 5
- Lecture Slides
- Application Problem 4 - Optimal Trade Order Execution
- Readings
- Optimal Trade Order Execution section in Chapter 10 of RLForFinanceBook
- Feb 7
- Lecture Slides
- RL for Prediction (Monte-Carlo and Temporal-Difference)
- Readings
- MC and TD sections in Chapter 11 of RLForFinanceBook
Week 6
- Feb 12
- Lecture Slides
- RL for Prediction (Eligibility Traces and TD(Lambda))
- Readings
- Eligibility Traces and TD(Lambda) sections in Chapter 11 of RLForFinanceBook
- Feb 14
- Lecture Slides
- RL for Prediction (Eligibility Traces and TD(Lambda))
- Readings
- Eligibility Traces and TD(Lambda) sections in Chapter 11 of RLForFinanceBook
Week 7
- Feb 19
- Lecture Slides
- RL for Control (Optimal Value Function/Optimal Policy)
- Readings
- Chapter 12 of RLForFinanceBook
- Feb 21
- Lecture Slides
- Batch RL, Experience-Replay, DQN, LSPI
- Readings
- Sections 13.1 to 13.6 in Chapter 13 of RLForFinanceBook
- Feb 23
- Assignment
- Assignment 3: Due Feb 23 11:59pm
Week 8
- Feb 26
- Lecture Slides
- Value Function Geometry and Gradient TD
- Readings
- Sections 13.7 and 13.8 in Chapter 13 of RLForFinanceBook
- Feb 28
- Lecture Slides
- Policy Gradient Algorithms
- Readings
- Chapter 14 of RLForFinanceBook
- Exam
- Timing
- February 28 @ 8:00 PM PST - March 2 @ 8:00 PM PST (48 Hours)
- Logistics
- Take home exam, to be completed individually and in compliance with the honor code, submitted via Gradescope
Week 9
- Mar 5
- Lecture Slides
- Exploration versus Exploitation (Multi-Armed Bandits)
- Readings
- Chapter 15 of RLForFinanceBook
- Mar 7
- Readings
- Chapter 16 of RLForFinanceBook + Bonus Topic
Week 10
- Mar 12
- Guest Lecture (TBD)
- Mar 14
- Final Project Presentations (Class extended from 4:30pm to 7:00pm)