Advanced Macroeconomics: Dynamic Programming and Reinforcement Learning

Objective

Recursive methods have become the cornerstone of dynamic macroeconomics. The objective of this course is to offer an intuitive yet rigorous introduction to recursive tools and their applications in macroeconomics. We will illustrate the economic implications of each concept by studying a series of classic papers. In particular, we will discuss how recursive methods have been used to analyze asset pricing, economic growth, optimal taxation and insurance, credible government policies, and unemployment.

Planning

Part I: Basic Approach
Chapter 1: From the Calculus of Variations to the Hamilton-Jacobi-Bellman Equation
Chapter 2: Dynamic Programming
- Application: Optimal Growth

Part II: Dynamic Programming under Uncertainty
Chapter 3: Markov Chains
- Application: Asset Pricing
Chapter 4: Convergence of Markov Processes
- Application: Equilibrium Search Unemployment
Chapter 5: Bayesian Learning
- Applications: Jovanovic’s Matching Model, Reputation

Part III: Dynamic Programming “Squared”
Chapter 6: Dynamic Programming with Commitment
- Application: Optimal Capital Tax
Chapter 7: Dynamic Participation Constraints
- Applications: Labor Contracts, Credible Fiscal-Monetary Policies
Chapter 8: Dynamic Incentive Constraints
- Application: New Dynamic Public Finance

References

We will cover a series of papers whose list will be communicated during the course. Most of the material will be contained in the slides, but students are encouraged to also consult the following books.

Main references

• Ljungqvist, Lars et Thomas Sargent, Recursive macroeconomic theory, MIT Press
• Stokey, Nancy, Lucas, Robert et Prescott Edward, Recursive Methods in Economic Dynamics, Harvard University Press
• Sutton, Richard and Andrew Barto. Reinforcement Learning: An Introduction. Bradford Books; second edition.

Additional references

• Fleming, Wendell et Raymond Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New-York 
• Kamien, Morton et Nancy Schwartz, Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management, Elsevier
• Luenberger, David, Optimization by Vector Space Methods, John Wiley & Sons 
• Liberzon, Daniel, Calculus of Variations and Optimal Control Theory: A Concise Introduction, Princeton University Press