STICERD Econometrics Seminar Series
Karun Adusumilli (University of Pennsylvania)
Thursday 24 February 2022 14:00 - 15:30
Many of our seminars and public events this year will continue as in person or as hybrid (online and in person) events. Please check our website listings and Twitter feed @STICERD_LSE for updates.
Unless otherwise specified, in-person seminars are open to the public.
Those unable to join the seminars in-person are welcome to participate via zoom if the event is hybrid.
About this event
We provide a decision theoretic analysis of bandit experiments. The bandit setting corresponds to a dynamic programming problem, but solving this directly is typically infeasible. Working within the framework of diffusion asymptotics, we define suitable notions of asymptotic Bayes and minimax risk for bandit settings. For normally distributed rewards, the minimal Bayes risk can be characterized as the solution to a nonlinear second-order partial differential equation (PDE). Using a limit of experiments approach, we show that this PDE characterization also holds asymptotically under both parametric and non-parametric distribution of the rewards. The approach further describes the state variables it is asymptotically sufficient to restrict attention to, and therefore suggests a practical strategy for dimension reduction. The upshot is that we can approximate the dynamic programming problem defining the bandit setting with a PDE which can be efficiently solved using sparse matrix routines. We derive near-optimal policies from the numerical solutions to these equations. The proposed policies substantially dominate existing methods such Thompson sampling. The framework also allows for substantial generalizations to the bandit problem such as time discounting and pure exploration motives.
STICERD Econometrics seminars are held on Thursdays in term time at 14.00-15.30, ONLINE, unless specified otherwise.
For further information please contact Lubala Chibwe, either by email: firstname.lastname@example.org.
Please use this link to subscribe or unsubscribe to STICERD Econometrics mailing list (emetrics).