Super Reinforcement Learning in Confounded Environments

About this event

We introduce super reinforcement learning in the batch setting, which takes the observed action as input for achieving a stronger oracle in policy learning. In the presence of unmeasured confounders, the recommendations from human agents recorded in the observed data allow us to recover certain unobserved information. Including this information in the policy search, the proposed super reinforcement learning will yield a super-policy that is guaranteed to outperform both the standard optimal policy and the behavior one (e.g., human agents' recommendations). Furthermore, to address the issue of unmeasured confounding in finding super-policies, a number of non-parametric identification results are established. Based on these identification results, we develop several super-policy learning algorithms and derive their corresponding finite-sample regret guarantees. Finally, we illustrate the superior performance of our proposal through extensive simulations and two real datasets related to improving the health policy.