Efficient Counterfactual Learning from Bandit Feedback

10 Sep 2018  ·  Yusuke Narita, Shota Yasui, Kohei Yata ·

What is the most statistically efficient way to do off-policy evaluation and optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical confidence compared to a state-of-the-art benchmark.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Causal Inference IDHP Average Treatment Effect Error -0.225 # 3
Visual Object Tracking VOT2014 Expected Average Overlap (EAO) 1.047 # 1

Methods


No methods listed for this paper. Add relevant methods here