Unbiased Offline Evaluation for Learning to Rank with Business Rules

no code implementations • 3 Nov 2023 • Matej Jakimov, Alexander Buchholz, Yannik Stein, Thorsten Joachims

For industrial learning-to-rank (LTR) systems, it is common that the output of a ranking model is modified, either as a results of post-processing logic that enforces business requirements, or as a result of unforeseen design flaws or bugs present in real-world production systems.

Learning-To-Rank Off-policy evaluation

Fair Effect Attribution in Parallel Online Experiments

no code implementations • 15 Oct 2022 • Alexander Buchholz, Vito Bellini, Giuseppe Di Benedetto, Yannik Stein, Matteo Ruffini, Fabian Moerchen

We suggest an approach to measure and disentangle the effect of simultaneous experiments by providing a cost sharing approach based on Shapley values.

Attribute Causal Inference +1

Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling

no code implementations • 12 May 2022 • Alexander Buchholz, Jan Malte Lichtenberg, Giuseppe Di Benedetto, Yannik Stein, Vito Bellini, Matteo Ruffini

When adopting the PL model as a ranking policy, both tasks require the computation of expectations with respect to the model.

Computational Efficiency counterfactual +1

Ranker-agnostic Contextual Position Bias Estimation

no code implementations • 28 Jul 2021 • Oriol Barbany Mayor, Vito Bellini, Alexander Buchholz, Giuseppe Di Benedetto, Diego Marco Granziol, Matteo Ruffini, Yannik Stein

This paper introduces a method for modeling the probability of an item being seen in different contexts, e. g., for different users, with a single estimator.

Learning-To-Rank Position

Learning to Rank in the Position Based Model with Bandit Feedback

no code implementations • 27 Apr 2020 • Beyza Ermis, Patrick Ernst, Yannik Stein, Giovanni Zappella

Personalization is a crucial aspect of many online experiences.

Learning-To-Rank Multi-Armed Bandits +2