Search Results for author:
Found 56 papers, 15 papers with code
REBEL: Reinforcement Learning via Regressing Relative Rewards
While originally developed for continuous control problems, Proximal Policy Optimization (PPO) has emerged as the work-horse of a variety of reinforcement learning (RL) applications, including the fine-tuning of generative models.
Language-Based User Profiles for Recommendation
The encoder LLM generates a compact natural-language profile of the user's interests from the user's rating history.
Reviewer2: Optimizing Review Generation Through Prompt Generation
In this paper, we envision a use case where authors can receive LLM-generated reviews that uncover weak points in the current draft.
POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decomposition
We also show that POTEC provides a strict generalization of policy- and regression-based approaches and their associated assumptions.
Unbiased Offline Evaluation for Learning to Rank with Business Rules
For industrial learning-to-rank (LTR) systems, it is common that the output of a ranking model is modified, either as a results of post-processing logic that enforces business requirements, or as a result of unforeseen design flaws or bugs present in real-world production systems.
Ranking with Slot Constraints
We introduce the problem of ranking with slot constraints, which can be used to model a wide range of application problems -- from college admission with limited slots for different majors, to composing a stratified cohort of eligible participants in a medical trial.
GPT as a Baseline for Recommendation Explanation Texts
In this work, we establish a baseline potential for how modern model-generated text explanations of movie recommendations may help users, and explore what different components of these text explanations that users like or dislike, especially in contrast to existing human movie reviews.
Fairness in Ranking under Disparate Uncertainty
While ranking can make human evaluation more effective by focusing attention on the most promising options, we argue that it can introduce unfairness if the uncertainty of the underlying relevance model differs between groups of options.
Ranking with Long-Term Constraints
The feedback that users provide through their choices (e. g., clicks, purchases) is one of the most common types of data readily available for training search and recommendation algorithms.
Augmenting Holistic Review in University Admission using Natural Language Processing for Essays and Recommendation Letters
University admission at many highly selective institutions uses a holistic review process, where all aspects of the application, including protected attributes (e. g., race, gender), grades, essays, and recommendation letters are considered, to compose an excellent and diverse class.
Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling
To circumvent this variance issue, we propose a new estimator, called OffCEM, that is based on the conjunct effect model (CEM), a novel decomposition of the causal effect into a cluster effect and a residual effect.
Off-policy evaluation for learning-to-rank via interpolating the item-position model and the position-based model
A critical need for industrial recommender systems is the ability to evaluate recommendation policies offline, before deploying them to production.
Boosted Off-Policy Learning
We propose the first boosting algorithm for off-policy learning from logged bandit feedback.
Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking
Our axioms of envy-freeness and dominance over uniform ranking postulate that for a fair ranking policy every item should prefer their own rank allocation over that of any other item, and that no item should be actively disadvantaged by the rankings.
Uncertainty Quantification for Fairness in Two-Stage Recommender Systems
To this end, motivated by recent advances in uncertainty quantification, we propose two threshold-policy selection rules that can provide distribution-free and finite-sample guarantees on fairness in first-stage recommenders.
Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion
A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions.
Off-Policy Evaluation for Large Action Spaces via Embeddings
Unfortunately, when the number of actions is large, existing OPE estimators -- most of which are based on inverse propensity score weighting -- degrade severely and can suffer from extreme bias and variance.
Variance-Optimal Augmentation Logging for Counterfactual Evaluation in Contextual Bandits
Methods for offline A/B testing and counterfactual learning are seeing rapid adoption in search and recommender systems, since they allow efficient reuse of existing log data.
Improving Screening Processes via Calibrated Subset Selection
Many selection processes such as finding patients qualifying for a medical trial or retrieval pipelines in search engines consist of multiple stages, where an initial screening stage focuses the resources on shortlisting the most promising candidates.
Fairness in Ranking under Uncertainty
We call an algorithm $\phi$-fair (for $\phi \in [0, 1]$) if it has the following property for all agents $x$ and all $k$: if agent $x$ is among the top $k$ agents with respect to merit with probability at least $\rho$ (according to the posterior merit distribution), then the algorithm places the agent among the top $k$ agents in its ranking with probability at least $\phi \rho$.
Optimizing Rankings for Recommendation in Matching Markets
Based on the success of recommender systems in e-commerce, there is growing interest in their use in matching markets (e. g., labor).
Fairness of Exposure in Stochastic Bandits
Contextual bandit algorithms have become widely used for recommendation in online systems (e. g. marketplaces, music streaming, news), where they now wield substantial influence on which items get exposed to the users.
MOReL: Model-Based Offline Reinforcement Learning
In this work, we present MOReL, an algorithmic framework for model-based offline RL.
User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets
The algorithm optimizes user and item fairness as a convex optimization problem which can be solved optimally.
Content-based Music Similarity with Triplet Networks
We compare two models that are trained using different ways of picking this third song: at random vs. based on shared genre labels.
Off-policy Bandits with Deficient Support
Learning effective contextual-bandit policies from past actions of a deployed system is highly desirable in many settings (e. g. voice assistants, recommendation, search), since it enables the reuse of large amounts of log data.
Controlling Fairness and Bias in Dynamic Learning-to-Rank
Rankings are the primary interface through which many online platforms match users to items (e. g. news, products, music, video).
MOReL : Model-Based Offline Reinforcement Learning
In this work, we present MOReL, an algorithmic framework for model-based offline RL.
Policy-Gradient Training of Fair and Unbiased Ranking Functions
is an abundant and attractive source of data for learning to rank, it can produce unfair ranking policies for both exogenous and endogenous reasons.
Policy Learning for Fairness in Ranking
Conventional Learning-to-Rank (LTR) methods optimize the utility of the rankings to the users, but they are oblivious to their impact on the ranked items.
CAB: Continuous Adaptive Blending Estimator for Policy Evaluation and Learning
In addition, it is sub-differentiable such that it can be used for learning, unlike the SWITCH estimator.
Intervention Harvesting for Context-Dependent Examination-Bias Estimation
To overcome this limitation, we propose a Contextual Position-Based Model (CPBM) where the examination bias may also depend on a context vector describing the query and the user.
Consistent Position Bias Estimation without Online Interventions for Learning-to-Rank
Presentation bias is one of the key challenges when learning from implicit feedback in search engines, as it confounds the relevance signal with uninformative signals due to position in the ranking, saliency, and other presentation factors.
A General Framework for Counterfactual Learning-to-Rank
Specifically, we derive a relaxation for propensity-weighted rank-based metrics which is subdifferentiable and thus suitable for gradient-based optimization.
Deep Learning with Logged Bandit Feedback
We propose a new output layer for deep neural networks that permits the use of logged contextual bandit feedback for training.
Effective Evaluation using Logged Bandit Feedback from Multiple Loggers
In this paper, we address the question of how to estimate the performance of a new target policy when we have log data from multiple historic policies.
Large-scale Validation of Counterfactual Learning Methods: A Test-Bed
The ability to perform effective off-policy learning would revolutionize the process of building better interactive systems, such as search engines and recommendation systems for e-commerce, computational advertising and news.
Unbiased Comparative Evaluation of Ranking Functions
Eliciting relevance judgments for ranking evaluation is labor-intensive and costly, motivating careful selection of which documents to judge.
Unbounded Human Learning: Optimal Scheduling for Spaced Repetition
Second, we use this memory model to develop a stochastic model for spaced repetition systems.
Latent Skill Embedding for Personalized Lesson Sequence Recommendation
In this paper, we present the Latent Skill Embedding (LSE), a probabilistic model of students and educational content that can be used to recommend personalized sequences of lessons with the goal of helping students prepare for specific assessments.
Recommendations as Treatments: Debiasing Learning and Evaluation
Most data for evaluating and training recommender systems is subject to selection biases, either through self-selection by the users or through the actions of the recommendation system itself.
Learning Preferences for Manipulation Tasks from Online Coactive Feedback
We consider the problem of learning preferences over trajectories for mobile manipulators such as personal robots and assembly line robots.
The Self-Normalized Estimator for Counterfactual Learning
This paper identifies a severe problem of the counterfactual risk estimator typically used in batch learning from logged bandit feedback (BLBF), and proposes the use of an alternative estimator that avoids this problem. In the BLBF setting, the learner does not receive full-information feedback like in supervised learning, but observes feedback only for the actions taken by a historical policy. This makes BLBF algorithms particularly attractive for training online systems (e. g., ad placement, web search, recommendation) using their historical logs. The Counterfactual Risk Minimization (CRM) principle offers a general recipe for designing BLBF algorithms.
Using Shortlists to Support Decision Making and Improve Recommender System Performance
From a machine learning perspective, adding items to the shortlist generates a new implicit feedback signal as a by-product of exploration and decision making which can improve recommendation quality.
Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback.
Reducing Dueling Bandits to Cardinal Bandits
We present algorithms for reducing the Dueling Bandits problem to the conventional (stochastic) Multi-Armed Bandits problem.
Methods for Ordinal Peer Grading
Thus, in this paper we study the problem of automatically inferring student grades from ordinal peer feedback, as opposed to existing methods that require cardinal peer feedback.
Structured learning of sum-of-submodular higher order energy functions
Rather than trying to formulate existing higher order priors as an SoS function, we take a discriminative learning approach, effectively searching the space of SoS functions for a higher order prior that performs well on our training set.
Learning Trajectory Preferences for Manipulators via Iterative Improvement
In this paper, we propose a co-active online learning framework for teaching robots the preferences of its users for object manipulation tasks.
Semantic Labeling of 3D Point Clouds for Indoor Scenes
In our experiments over a total of 52 3D scenes of homes and offices (composed from about 550 views, having 2495 segments labeled with 27 object classes), we get a performance of 84. 06% in labeling 17 object classes for offices, and 73. 38% in labeling 17 object classes for home scenes.
Learning diverse rankings with multi-armed bandits
Algorithms for learning to rank Web documents usually assume a document's relevance is independent of other documents.
