Search Results for author:
Found 28 papers, 10 papers with code
Combining Open-box Simulation and Importance Sampling for Tuning Large-Scale Recommenders
Growing scale of recommender systems require extensive tuning to respond to market dynamics and system changes.
How to Solve Contextual Goal-Oriented Problems with Offline Datasets?
We present a novel method, Contextual goal-Oriented Data Augmentation (CODA), which uses commonly available unlabeled trajectories and context-goal pairs to solve Contextual Goal-Oriented (CGO) problems.
Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs
We investigate end-to-end generative optimization -- using generative models such as LLMs within the optimizer for automatic updating of general computational workflows.
On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots
We explore the use of Large Language Model (LLM-based) chatbots to power recommender systems.
The Importance of Directional Feedback for LLM-based Optimizers
We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feedback.
AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks
Large language models (LLMs) have demonstrated impressive results on natural language tasks, and security researchers are beginning to employ them in both offensive and defensive systems.
LLF-Bench: Benchmark for Interactive Learning from Language Feedback
We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from natural language feedback and instructions.
Interactive Robot Learning from Verbal Correction
A key feature of OLAF is its ability to update the robot's visuomotor neural policy based on the verbal feedback to avoid repeating mistakes in the future.
Hindsight Learning for MDPs with Exogenous Inputs
Many resource management problems require sequential decision-making under uncertainty, where the only uncertainty affecting the decision outcomes are exogenous variables outside the control of the decision-maker.
Heuristic-Guided Reinforcement Learning
On the theoretical side, we characterize properties of a good heuristic and its impact on RL acceleration.
Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
Targeting immediately measurable proxies such as clicks can lead to suboptimal recommendations due to misalignment with the long-term metric.
Provably Good Batch Off-Policy Reinforcement Learning Without Great Exploration
Doing batch RL in a way that yields a reliable new policy in large domains is challenging: a new decision policy may visit states and actions outside the support of the batch data, and function approximation and optimization with limited samples can further increase the potential of learning policies with overly optimistic estimates of their future performance.
Provably Good Batch Reinforcement Learning Without Great Exploration
Doing batch RL in a way that yields a reliable new policy in large domains is challenging: a new decision policy may visit states and actions outside the support of the batch data, and function approximation and optimization with limited samples can further increase the potential of learning policies with overly optimistic estimates of their future performance.
Improved Image Wasserstein Attacks and Defenses
Robustness against image perturbations bounded by a $\ell_p$ ball have been well-studied in recent literature.
Working Memory Graphs
Transformers have increasingly outperformed gated RNNs in obtaining new state-of-the-art results on supervised tasks involving text sequences.
Learning Calibratable Policies using Programmatic Style-Consistency
We study the problem of controllable generation of long-term sequential behaviors, where the goal is to calibrate to multiple behavior styles simultaneously.
Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
We address the opportunity to maximize the utility of an overall computing system by employing reinforcement learning to guide the configuration of the set of interacting modules that comprise the system.
Off-Policy Policy Gradient with State Distribution Correction
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method.
Multi-Preference Actor Critic
Policy gradient algorithms typically combine discounted future rewards with an estimated value function, to compute the direction and magnitude of parameter updates.
NAIL: A General Interactive Fiction Agent
Interactive Fiction (IF) games are complex textual decision making problems.
Deep Learning with Logged Bandit Feedback
We propose a new output layer for deep neural networks that permits the use of logged contextual bandit feedback for training.
Large-scale Validation of Counterfactual Learning Methods: A Test-Bed
The ability to perform effective off-policy learning would revolutionize the process of building better interactive systems, such as search engines and recommendation systems for e-commerce, computational advertising and news.
Off-policy evaluation for slate recommendation
This paper studies the evaluation of policies that recommend an ordered set of items (e. g., a ranking) based on some context---a common scenario in web search, ads, and recommendation.
Unbiased Comparative Evaluation of Ranking Functions
Eliciting relevance judgments for ranking evaluation is labor-intensive and costly, motivating careful selection of which documents to judge.
Recommendations as Treatments: Debiasing Learning and Evaluation
Most data for evaluating and training recommender systems is subject to selection biases, either through self-selection by the users or through the actions of the recommendation system itself.
The Self-Normalized Estimator for Counterfactual Learning
This paper identifies a severe problem of the counterfactual risk estimator typically used in batch learning from logged bandit feedback (BLBF), and proposes the use of an alternative estimator that avoids this problem. In the BLBF setting, the learner does not receive full-information feedback like in supervised learning, but observes feedback only for the actions taken by a historical policy. This makes BLBF algorithms particularly attractive for training online systems (e. g., ad placement, web search, recommendation) using their historical logs. The Counterfactual Risk Minimization (CRM) principle offers a general recipe for designing BLBF algorithms.
Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback.
