Search Results for author:
Found 34 papers, 13 papers with code
Aligning LLM Agents by Learning Latent Preference from User Edits
In a typical setting such as writing assistants, the user interacts with a language agent to generate a response given a context, and may optionally edit the agent response to personalize it based on their latent preference, in addition to improving the correctness.
Efficient Contextual Bandits with Uninformed Feedback Graphs
Bandits with feedback graphs are powerful online learning models that interpolate between the full information and classic bandit problems, capturing many real-life applications.
Infinite Action Contextual Bandits with Reusable Data Exhaust
For infinite action contextual bandits, smoothed regret and reduction to regression results in state-of-the-art online performance with computational cost independent of the action set: unfortunately, the resulting data exhaust does not have well-defined importance-weights.
Personalized Reward Learning with Interaction-Grounded Learning (IGL)
In an era of countless content offerings, recommender systems alleviate information overload by providing users with personalized content suggestions.
Towards Data-Driven Offline Simulations for Online Reinforcement Learning
Modern decision-making systems, from robots to web recommendation engines, are expected to adapt: to user preferences, changing circumstances or even new tasks.
Eigen Memory Trees
This work introduces the Eigen Memory Tree (EMT), a novel online memory model for sequential learning scenarios.
Deploying a Steered Query Optimizer in Production at Microsoft
Modern analytical workloads are highly heterogeneous and massively complex, making generic query optimizers untenable for many customers and scenarios.
Conditionally Risk-Averse Contextual Bandits
Contextual bandits with average-case statistical guarantees are inadequate in risk-averse situations because they might trade off degraded worst-case behaviour for better average performance.
A lower confidence sequence for the changing mean of non-negative right heavy-tailed observations with bounded mean
In certain cases this lower CS can be converted into a closed-interval CS whose width converges to zero, e. g., any bounded realization, or post contextual-bandit inference with bounded rewards and unbounded importance weights.
Anytime-valid off-policy inference for contextual bandits
Importantly, our methods can be employed while the original experiment is still running (that is, not necessarily post-hoc), when the logging policy may be itself changing (due to learning), and even if the context distributions are a highly dependent time-series (such as if they are drifting over time).
Contextual Bandits with Smooth Regret: Efficient Learning in Continuous Action Spaces
Designing efficient general-purpose contextual bandit algorithms that work with large -- or even continuous -- action spaces would facilitate application to important scenarios such as information retrieval, recommendation systems, and continuous control.
Contextual Bandits with Large Action Spaces: Made Practical
Focusing on the contextual bandit problem, recent progress provides provably efficient algorithms with strong empirical performance when the number of possible alternatives ("actions") is small, but guarantees for decision making in large, continuous action spaces have remained elusive, leading to a significant gap between theory and practice.
Interaction-Grounded Learning with Action-inclusive Feedback
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its policies.
Bellman-consistent Pessimism for Offline Reinforcement Learning
The use of pessimism, when reasoning about datasets lacking exhaustive exploration has recently gained prominence in offline reinforcement learning.
Interaction-Grounded Learning
We propose Interaction-Grounded Learning for this novel setting, in which a learner's goal is to interact with the environment with no grounding or explicit reward to optimize its policies.
ChaCha for Online AutoML
We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings.
Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
Targeting immediately measurable proxies such as clicks can lead to suboptimal recommendations due to misalignment with the long-term metric.
Off-policy Confidence Sequences
We develop confidence bounds that hold uniformly over time for off-policy evaluation in the contextual bandit setting.
Empirical Likelihood for Contextual Bandits
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting.
Lessons from Contextual Bandit Learning in a Customer Support Bot
In this work, we describe practical lessons we have learned from successfully using contextual bandits (CBs) to improve key business metrics of the Microsoft Virtual Agent for customer support.
Contextual Memory Trees
We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size.
Logarithmic Time One-Against-Some
Compared to previous approaches, we obtain substantially better statistical performance for two reasons: First, we prove a tighter and more complete boosting theorem, and second we translate the results more directly into an algorithm.
Active Information Acquisition
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task.
General Reinforcement Learning
Reinforcement Learning (RL)
+1
A Hierarchical Spectral Method for Extreme Classification
Extreme classification problems are multiclass and multilabel classification problems where the number of outputs is so large that straightforward strategies are neither statistically nor computationally viable.
Fast Label Embeddings for Extremely Large Output Spaces
Many modern multiclass and multilabel problems are characterized by increasingly large output spaces.
Learning Reductions that Really Work
We provide a summary of the mathematical and computational techniques that have enabled learning reductions to effectively address a wide class of problems, and show that this approach to solving machine learning problems can be broadly useful.
Scalable Multilabel Prediction via Randomized Methods
In this work we show that a generic regularized nonlinearity mapping independent predictions to joint predictions is sufficient to achieve state-of-the-art performance on a variety of benchmark problems.
Fast Label Embeddings via Randomized Linear Algebra
Many modern multiclass and multilabel problems are characterized by increasingly large output spaces.
A Randomized Algorithm for CCA
We present RandomizedCCA, a randomized algorithm for computing canonical analysis, suitable for large datasets stored either out of core or on a distributed file system.
Normalized Online Learning
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale.
Combining Structured and Unstructured Randomness in Large Scale PCA
Principal Component Analysis (PCA) is a ubiquitous tool with many applications in machine learning including feature construction, subspace embedding, and outlier detection.
Discriminative Features via Generalized Eigenvectors
Representing examples in a way that is compatible with the underlying classifier can greatly enhance the performance of a learning system.
Loss-Proportional Subsampling for Subsequent ERM
We propose a sampling scheme suitable for reducing a data set prior to selecting a hypothesis with minimum empirical risk.
Normalized Online Learning
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale.
