Search Results for author:
Found 77 papers, 25 papers with code
Towards Principled Representation Learning from Videos for Reinforcement Learning
We study two types of settings: one where there is iid noise in the observation, and a more challenging setting where there is also the presence of exogenous noise, which is non-iid noise that is temporally correlated, such as the motion of people or cars in the background.
Position Paper: Agent AI Towards a Holistic Intelligence
Recent advancements in large foundation models have remarkably enhanced our understanding of sensory information in open-world environments.
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks.
PcLast: Discovering Plannable Continuous Latent States
Goal-conditioned planning benefits from learned low-dimensional representations of rich, high-dimensional observations.
Streaming Active Learning with Deep Neural Networks
Active learning is perhaps most naturally posed as an online learning problem.
Towards Data-Driven Offline Simulations for Online Reinforcement Learning
Modern decision-making systems, from robots to web recommendation engines, are expected to adapt: to user preferences, changing circumstances or even new tasks.
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information
We find that contemporary representation learning techniques can fail on datasets where the noise is a complex and time dependent process, which is prevalent in practical applications.
Eigen Memory Trees
This work introduces the Eigen Memory Tree (EMT), a novel online memory model for sequential learning scenarios.
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models
In many sequential decision-making tasks, the agent is not able to model the full complexity of the world, which consists of multitudes of relevant and irrelevant information.
Contextual Bandits with Large Action Spaces: Made Practical
Focusing on the contextual bandit problem, recent progress provides provably efficient algorithms with strong empirical performance when the number of possible alternatives ("actions") is small, but guarantees for decision making in large, continuous action spaces have remained elusive, leading to a significant gap between theory and practice.
Interaction-Grounded Learning with Action-inclusive Feedback
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its policies.
Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information
In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at hand.
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning
Large-scale machine learning systems often involve data distributed across a collection of users.
Provable RL with Exogenous Distractors via Multistep Inverse Dynamics
We initiate the formal study of latent state discovery in the presence of such exogenous noise sources by proposing a new model, the Exogenous Block MDP (EX-BMDP), for rich observation RL.
Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics
We initiate the formal study of latent state discovery in the presence of such exogenous noise sources by proposing a new model, the Exogenous Block MDP (EX-BMDP), for rich observation RL.
Interaction-Grounded Learning
We propose Interaction-Grounded Learning for this novel setting, in which a learner's goal is to interact with the environment with no grounding or explicit reward to optimize its policies.
ChaCha for Online AutoML
We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings.
Provable Rich Observation Reinforcement Learning with Combinatorial Latent States
We propose a novel setting for reinforcement learning that combines two common real-world difficulties: presence of observations (such as camera images) and factored states (such as location of objects).
Resonance: Replacing Software Constants with Context-Aware Models in Real-time Communication
Large software systems tune hundreds of 'constants' to optimize their runtime performance.
Learning the Linear Quadratic Regulator from Nonlinear Observations
We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class.
Better Parameter-free Stochastic Optimization with ODE Updates for Coin-Betting
Parameter-free stochastic gradient descent (PFSGD) algorithms do not require setting learning rates while achieving optimal theoretical performance.
Efficient Contextual Bandits with Continuous Actions
We create a computationally tractable algorithm for contextual bandits with continuous actions having unknown structure.
Federated Residual Learning
We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model.
Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning
We present an algorithm, HOMER, for exploration and reinforcement learning in rich observation environments that are summarizable by an unknown latent state space.
Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds
We design a new algorithm for batch active learning with deep neural network models.
Empirical Likelihood for Contextual Bandits
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting.
Efficient Forward Architecture Search
We propose a neural architecture search (NAS) algorithm, Petridish, to iteratively add shortcut connections to existing network layers.
Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting
We study contextual bandit learning with an abstract policy class and continuous action space.
Provably efficient RL with Rich Observations via Latent State Decoding
We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states.
Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback
We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data.
Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches
We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy.
Contextual Memory Trees
We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size.
A Reductions Approach to Fair Classification
We present a systematic approach for achieving fairness in a binary classification setting.
On Oracle-Efficient PAC RL with Rich Observations
We study the computational tractability of PAC reinforcement learning with rich observations.
A Contextual Bandit Bake-off
Contextual bandit algorithms are essential for solving many real-world interactive machine learning problems.
Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback
We consider reinforcement learning and bandit structured prediction problems with very sparse loss feedback: only at the end of an episode.
Efficient Contextual Bandits in Non-stationary Worlds
In this work, we develop several efficient contextual bandit algorithms for non-stationary environments by equipping existing methods for i. i. d.
Learning Deep ResNet Blocks Sequentially using Boosting Theory
We prove that the training error decays exponentially with the depth $T$ if the \emph{weak module classifiers} that we train perform slightly better than some weak baseline.
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
We propose to directly map raw visual observations and text input to actions for instruction execution.
Active Learning for Cost-Sensitive Classification
We design an active learning algorithm for cost-sensitive multiclass classification: problems where different errors have different costs.
Contextual Decision Processes with Low Bellman Rank are PAC-Learnable
Our first contribution is a complexity measure, the Bellman rank, that we show enables tractable learning of near-optimal behavior in these processes and is naturally small for many well-studied reinforcement learning settings.
Logarithmic Time One-Against-Some
Compared to previous approaches, we obtain substantially better statistical performance for two reasons: First, we prove a tighter and more complete boosting theorem, and second we translate the results more directly into an algorithm.
Making Contextual Decisions with Low Technical Debt
The Decision Service enables all aspects of contextual bandit learning using four system abstractions which connect together in a loop: explore (the decision space), log, learn, and deploy.
Off-policy evaluation for slate recommendation
This paper studies the evaluation of policies that recommend an ordered set of items (e. g., a ranking) based on some context---a common scenario in web search, ads, and recommendation.
Search Improves Label for Active Learning
We investigate active learning with access to two distinct oracles: Label (which is standard) and Search (which is not).
PAC Reinforcement Learning with Rich Observations
We prove that the algorithm learns near optimal behavior after a number of episodes that is polynomial in all relevant parameters, logarithmic in the number of policies, and independent of the size of the observation space.
Efficient Second Order Online Learning by Sketching
We propose Sketched Online Newton (SON), an online second order learning algorithm that enjoys substantially improved regret guarantees for ill-conditioned data.
Efficient and Parsimonious Agnostic Active Learning
We develop a new active learning algorithm for the streaming setting satisfying three important properties: 1) It provably works for any classifier representation and classification problem including those with severe noise.
Learning to Search for Dependencies
We demonstrate that a dependency parser can be built using a credit assignment compiler which removes the burden of worrying about low-level machine learning details from the parser implementation.
Doubly Robust Policy Evaluation and Optimization
As such, we expect the doubly robust approach to become common practice in policy evaluation and optimization.
Learning Reductions that Really Work
We provide a summary of the mathematical and computational techniques that have enabled learning reductions to effectively address a wide class of problems, and show that this approach to solving machine learning problems can be broadly useful.
Learning to Search Better Than Your Teacher
Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference.
Scalable Non-linear Learning with Adaptive Polynomial Expansions
Can we effectively learn a nonlinear representation in time comparable to linear learning?
Scalable Nonlinear Learning with Adaptive Polynomial Expansions
Can we effectively learn a nonlinear representation in time comparable to linear learning?
Normalized Online Learning
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale.
Conditional Probability Tree Estimation Analysis and Algorithms
We consider the problem of estimating the conditional probability of a label in time O(log n), where n is the number of possible labels.
A Credit Assignment Compiler for Joint Prediction
Many machine learning applications involve jointly predicting multiple mutually dependent output variables.
Logarithmic Time Online Multiclass prediction
We develop top-down tree construction approaches for constructing logarithmic depth trees.
Resourceful Contextual Bandits
We study contextual bandits with ancillary constraints on resources, which are common in real-world applications such as choosing ads or dynamic pricing of items.
Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
We present a new algorithm for the contextual bandit learning problem, where the learner repeatedly takes one of $K$ actions in response to the observed context, and observes the reward only for that chosen action.
Efficient Online Bootstrapping for Large Scale Learning
Bootstrapping is a useful technique for estimating the uncertainty of a predictor, for example, confidence intervals for prediction.
Para-active learning
We leverage the same observation to build a generic strategy for parallelizing learning algorithms.
Normalized Online Learning
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale.
A Reliable Effective Terascale Linear Learning System
We present a system and a set of techniques for learning linear predictors with convex losses on terascale datasets, with trillions of features, {The number of features here refers to the number of non-zero entries in the data matrix.}
Doubly Robust Policy Evaluation and Learning
The key challenge is that the past data typically does not faithfully represent proportions of actions taken by a new policy.
Agnostic Active Learning Without Constraints
We present and analyze an agnostic active learning algorithm that works without keeping a version space.
Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
\emph{Offline} evaluation of the effectiveness of new algorithms in these applications is critical for protecting online user experiences but very challenging due to their "partial-label" nature.
A Contextual-Bandit Approach to Personalized News Article Recommendation
In this work, we model personalized recommendation of news articles as a contextual bandit problem, a principled approach in which a learning algorithm sequentially selects articles to serve users based on contextual information about the users and articles, while simultaneously adapting its article-selection strategy based on user-click feedback to maximize total user clicks.
Learning from Logged Implicit Exploration Data
We provide a sound and consistent foundation for the use of \emph{nonrandom} exploration data in "contextual bandit" or "partially labeled" settings where only the value of a chosen action is learned.
Slow Learners are Fast
Online learning algorithms have impressive convergence properties when it comes to risk minimization and convex games on very large problems.
Search-based Structured Prediction
We present Searn, an algorithm for integrating search and learning to solve complex structured prediction problems such as those that occur in natural language, speech, computational biology, and vision.
Feature Hashing for Large Scale Multitask Learning
Empirical evidence suggests that hashing is an effective strategy for dimensionality reduction and practical nonparametric estimation.
The Offset Tree for Learning with Partial Labels
We show that the Offset Tree is an optimal reduction to binary classification.
Sparse Online Learning via Truncated Gradient
We propose a general method called truncated gradient to induce sparsity in the weights of online-learning algorithms with convex loss.
Predictive Indexing for Fast Search
We tackle the computational problem of query-conditioned search.
The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information
We present Epoch-Greedy, an algorithm for multi-armed bandits with observable side information.
