Search Results for author:
Found 106 papers, 16 papers with code
Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks
Multitask Reinforcement Learning (MTRL) approaches have gained increasing attention for its wide applications in many important Reinforcement Learning (RL) tasks.
Optimal Thresholding Linear Bandit
We study a novel pure exploration problem: the $\epsilon$-Thresholding Bandit Problem (TBP) with fixed confidence in stochastic linear bandits.
The Complexity of Sequential Prediction in Dynamical Systems
We study the problem of learning to predict the next state of a dynamical system when the underlying evolution function is unknown.
A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Low-Rank MDPs
Our algorithm is the first computationally efficient algorithm in this setting that achieves sample complexity of $O(\epsilon^{-2})$ with partial data coverage assumption.
A Framework for Partially Observed Reward-States in RLHF
We show reductions from the the two dominant forms of human feedback in RLHF - cardinal and dueling feedback to PORRL.
Apple Tasting: Combinatorial Dimensions and Minimax Rates
In particular, we show that in the realizable setting, the expected number of mistakes of any learner, under apple tasting feedback, can be $\Theta(1), \Theta(\sqrt{T})$, or $\Theta(T)$.
Sequence Length Independent Norm-Based Generalization Bounds for Transformers
This paper provides norm-based generalization bounds for the Transformer architecture that do not depend on the input sequence length.
Conformal Contextual Robust Optimization
Data-driven approaches to predict-then-optimize decision-making problems seek to mitigate the risk of uncertainty region misspecification in safety-critical settings.
On the Computational Complexity of Private High-dimensional Model Selection
We consider the problem of model selection in a high-dimensional sparse linear regression model under privacy constraints.
Online Infinite-Dimensional Regression: Learning Linear Operators
Finally, we prove that the impossibility result and the separation between uniform convergence and learnability also hold in the batch setting.
On the Minimax Regret in Online Ranking with Top-k Feedback
In online ranking, a learning algorithm sequentially ranks a set of items and receives feedback on its ranking in the form of relevance scores.
Multiclass Online Learnability under Bandit Feedback
We study online multiclass classification under bandit feedback.
A Combinatorial Characterization of Supervised Online Learnability
We study the online learnability of hypothesis classes with respect to arbitrary, but bounded loss functions.
A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learning
Offline constrained reinforcement learning (RL) aims to learn a policy that maximizes the expected cumulative reward subject to constraints on expected cumulative cost using an existing dataset.
Online Learning with Set-Valued Feedback
We study a variant of online multiclass classification where the learner predicts a single label but receives a \textit{set of labels} as feedback.
Amortized Variational Inference with Coverage Guarantees
We prove lower bounds on the predictive efficiency of the regions produced by CANVI and explore how the quality of a posterior approximation relates to the predictive efficiency of prediction regions based on that approximation.
Multiclass Online Learning and Uniform Convergence
We argue that the best expert has regret at most Littlestone dimension relative to the best concept in the class.
Quantum Learning Theory Beyond Batch Binary Classification
Arunachalam and de Wolf (2018) showed that the sample complexity of quantum batch learning of boolean functions, in the realizable and agnostic settings, has the same form and order as the corresponding classical sample complexities.
An Asymptotically Optimal Algorithm for the Convex Hull Membership Problem
This work studies the pure-exploration setting for the convex hull membership (CHM) problem where one aims to efficiently and accurately determine if a given point lies in the convex hull of means of a finite set of distributions.
Understanding Best Subset Selection: A Tale of Two C(omplex)ities
Furthermore, we show that a margin condition depending on similar margin quantity and complexity measures is also necessary for model consistency of BSS.
A Characterization of Multioutput Learnability
This provides a complete characterization of the learnability of multilabel classification and multioutput regression in both batch and online settings.
Offline Policy Evaluation and Optimization under Confounding
Evaluating and optimizing policies in the presence of unobserved confounders is a problem of growing interest in offline reinforcement learning.
RL Boltzmann Generators for Conformer Generation in Data-Sparse Environments
The generation of conformers has been a long-standing interest to structural chemists and biologists alike.
Learning Mixtures of Markov Chains and MDPs
We present an algorithm for learning mixtures of Markov chains and Markov decision processes (MDPs) from short unlabeled trajectories.
Thompson Sampling for High-Dimensional Sparse Linear Contextual Bandits
We consider the stochastic linear contextual bandit problem with high-dimensional features.
Online Agnostic Multiclass Boosting
In this way, boosting algorithms convert weak learners into strong ones.
Adaptive Sampling for Discovery
Starting with a large unlabeled dataset, algorithms for ASD adaptively label the points with the goal to maximize the sum of responses.
An Optimization-based Algorithm for Non-stationary Kernel Bandits without Prior Knowledge
Moreover, when applied to the non-stationary linear bandit setting by using a linear kernel, our algorithm is nearly minimax optimal, solving an open problem in the non-stationary linear bandit literature.
Achieving Representative Data via Convex Hull Feasibility Sampling Algorithms
Sampling biases in training data are a major source of algorithmic biases in machine learning systems.
Joint Learning of Linear Time-Invariant Dynamical Systems
Linear time-invariant systems are very popular models in system theory and applications.
Balancing Adaptability and Non-exploitability in Repeated Games
We study the problem of guaranteeing low regret in repeated games against an opponent with unknown membership in one of several classes.
On the Statistical Benefits of Curriculum Learning
For both settings, we derive the minimax rates for CL with the oracle that provides the optimal curriculum and without the oracle, where the agent has to adaptively learn a good curriculum.
Learning in Online MDPs: Is there a Price for Handling the Communicating Case?
In contrast, it has been shown that handling online MDPs with communicating structure and bandit information incurs $\Omega(T^{2/3})$ regret even in the case of deterministic transitions.
Bandit Algorithms for Precision Medicine
However, the modern precision medicine movement has been enabled by a confluence of events: scientific advances in fields such as genetics and pharmacology, technological advances in mobile devices and wearable sensors, and methodological advances in computing and data sciences.
Weighted Gaussian Process Bandits for Non-stationary Environments
To this end, we develop WGP-UCB, a novel UCB-type algorithm based on weighted Gaussian process regression.
Causal Bandits with Unknown Graph Structure
In causal bandit problems, the action set consists of interventions on variables of a causal graph.
Representation Learning Beyond Linear Prediction Functions
This motivates us to ask whether diversity can be achieved when source tasks and the target task use different prediction function spaces beyond linear functions.
Causal Markov Decision Processes: Learning Good Interventions Efficiently
We introduce causal Markov Decision Processes (C-MDPs), a new formalism for sequential decision making which combines the standard MDP formulation with causal structures over state transition and reward functions.
Decision Making Problems with Funnel Structure: A Multi-Task Learning Approach with Application to Email Marketing Campaigns
We analyze both the prediction error and the regret of our algorithms.
Federated Learning via Synthetic Data
Federated learning allows for the training of a model using data on multiple clients without the clients transmitting that raw data.
TorsionNet: A Reinforcement Learning Approach to Sequential Conformer Search
Molecular geometry prediction of flexible molecules, or conformer search, is a long-standing challenge in computational chemistry.
Low-Rank Generalized Linear Bandit Problems
To get around the computational intractability of covering based approaches, we propose an efficient algorithm by extending the "Explore-Subspace-Then-Refine" algorithm of~\citet{jun2019bilinear}.
On the Equivalence between Online and Private Learnability beyond Binary Classification
First, we show that private learnability implies online learnability in both settings.
On Learnability under General Stochastic Processes
Statistical learning theory under independent and identically distributed (iid) sampling and online learning theory for worst case individual sequences are two of the best developed branches of learning theory.
Reinforcement Learning in Factored MDPs: Oracle-Efficient Algorithms and Tighter Regret Bounds for the Non-Episodic Setting
We study reinforcement learning in non-episodic factored Markov decision processes (FMDPs).
Randomized Exploration for Non-Stationary Stochastic Linear Bandits
We investigate two perturbation approaches to overcome conservatism that optimism based algorithms chronically suffer from in practice.
Online Boosting for Multilabel Ranking with Top-k Feedback
We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides.
Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
As an extension, we also consider the more challenging problem of model selection, where the state features are unknown and can be chosen from a large candidate set.
Thompson Sampling in Non-Episodic Restless Bandits
Restless bandit problems assume time-varying reward distributions of the arms, which adds flexibility to the model but makes the analysis more challenging.
What You See May Not Be What You Get: UCB Bandit Algorithms Robust to ε-Contamination
We define the $\varepsilon$-contaminated stochastic bandit problem and use our robust mean estimators to give two variants of a robust Upper Confidence Bound (UCB) algorithm, crUCB.
Regret Analysis of Bandit Problems with Causal Background Knowledge
For example, we observe that even with a few hundreds of iterations, the regret of causal algorithms is less than that of standard algorithms by a factor of three.
Not All are Made Equal: Consistency of Weighted Averaging Estimators Under Active Learning
We generalize Stone's Theorem in the noise free setting, proving consistency for well known classifiers such as $k$-NN, histogram and kernel estimators under conditions which mirror classical results.
Regret Bounds for Thompson Sampling in Episodic Restless Bandit Problems
These problems have been studied well from the optimization perspective, where the goal is to efficiently find a near-optimal policy when system parameters are known.
Generalization Bounds in the Predict-then-Optimize Framework
A natural loss function in this environment is to consider the cost of the decisions induced by the predicted parameters, in contrast to the prediction error of the parameters.
Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systems
We provide numerical analyses for the performance of two methods: stochastic feedback, and stochastic parameter.
No-regret Exploration in Contextual Reinforcement Learning
We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs.
On Applications of Bootstrap in Continuous Space Reinforcement Learning
In decision making problems for continuous state and action spaces, linear dynamical models are widely employed.
On the Optimality of Perturbations in Stochastic and Adversarial Multi-armed Bandit Problems
We investigate the optimality of perturbation based algorithms in the stochastic and adversarial multi-armed bandit problems.
Active Learning for Non-Parametric Regression Using Purely Random Trees
Active learning is the task of using labelled data to select additional points to label, with the goal of fitting the most accurate model with a fixed budget of labelled points.
Input Perturbations for Adaptive Control and Learning
This paper studies adaptive algorithms for simultaneous regulation (i. e., control) and estimation (i. e., learning) of Multiple Input Multiple Output (MIMO) linear dynamical systems.
Online Multiclass Boosting with Bandit Feedback
We propose an unbiased estimate of the loss using a randomized prediction, allowing the model to update its weak learners with limited information.
Fighting Contextual Bandits with Stochastic Smoothing
We propose a general algorithm template that represents random perturbation based algorithms and identify several perturbation distributions that lead to strong regret bounds.
On the Approximation Properties of Random ReLU Features
We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS).
But How Does It Work in Theory? Linear SVM with Random Features
We prove that, under low noise assumptions, the support vector machine with $N\ll m$ random features (RFSVM) can achieve the learning rate faster than $O(1/\sqrt{m})$ on a training set with $m$ samples when an optimized feature map is used.
Finite Time Adaptive Stabilization of LQ Systems
There are only a few existing non-asymptotic results and a full treatment of the problem is not currently available.
Online Learning via the Differential Privacy Lens
In this paper, we use differential privacy as a lens to examine online learning in both full and partial information settings.
Optimism-Based Adaptive Regulation of Linear-Quadratic Systems
The main challenge for adaptive regulation of linear-quadratic systems is the trade-off between identification and control.
Markov Decision Processes with Continuous Side Information
Because our lower bound has an exponential dependence on the dimension, we consider a tractable linear setting where the context is used to create linear combinations of a finite set of MDPs.
Online Boosting Algorithms for Multi-label Ranking
We consider the multi-label ranking approach to multi-label learning.
An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions
Increasing technological sophistication and widespread use of smartphones and wearable devices provide opportunities for innovative and highly personalized health interventions.
Online Multiclass Boosting
Recent work has extended the theoretical analysis of boosting algorithms to multiclass problems and to online settings.
Beyond the Hazard Rate: More Perturbation Algorithms for Adversarial Multi-armed Bandits
Assuming that the hazard rate is bounded, it is possible to provide regret analyses for a variety of FTPL algorithms for the multi-armed bandit problem.
Sampled Fictitious Play is Hannan Consistent
Fictitious play is a simple and widely studied adaptive heuristic for playing repeated games.
Phased Exploration with Greedy Exploitation in Stochastic Combinatorial Partial Monitoring Games
The implementation of their algorithm depends on two separate offline oracles and the distribution dependent regret additionally requires existence of a unique optimal action for the learner.
Online Learning to Rank with Top-k Feedback
We consider two settings of online learning to rank where feedback is restricted to top ranked items.
Mixture Proportion Estimation via Kernel Embedding of Distributions
Mixture proportion estimation (MPE) is the problem of estimating the weight of a component distribution in a mixture, given samples from the mixture and component.
Online Learning to Rank with Feedback at the Top
We consider an online learning to rank setting in which, at each round, an oblivious adversary generates a list of $m$ documents, pertaining to a query, and the learner produces scores to rank the documents.
Generalization error bounds for learning to rank: Does the length of document lists matter?
We consider the generalization ability of algorithms for learning to rank at a query level, a problem also called subset ranking.
Lasso Guarantees for Time Series Estimation Under Subgaussian Tails and $ β$-Mixing
Many theoretical results on estimation of high dimensional time series require specifying an underlying data generating model (DGM).
Fighting Bandits with a New Kind of Smoothness
We define a novel family of algorithms for the adversarial multi-armed bandit problem, and provide a simple analysis technique based on convex smoothing.
Alternating Minimization for Regression Problems with Vector-valued Outputs
In regression problems involving vector-valued outputs (or equivalently, multiple responses), it is well known that the maximum likelihood estimator (MLE), which takes noise covariance structure into account, can be significantly more accurate than the ordinary least squares (OLS) estimator.
Predtron: A Family of Online Algorithms for General Prediction Problems
We offer a general framework to derive mistake driven online algorithms and associated loss bounds.
Handling Class Imbalance in Link Prediction using Learning to Rank Techniques
We consider the link prediction problem in a partially observed network, where the objective is to make predictions in the unobserved portion of the network.
Perceptron like Algorithms for Online Learning to Rank
We show that, if there exists a perfect oracle ranker which can correctly rank each instance in an online sequence of ranking data, with some margin, the cumulative loss of perceptron algorithm on that sequence is bounded by a constant, irrespective of the length of the sequence.
Spectral Smoothing via Random Matrix Perturbations
Smoothing the maximum eigenvalue function is important for applications in semidefinite optimization and online learning.
Consistent Algorithms for Multiclass Classification with a Reject Option
We consider the problem of $n$-class classification ($n\geq 2$), where the classifier can choose to abstain from making predictions at a given cost, say, a factor $\alpha$ of the cost of misclassification.
On Iterative Hard Thresholding Methods for High-dimensional M-Estimation
Our results rely on a general analysis framework that enables us to analyze several popular hard thresholding style algorithms (such as HTP, CoSaMP, SP) in the high dimensional regression setting.
Online Ranking with Top-1 Feedback
We consider a setting where a system learns to rank a fixed set of $m$ items.
Online Linear Optimization via Smoothing
We present a new optimization-theoretic approach to analyzing Follow-the-Leader style algorithms, particularly in the setting where perturbations are used as a tool for regularization.
Perceptron-like Algorithms and Generalization Bounds for Learning to Rank
En route to developing the online algorithm and generalization bound, we propose a novel family of listwise large margin ranking surrogates.
On Lipschitz Continuity and Smoothness of Loss Functions in Learning to Rank
In binary classification and regression problems, it is well understood that Lipschitz continuity and smoothness of the loss function play key roles in governing generalization error bounds for empirical risk minimization algorithms.
Convex Calibrated Surrogates for Low-Rank Loss Matrices with Applications to Subset Ranking Losses
The design of convex, calibrated surrogate losses, whose minimization entails consistency with respect to a desired target loss, is an important concept to have emerged in the theory of machine learning in recent years.
Learning with Noisy Labels
In this paper, we theoretically study the problem of binary classification in the presence of random classification noise --- the learner, instead of seeing the true labels, sees labels that have independently been flipped with some small probability.
Feature Clustering for Accelerating Parallel Coordinate Descent
We give a unified convergence analysis for the family of block-greedy algorithms.
Online Learning: Stochastic, Constrained, and Smoothed Adversaries
We define the minimax value of a game where the adversary is restricted in his moves, capturing stochastic and non-stochastic assumptions on data.
Greedy Algorithms for Structurally Constrained High Dimensional Problems
A hallmark of modern machine learning is its ability to deal with high dimensional problems by exploiting structural assumptions that limit the degrees of freedom in the underlying model.
Nearest Neighbor based Greedy Coordinate Descent
In particular, we investigate the greedy coordinate descent algorithm, and note that performing the greedy step efficiently weakens the costly dependence on the problem size provided the solution is sparse.
On the Universality of Online Mirror Descent
We show that for a general class of convex online learning problems, Mirror Descent can always achieve a (nearly) optimal regret guarantee.
Orthogonal Matching Pursuit with Replacement
Our proof techniques are novel and flexible enough to also permit the tightest known analysis of popular iterative algorithms such as CoSaMP and Subspace Pursuit.
Online Learning: Random Averages, Combinatorial Parameters, and Learnability
We develop a theory of online learning by defining several complexity measures.
Smoothness, Low Noise and Fast Rates
We establish an excess risk bound of O(H R_n^2 + sqrt{H L*} R_n) for ERM with an H-smooth loss function and a hypothesis class with Rademacher complexity R_n, where L* is the best risk achievable by the hypothesis class.
Online Learning via Sequential Complexities
We consider the problem of sequential prediction and provide tools to study the minimax value of the associated game.
Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model.
On the Generalization Ability of Online Strongly Convex Programming Algorithms
This paper examines the generalization properties of online convex programming algorithms when the loss function is Lipschitz and strongly convex.
On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization
We provide sharp bounds for Rademacher and Gaussian complexities of (constrained) linear classes.
Optimistic Linear Programming gives Logarithmic Regret for Irreducible MDPs
OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities and the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm.
