Search Results for author:
Found 44 papers, 5 papers with code
Neural Contextual Bandits with UCB-based Exploration
To the best of our knowledge, it is the first neural network-based contextual bandit algorithm with a near-optimal regret guarantee.
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
Experiments on standard benchmarks show that our proposed algorithm can maintain a fast convergence rate as Adam/Amsgrad while generalizing as well as SGD in training deep neural networks.
Neural Thompson Sampling
Thompson Sampling (TS) is one of the most effective algorithms for solving contextual multi-armed bandit problems.
A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
Depending on how much information an adversary can access to, adversarial attacks can be classified as white-box attack and black-box attack.
Iterative Teacher-Aware Learning
Recently, the benefits of integrating this cooperative pedagogy into machine concept learning in discrete spaces have been proved by multiple works.
Stochastic Variance-Reduced Cubic Regularized Newton Method
At the core of our algorithm is a novel semi-stochastic gradient along with a semi-stochastic Hessian, which are specifically designed for cubic regularization method.
Stochastic Nested Variance Reduction for Nonconvex Optimization
We study finite-sum nonconvex optimization problems, where the objective function is an average of $n$ nonconvex functions.
Finding Local Minima via Stochastic Nested Variance Reduction
For general stochastic optimization problems, the proposed $\text{SNVRG}^{+}+\text{Neon2}^{\text{online}}$ achieves $\tilde{O}(\epsilon^{-3}+\epsilon_H^{-5}+\epsilon^{-2}\epsilon_H^{-3})$ gradient complexity, which is better than both $\text{SVRG}+\text{Neon2}^{\text{online}}$ (Allen-Zhu and Li, 2017) and Natasha2 (Allen-Zhu, 2017) in certain regimes.
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
In this paper, we provide a fine-grained convergence analysis for a general class of adaptive gradient methods including AMSGrad, RMSProp and AdaGrad.
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
In particular, we study the binary classification problem and show that for a broad family of loss functions, with proper random weight initialization, both gradient descent and stochastic gradient descent can find the global minima of the training loss for an over-parameterized deep ReLU network, under mild assumption on the training data.
Sample Efficient Stochastic Variance-Reduced Cubic Regularization Method
The proposed algorithm achieves a lower sample complexity of Hessian matrix computation than existing cubic regularization based methods.
Stochastic Nested Variance Reduced Gradient Descent for Nonconvex Optimization
We study finite-sum nonconvex optimization problems, where the objective function is an average of $n$ nonconvex functions.
Stochastic Recursive Variance-Reduced Cubic Regularization Methods
Built upon SRVRC, we further propose a Hessian-free SRVRC algorithm, namely SRVRC$_{\text{free}}$, which only requires stochastic gradient and Hessian-vector product computations, and achieves $\tilde O(dn\epsilon^{-2} \land d\epsilon^{-3})$ runtime complexity, where $n$ is the number of component functions in the finite-sum structure, $d$ is the problem dimension, and $\epsilon$ is the optimization precision.
Lower Bounds for Smooth Nonconvex Finite-Sum Optimization
We prove tight lower bounds for the complexity of finding $\epsilon$-suboptimal point and $\epsilon$-approximate stationary point in different settings, for a wide regime of the smallest eigenvalue of the Hessian of the objective function (or each component function).
Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mapping
We propose a novel algorithm that makes use of the feature mapping and obtains a $\tilde O(d\sqrt{T}/(1-\gamma)^2)$ regret, where $d$ is the dimension of the feature space, $T$ is the time horizon and $\gamma$ is the discount factor of the MDP.
Nearly Minimax Optimal Reinforcement Learning for Discounted MDPs
We study the reinforcement learning problem for discounted Markov Decision Processes (MDPs) under the tabular setting.
Provable Multi-Objective Reinforcement Learning with Generative Models
Multi-objective reinforcement learning (MORL) is an extension of ordinary, single-objective reinforcement learning (RL) that is applicable to many real-world tasks where multiple objectives exist without known relative costs.
Logarithmic Regret for Reinforcement Learning with Linear Function Approximation
Reinforcement learning (RL) with linear function approximation has received increasing attention recently.
Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processes
Based on the new inequality, we propose a new, computationally efficient algorithm with linear function approximation named $\text{UCRL-VTR}^{+}$ for the aforementioned linear mixture MDPs in the episodic undiscounted setting.
Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints
In specific, for the batch learning model, our proposed LSVI-UCB-Batch algorithm achieves an $\tilde O(\sqrt{d^3H^3T} + dHT/B)$ regret, where $d$ is the dimension of the feature mapping, $H$ is the episode length, $T$ is the number of interactions and $B$ is the number of batches.
Nearly Minimax Optimal Regret for Learning Infinite-horizon Average-reward MDPs with Linear Function Approximation
We study reinforcement learning in an infinite-horizon average-reward setting with linear function approximation, where the transition probability function of the underlying Markov Decision Process (MDP) admits a linear form over a feature mapping of the current state, action, and next state.
Almost Optimal Algorithms for Two-player Zero-Sum Linear Mixture Markov Games
To assess the optimality of our algorithm, we also prove an $\tilde{\Omega}( dH\sqrt{T})$ lower bound on the regret.
Near-optimal Policy Optimization Algorithms for Learning Adversarial Linear Mixture MDPs
In this paper, we study RL in episodic MDPs with adversarial reward and full information feedback, where the unknown transition probability function is a linear function of a given feature mapping, and the reward function can change arbitrarily episode by episode.
Batched Neural Bandits
In many sequential decision-making problems, the individuals are split into several batches and the decision-maker is only allowed to change her policy at the end of batches.
Variance-Aware Off-Policy Evaluation with Linear Function Approximation
We study the off-policy evaluation (OPE) problem in reinforcement learning with linear function approximation, which aims to estimate the value function of a target policy based on the offline data collected by a behavior policy.
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
For the offline counterpart, ReLEX-LCB, we show that the algorithm can find the optimal policy if the representation class can cover the state-action space and achieves gap-dependent sample complexity.
Uniform-PAC Bounds for Reinforcement Learning with Linear Function Approximation
The uniform-PAC guarantee is the strongest possible guarantee for reinforcement learning in the literature, which can directly imply both PAC and high probability regret bounds, making our algorithm superior to all existing algorithms with linear function approximation.
Pure Exploration in Kernel and Neural Bandits
To overcome the curse of dimensionality, we propose to adaptively embed the feature representation of each arm into a lower-dimensional space and carefully deal with the induced model misspecification.
Reward-Free Model-Based Reinforcement Learning with Linear Function Approximation
By constructing a special class of linear Mixture MDPs, we also prove that for any reward-free algorithm, it needs to sample at least $\tilde \Omega(H^2d\epsilon^{-2})$ episodes to obtain an $\epsilon$-optimal policy.
Model-based Reinforcement Learning
reinforcement-learning
+1
Linear Contextual Bandits with Adversarial Corruptions
We study the linear contextual bandit problem in the presence of adversarial corruption, where the interaction between the player and a possibly infinite decision set is contaminated by an adversary that can corrupt the reward up to a corruption level $C$ measured by the sum of the largest alteration on rewards in each round.
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima
In this paper, we propose LENA (Last stEp shriNkAge), a faster perturbed stochastic gradient framework for finding local minima.
Training Deep Neural Networks with Partially Adaptive Momentum
Experiments on standard benchmarks show that our proposed algorithm can maintain fast convergence rate as Adam/Amsgrad while generalizing as well as SGD in training deep neural networks.
NeuralUCB: Contextual Bandits with Neural Network-Based Exploration
To the best of our knowledge, our algorithm is the first neural network-based contextual bandit algorithm with near-optimal regret guarantee.
Learning Neural Contextual Bandits Through Perturbed Rewards
Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts.
Optimal Online Generalized Linear Regression with Stochastic Noise and Its Application to Heteroscedastic Bandits
We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additive noise.
Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions
We show that for both known $C$ and unknown $C$ cases, our algorithm with proper choice of hyperparameter achieves a regret that nearly matches the lower bounds.
Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPs
When applying our weighted least square estimator to heterogeneous linear bandits, we can obtain an $\tilde O(d\sqrt{\sum_{k=1}^K \sigma_k^2} +d)$ regret in the first $K$ rounds, where $d$ is the dimension of the context and $\sigma_k^2$ is the variance of the reward in the $k$-th round.
Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium
We consider learning Nash equilibria in two-player zero-sum Markov Games with nonlinear function approximation, where the action-value function is approximated by a function in a Reproducing Kernel Hilbert Space (RKHS).
Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes
We study reinforcement learning (RL) with linear function approximation.
Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency
We propose a variance-adaptive algorithm for linear mixture MDPs, which achieves a problem-dependent horizon-free regret bound that can gracefully reduce to a nearly constant regret for deterministic MDPs.
Risk Bounds of Accelerated SGD for Overparameterized Linear Regression
Additionally, when our analysis is specialized to linear regression in the strongly convex setting, it yields a tighter bound for bias error than the best-known result.
Nearly Minimax Optimal Regret for Learning Linear Mixture Stochastic Shortest Path
Our algorithm achieves an $\tilde{\mathcal O}(dB_*\sqrt{K})$ regret bound, where $d$ is the dimension of the feature mapping in the linear transition kernel, $B_*$ is the upper bound of the total cumulative cost for the optimal policy, and $K$ is the number of episodes.
DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training
DPAdapter modifies and enhances the sharpness-aware minimization (SAM) technique, utilizing a two-batch strategy to provide a more accurate perturbation estimate and an efficient gradient descent, thereby improving parameter robustness against noise.
Variance-Dependent Regret Bounds for Non-stationary Linear Bandits
We investigate the non-stationary stochastic linear bandit problem where the reward distribution evolves each round.
