Search Results for author:
Found 52 papers, 5 papers with code
Decentralized and Uncoordinated Learning of Stable Matchings: A Game-Theoretic Approach
Our proposed game-theoretic framework bridges the discrete problem of learning stable matchings with the problem of learning NE in continuous-action games.
Performance of NPG in Countable State-Space Average-Cost RL
We consider policy optimization methods in reinforcement learning settings where the state space is arbitrarily large, or even countably infinite.
On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
We present the first finite time global convergence analysis of policy gradient in the context of infinite horizon average reward Markov decision processes (MDPs).
Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
The algorithm is based on the popular Policy Cover-Policy Gradient (PC-PG) algorithm, which assumes knowledge of the reward function.
Convergence for Natural Policy Gradient on Infinite-State Average-Reward Markov Decision Processes
In the reinforcement learning (RL) context, a variety of algorithms have been developed to learn and optimize these MDPs.
Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning
We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains.
Cascading Reinforcement Learning
In the cascading bandit model, at each timestep, an agent recommends an ordered subset of items (called an item list) from a pool of items, each associated with an unknown attraction probability.
Striking a Balance: An Optimal Mechanism Design for Heterogenous Differentially Private Data Acquisition for Logistic Regression
We investigate the problem of performing logistic regression on data collected from privacy-sensitive sellers.
Collaborative Multi-Agent Heterogeneous Multi-Armed Bandits
The study of collaborative multi-agent bandits has attracted significant attention recently.
A New Policy Iteration Algorithm For Reinforcement Learning in Zero-Sum Markov Games
We further show that lookahead can be implemented efficiently in the function approximation setting of linear Markov games, which are the counterpart of the much-studied linear MDPs.
Model-based Reinforcement Learning
Multi-agent Reinforcement Learning
+2
Spectral Clustering for Crowdsourcing with Inherently Distinct Task Types
The Dawid-Skene model is the most widely assumed model in the analysis of crowdsourcing algorithms that estimate ground-truth labels from noisy worker responses.
On the Convergence of Modified Policy Iteration in Risk Sensitive Exponential Cost Markov Decision Processes
Since the exponential cost formulation deals with the multiplicative Bellman equation, our main contribution is a convergence proof which is quite different than existing results for discounted and risk-neutral average-cost problems as well as risk sensitive value and policy iteration approaches.
On The Convergence Of Policy Iteration-Based Reinforcement Learning With Monte Carlo Policy Evaluation
A common technique in reinforcement learning is to evaluate the value function from Monte Carlo simulations of a given policy, and use the estimated value function to obtain a new policy which is greedy with respect to the estimated value function.
Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation
We provide performance guarantees for a variant of simulation-based policy iteration for controlling Markov decision processes that involves the use of stochastic approximation algorithms along with state-of-the-art techniques that are useful for very large MDPs, including lookahead, function approximation, and gradient descent.
Learning While Scheduling in Multi-Server Systems with Unknown Statistics: MaxWeight with Discounted UCB
We prove that under our algorithm the asymptotic average queue length is bounded by one divided by the traffic slackness, which is order-wise optimal.
Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm
Natural actor-critic (NAC) and its variants, equipped with the representation power of neural networks, have demonstrated impressive empirical success in solving Markov decision problems with large state spaces.
Minimax Regret for Cascading Bandits
Cascading bandits is a natural and popular model that frames the task of learning to rank from Bernoulli click feedback in a bandit setting.
Robust Multi-Agent Bandits Over Undirected Graphs
Thus, we generalize existing regret bounds beyond the complete graph (where $d_{\text{mal}}(i) = m$), and show the effect of malicious agents is entirely local (in the sense that only the $d_{\text{mal}}(i)$ malicious agents directly connected to $i$ affect its long-term regret).
Finite-Time Analysis of Natural Actor-Critic for POMDPs
We consider the reinforcement learning problem for partially observed Markov decision processes (POMDPs) with large or even countably infinite state spaces, where the controller has access to only noisy observations of the underlying controlled Markov chain.
A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost MDP
We study the risk-sensitive exponential cost MDP formulation and develop a trajectory-based gradient algorithm to find the stationary point of the cost associated with a set of parameterized policies.
The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation
Therefore, techniques such as lookahead for policy improvement and m-step rollout for policy evaluation are used in practice to improve the performance of approximate dynamic programming with function approximation.
Improved Algorithms for Misspecified Linear Markov Decision Processes
(P1) Its regret after $K$ episodes scales as $K \max \{ \varepsilon_{\text{mis}}, \varepsilon_{\text{tol}} \}$, where $\varepsilon_{\text{mis}}$ is the degree of misspecification and $\varepsilon_{\text{tol}}$ is a user-specified error tolerance.
Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
Furthermore, under mild regularity conditions on the concentrability coefficient and basis vectors, we prove that entropy-regularized NPG exhibits \emph{linear convergence} up to a function approximation error.
Regret Bounds for Stochastic Shortest Path Problems with Linear Function Approximation
We propose an algorithm that uses linear function approximation (LFA) for stochastic shortest path (SSP).
Achieving Small Test Error in Mildly Overparameterized Neural Networks
Recent theoretical works on over-parameterized neural nets have focused on two aspects: optimization and generalization.
Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation
In this paper, we study the dynamics of temporal difference learning with neural network-based value function approximation over a general state space, namely, \emph{Neural TD learning}.
Optimistic Policy Iteration for MDPs with Acyclic Transient State Structure
We consider Markov Decision Processes (MDPs) in which every stationary policy induces the same graph structure for the underlying Markov chain and further, the graph has the following property: if we replace each recurrent class by a node, then the resulting graph is acyclic.
One-bit feedback is sufficient for upper confidence bound policies
We consider a variant of the traditional multi-armed bandit problem in which each arm is only able to provide one-bit feedback during each pull based on its past history of rewards.
Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging
We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp.
On the Consistency of Maximum Likelihood Estimators for Causal Network Identification
Incoming edges to a node in the graph indicate that the state of the node at a particular time instant is influenced by the states of the corresponding parental nodes in the previous time instant.
Adaptive KL-UCB based Bandit Algorithms for Markovian and i.i.d. Settings
rewards are a special case of Markov rewards and it is difficult to design an algorithm that works well independent of whether the underlying model is truly Markovian or i. i. d.
The Mean-Squared Error of Double Q-Learning
In this paper, we establish a theoretical comparison between the asymptotic mean-squared error of Double Q-learning and Q-learning.
Robust Multi-Agent Multi-Armed Bandits
Recent works have shown that agents facing independent instances of a stochastic $K$-armed bandit can collaborate to decrease regret.
The Global Landscape of Neural Networks: An Overview
Second, we discuss a few rigorous results on the geometric properties of wide networks such as "no bad basin", and some modifications that eliminate sub-optimal local minima and/or decreasing paths to infinity.
Continuous-Time Multi-Armed Bandits with Controlled Restarts
Time-constrained decision processes have been ubiquitous in many fundamental applications in physics, biology and computer science.
Budget-Constrained Bandits over General Cost and Reward Distributions
We prove a regret lower bound for this problem, and show that the proposed algorithms achieve tight problem-dependent regret bounds, which are optimal up to a universal constant factor in the case of jointly Gaussian cost and reward pairs.
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity
More specifically, for a large class of over-parameterized deep neural networks with appropriate regularizers, the loss function has no bad local minima and no decreasing paths to infinity.
Finite-Time Performance Bounds and Adaptive Learning Rate Selection for Two Time-Scale Reinforcement Learning
We study two time-scale linear stochastic approximation algorithms, which can be used to model well-known reinforcement learning algorithms such as GTD, GTD2, and TDC.
Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learning
We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i. e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE).
Almost Boltzmann Exploration
In this paper, we show that a simple modification to Boltzmann exploration, motivated by a variation of the standard doubling trick, achieves $O(K\log^{1+\alpha} T)$ regret for a stochastic MAB problem with $K$ arms, where $\alpha>0$ is a parameter of the algorithm.
Adding One Neuron Can Eliminate All Bad Local Minima
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima.
Learning Latent Events from Network Message Logs
One of the main contributions of the paper is a novel mapping of our problem which transforms it into a problem of topic discovery in documents.
Understanding the Loss Surface of Neural Networks for Binary Classification
Here we focus on the training performance of single-layered neural networks for binary classification, and provide conditions under which the training error is zero at all local minima of a smooth hinge loss function.
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
We show in a series of experiments that ODIN is compatible with diverse network architectures and datasets.
Mixing Times and Structural Inference for Bernoulli Autoregressive Processes
For a network with $p$ nodes, where each node has in-degree at most $d$ and corresponds to a scalar Bernoulli process generated by a BAR, we provide a greedy algorithm that can efficiently learn the structure of the underlying directed graph with a sample complexity proportional to the mixing time of the BAR process.
Why Deep Neural Networks for Function Approximation?
We show that, for a large class of piecewise smooth functions, the number of neurons needed by a shallow network to approximate a function is exponentially larger than the corresponding number of neurons needed by a deep network for a given degree of function approximation.
On Projected Stochastic Gradient Descent Algorithm with Weighted Averaging for Least Squares Regression
The problem of least squares regression of a $d$-dimensional unknown parameter is considered.
Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits
To the best of our knowledge, this is the first work that shows how to achieve logarithmic regret in constrained contextual bandits.
Clustering and Inference From Pairwise Comparisons
We propose an efficient algorithm that accurately estimates the individual preferences for almost all users, if there are $r \max \{m, n\}\log m \log^2 n$ pairwise comparisons per type, which is near optimal in sample complexity when $r$ only grows logarithmically with $m$ or $n$.
Collaborative Filtering with Information-Rich and Information-Sparse Entities
In particular, we consider both the clustering model, where only users (or items) are clustered, and the co-clustering model, where both users and items are clustered, and further, we assume that some users rate many items (information-rich users) and some users rate only a few items (information-sparse users).
Jointly Clustering Rows and Columns of Binary Matrices: Algorithms and Trade-offs
In standard clustering problems, data points are represented by vectors, and by stacking them together, one forms a data matrix with row or column cluster structure.
Learning Loosely Connected Markov Random Fields
We consider the structure learning problem for graphical models that we call loosely connected Markov random fields, in which the number of short paths between any pair of nodes is small, and present a new conditional independence test based algorithm for learning the underlying graph structure.
