Search Results for author:
Found 17 papers, 3 papers with code
Stochastic Gradient Succeeds for Bandits
We show that the \emph{stochastic gradient} bandit algorithm converges to a \emph{globally optimal} policy at an $O(1/t)$ rate, even with a \emph{constant} step size.
Beyond Expectations: Learning with Stochastic Dominance Made Practical
Stochastic dominance models risk-averse preferences for decision making with uncertain outcomes, which naturally captures the intrinsic structure of the underlying uncertainty, in contrast to simply resorting to the expectations.
Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Mirror descent value iteration (MDVI), an abstraction of Kullback-Leibler (KL) and entropy-regularized reinforcement learning (RL), has served as the basis for recent high-performing practical RL algorithms.
The Role of Baselines in Policy Gradient Optimization
Instead, the analysis reveals that the primary effect of the value baseline is to \textbf{reduce the aggressiveness of the updates} rather than their variance.
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model.
Understanding the Effect of Stochasticity in Policy Optimization
We study the effect of stochasticity in on-policy policy optimization, and make the following four contributions.
Understanding and Leveraging Overparameterization in Recursive Value Estimation
To better understand the utility of deep models in RL we present an analysis of recursive value estimation using overparameterized linear representations that provides useful, transferable findings.
Leveraging Non-uniformity in First-order Non-convex Optimization
Classical global convergence results for first-order methods rely on uniform smoothness and the \L{}ojasiewicz inequality.
On the Optimality of Batch Policy Optimization Algorithms
First, we introduce a class of confidence-adjusted index algorithms that unifies optimistic and pessimistic principles in a common framework, which enables a general analysis.
Escaping the Gravitational Pull of Softmax
Both findings are based on an analysis of convergence rates using the Non-uniform \L{}ojasiewicz (N\L{}) inequalities.
Beyond Prioritized Replay: Sampling States in Model-Based RL via Simulated Priorities
The prioritized Experience Replay (ER) method has attracted great attention; however, there is little theoretical understanding of such prioritization strategy and why they help.
Understanding and Mitigating the Limitations of Prioritized Experience Replay
Prioritized Experience Replay (ER) has been empirically shown to improve sample efficiency across many domains and attracted great attention; however, there is little theoretical understanding of why such prioritized sampling helps and its limitations.
On the Global Convergence Rates of Softmax Policy Gradient Methods
First, we show that with the true gradient, policy gradient with a softmax parametrization converges at a $O(1/t)$ rate, with constants depending on the problem and initialization.
Frequency-based Search-control in Dyna
This suggests a search-control strategy: we should use states from high frequency regions of the value function to query the model to acquire more samples.
Maximum Entropy Monte-Carlo Planning
We then extend this approach to general sequential decision making by developing a general MCTS algorithm, Maximum Entropy for Tree Search (MENTS).
Identifying and Tracking Sentiments and Topics from Social Media Texts during Natural Disasters
We study the problem of identifying the topics and sentiments and tracking their shifts from social media texts in different geographical regions during emergencies and disasters.
On the Reducibility of Submodular Functions
The scalability of submodular optimization methods is critical for their usability in practice.
