Search Results for author:
Found 31 papers, 6 papers with code
A Tutorial on Thompson Sampling
Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immediate performance and investing to accumulate new information that may improve future performance.
Empirical Standards for Software Engineering Research
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e. g. a questionnaire survey).
Software Engineering General Literature
Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay
In this context, we study a content exploration task, which we formalize as a multi-armed bandit problem with delayed rewards.
Benchmarking the Generation of Fact Checking Explanations
Results show that in justification production summarization benefits from the claim information, and, in particular, that a claim-driven extractive step improves abstractive summarization performances.
Countering Misinformation via Emotional Response Generation
The proliferation of misinformation on social media platforms (SMPs) poses a significant danger to public health, social cohesion and ultimately democracy.
Simple Bayesian Algorithms for Best Arm Identification
This paper proposes three simple and intuitive Bayesian algorithms for adaptively allocating measurement effort, and formalizes a sense in which these seemingly naive rules are the best possible.
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation
Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process.
Deep Exploration via Randomized Value Functions
We study the use of randomized value functions to guide deep exploration in reinforcement learning.
Satisficing in Time-Sensitive Bandit Learning
Much of the recent literature on bandit learning focuses on algorithms that aim to converge on an optimal action.
Learning to Optimize via Information-Directed Sampling
We propose information-directed sampling -- a new approach to online optimization problems in which a decision-maker must balance between exploration and exploitation while learning from partial feedback.
Improving the Expected Improvement Algorithm
To overcome this shortcoming, we introduce a simple modification of the expected improvement algorithm.
Time-Sensitive Bandit Learning and Satisficing Thompson Sampling
We propose satisficing Thompson sampling -- a variation of Thompson sampling -- and establish a strong discounted regret bound for this new algorithm.
How much does your data exploration overfit? Controlling bias via information usage
But while %the adaptive nature of exploration any data-exploration renders standard statistical theory invalid, experience suggests that different types of exploratory analysis can lead to disparate levels of bias, and the degree of bias also depends on the particulars of the data set.
An Information-Theoretic Analysis of Thompson Sampling
We provide an information-theoretic analysis of Thompson sampling that applies across a broad range of online optimization problems in which a decision-maker must learn from partial feedback.
Learning to Optimize Via Posterior Sampling
This paper considers the use of a simple posterior sampling algorithm to balance between exploration and exploitation when learning to optimize actions such as in multi-armed bandit problems.
(More) Efficient Reinforcement Learning via Posterior Sampling
This bound is one of the first for an algorithm not based on optimism, and close to the state of the art for any reinforcement learning algorithm.
Eluder Dimension and the Sample Complexity of Optimistic Exploration
This paper considers the sample complexity of the multi-armed bandit with dependencies among the arms.
A Note on the Equivalence of Upper Confidence Bounds and Gittins Indices for Patient Agents
This note gives a short, self-contained, proof of a sharp connection between Gittins indices and Bayesian upper confidence bound algorithms.
Global Optimality Guarantees For Policy Gradient Methods
Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices.
Worst-Case Regret Bounds for Exploration via Randomized Value Functions
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning.
SQuAP-Ont: an Ontology of Software Quality Relational Factors from Financial Systems
The SQuAP model (Software Quality, Architecture, Process) describes twenty-eight main factors that impact on software quality in banking systems, and each factor is described as a relation among some characteristics from the three ISO standards.
On Linear Convergence of Policy Gradient Methods for Finite MDPs
We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations.
Approximation Benefits of Policy Gradient Methods with Aggregated States
With the same representation, both approximate policy iteration and approximate value iteration can produce policies whose per-period regret scales as $\epsilon/(1-\gamma)$, where $\gamma$ is a discount factor.
Learning to Stop with Surprisingly Few Samples
We consider a discounted infinite horizon optimal stopping problem.
Adaptive Experimentation in the Presence of Exogenous Nonstationary Variation
We investigate experiments that are designed to select a treatment arm for population deployment.
On the Statistical Benefits of Temporal Difference Learning
Given a dataset on actions and resulting long-term rewards, a direct estimation approach fits value functions that minimize prediction error on the training data.
Optimizing Audio Recommendations for the Long-Term: A Reinforcement Learning Perspective
We study the problem of optimizing a recommender system for outcomes that occur over several weeks or months.
An Information-Theoretic Analysis of Nonstationary Bandit Learning
In nonstationary bandit learning problems, the decision-maker must continually gather information and adapt their action selection as the latent state of the environment evolves.
Neural Inventory Control in Networks via Hindsight Differentiable Policy Optimization
The first is Hindsight Differentiable Policy Optimization (HDPO), which performs stochastic gradient descent to optimize policy performance while avoiding the need to repeatedly deploy randomized policies in the environment-as is common with generic policy gradient methods.
Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification
Practitioners conducting adaptive experiments often encounter two competing priorities: reducing the cost of experimentation by effectively assigning treatments during the experiment itself, and gathering information swiftly to conclude the experiment and implement a treatment across the population.
On the Limited Representational Power of Value Functions and its Links to Statistical (In)Efficiency
In several, there is no information loss and value-based methods are as statistically efficient as model based ones.
