Search Results for author:
Found 67 papers, 14 papers with code
Active learning of neural population dynamics using two-photon holographic optogenetics
Using neural population responses to photostimulation in mouse motor cortex, we demonstrate the efficacy of a low-rank linear dynamical systems model, and develop an active learning procedure which takes advantage of low-rank structure to determine informative photostimulation patterns.
Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation
Recent advances in large language model (LLM) training have highlighted the need for diverse, high-quality instruction data.
Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
To the best of our knowledge, this is the first evidence that simulation transfer yields a provable gain in reinforcement learning in settings where direct sim2real transfer fails.
Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
Reinforcement learning with human feedback (RLHF), as a widely adopted approach in current large language model pipelines, is \textit{bottlenecked by the size of human preference data}.
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning
We present a novel multimodal preference dataset for creative tasks, consisting of over 250 million human ratings on more than 2. 2 million captions, collected through crowdsourcing rating data for The New Yorker's weekly cartoon caption contest over the past eight years.
Sample Complexity Reduction via Policy Difference Estimation in Tabular Reinforcement Learning
In this paper, we study the non-asymptotic sample complexity for the pure exploration problem in contextual bandits and tabular reinforcement learning (RL): identifying an epsilon-optimal policy from a set of policies with high probability.
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning
Three main data selection approaches are: (1) leveraging external non-CLIP models to aid data selection, (2) training new CLIP-style embedding models that are more effective at selecting high-quality data than the original OpenAI CLIP model, and (3) designing better metrics or strategies universally applicable to any CLIP embedding without requiring specific model properties (e. g., CLIPScore is one popular metric).
Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning
In recent years, data selection has emerged as a core issue for large-scale visual-language model pretraining, especially on noisy web-curated datasets.
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
To mitigate the annotation cost of SFT and circumvent the computational bottlenecks of active learning, we propose using experimental design.
Fair Active Learning in Low-Data Regimes
In this work, we address the challenges of reducing bias and improving accuracy in data-scarce environments, where the cost of collecting labeled data prohibits the use of large, labeled datasets.
Nearly Minimax Optimal Submodular Maximization with Bandit Feedback
In this work, we establish the first minimax lower bound for this setting that scales like $\tilde{\Omega}(\min_{L \le k}(L^{1/3}n^{1/3}T^{2/3} + \sqrt{{n \choose k - L}T}))$.
Near-Optimal Pure Exploration in Matrix Games: A Generalization of Stochastic Bandits & Dueling Bandits
We design a near-optimal algorithm whose sample complexity matches the lower bound, up to log factors.
Optimal Exploration is no harder than Thompson Sampling
In this work, we pose a natural question: is there an algorithm that can explore optimally and only needs the same computational primitives as Thompson Sampling?
Pick Planning Strategies for Large-Scale Package Manipulation
Automating warehouse operations can reduce logistics overhead costs, ultimately driving down the final price for consumers, increasing the speed of delivery, and enhancing the resiliency to market fluctuations.
A/B Testing and Best-arm Identification for Linear Bandits with Robustness to Non-stationarity
For robust identification, it is well-known that if arms are chosen randomly and non-adaptively from a G-optimal design over $\mathcal{X}$ at each time then the error probability decreases as $\exp(-T\Delta^2_{(1)}/d)$, where $\Delta_{(1)} = \min_{x \neq x^*} (x^* - x)^\top \frac{1}{T}\sum_{t=1}^T \theta_t$.
Logarithmic Regret for Matrix Games against an Adversary with Noisy Bandit Feedback
If the row player uses the EXP3 strategy, an algorithm known for obtaining $\sqrt{T}$ regret against an arbitrary sequence of rewards, it is immediate that the row player also achieves $\sqrt{T}$ regret relative to the Nash equilibrium in this game setting.
LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning
Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive.
Improved Active Multi-Task Representation Learning via Lasso
In addition to our sample complexity results, we also characterize the potential of our $\nu^1$-based strategy in sample-cost-sensitive settings.
Demonstrating Large-Scale Package Manipulation via Learned Metrics of Pick Success
This paper demonstrates a large-scale package manipulation from unstructured piles in Amazon Robotics' Robot Induction (Robin) fleet, which utilizes a pick success predictor trained on real production data.
Instance-dependent Sample Complexity Bounds for Zero-sum Matrix Games
We study the sample complexity of identifying an approximate equilibrium for two-player zero-sum $n\times 2$ matrix games.
Instance-Dependent Near-Optimal Policy Identification in Linear MDPs via Online Experiment Design
While much progress has been made in understanding the minimax sample complexity of reinforcement learning (RL) -- the complexity of learning on the "worst-case" instance -- such measures of complexity often do not capture the true difficulty of learning.
Instance-optimal PAC Algorithms for Contextual Bandits
In the stochastic contextual bandit setting, regret-minimizing algorithms have been extensively researched, but their instance-minimizing best-arm identification counterparts remain seldom studied.
Active Learning with Safety Constraints
To our knowledge, our results are the first on best-arm identification in linear bandits with safety constraints.
Active Multi-Task Representation Learning
To leverage the power of big data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications.
Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes
We first develop a computationally efficient algorithm for reward-free RL in a $d$-dimensional linear MDP with sample complexity scaling as $\widetilde{\mathcal{O}}(d^2 H^5/\epsilon^2)$.
First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach
Obtaining first-order regret bounds -- regret bounds scaling not as the worst-case but with some measure of the performance of the optimal policy on a given instance -- is a core question in sequential decision-making.
Best Arm Identification with Safety Constraints
The best arm identification problem in the multi-armed bandit setting is an excellent model of many real-world decision-making problems, yet it fails to capture the fact that in the real-world, safety constraints often must be met while learning.
Practical, Provably-Correct Interactive Learning in the Realizable Setting: The Power of True Believers
We begin our investigation with the observation that agnostic algorithms \emph{cannot} be minimax-optimal in the realizable setting.
Nearly Optimal Algorithms for Level Set Estimation
The threshold value $\alpha$ can either be \emph{explicit} and provided a priori, or \emph{implicit} and defined relative to the optimal function value, i. e. $\alpha = (1-\epsilon)f(x_\ast)$ for a given $\epsilon > 0$ where $f(x_\ast)$ is the maximal function value and is unknown.
Selective Sampling for Online Best-arm Identification
The main results of this work precisely characterize this trade-off between labeled samples and stopping time and provide an algorithm that nearly-optimally achieves the minimal label complexity given a desired stopping time.
Beyond No Regret: Instance-Dependent PAC Reinforcement Learning
We show this is not possible -- there exists a fundamental tradeoff between achieving low regret and identifying an $\epsilon$-optimal policy at the instance-optimal rate.
Corruption Robust Active Learning
We conduct theoretical studies on streaming-based active learning for binary classification under unknown adversarial label corruptions.
Improved Algorithms for Agnostic Pool-based Active Classification
We consider active learning for binary classification in the agnostic pool-based setting.
High-Dimensional Experimental Design and Kernel Bandits
We also leverage our new approach in a new algorithm for kernelized bandits to obtain state of the art results for regret minimization and pure exploration.
Improved Corruption Robust Algorithms for Episodic Reinforcement Learning
We study episodic reinforcement learning under unknown adversarial corruptions in both the rewards and the transition probabilities of the underlying system.
Task-Optimal Exploration in Linear Dynamical Systems
Along the way, we establish that certainty equivalence decision making is instance- and task-optimal, and obtain the first algorithm for the linear quadratic regulator problem which is instance-optimal.
Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding
Our key insight is that we can leverage the haptic context we collect during and after manipulation (i. e., "post hoc") to learn some of these properties and more quickly adapt our visual model to previously unseen food.
Experimental Design for Regret Minimization in Linear Bandits
In this paper we propose a novel experimental design-based algorithm to minimize regret in online stochastic linear and combinatorial bandits.
Learning to Actively Learn: A Robust Approach
Unlike the design of traditional adaptive algorithms that rely on concentration of measure and careful analysis to justify the correctness and sample complexity of the procedure, our adaptive algorithm is learned via adversarial training over equivalence classes of problems derived from information theoretic lower bounds.
A New Perspective on Pool-Based Active Classification and False-Discovery Control
In many scientific settings there is a need for adaptive experimental design to guide the process of identifying regions of the search space that contain as many true positives as possible subject to a low rate of false discoveries (i. e. false alarms).
An Empirical Process Approach to the Union Bound: Practical Algorithms for Combinatorial and Linear Bandits
This paper proposes near-optimal algorithms for the pure-exploration linear bandit problem in the fixed confidence and fixed budget settings.
Estimating the number and effect sizes of non-null hypotheses
We study the problem of estimating the distribution of effect sizes (the mean of the test statistic under the alternate hypothesis) in a multiple testing setting.
Active Learning for Identification of Linear Dynamical Systems
We propose an algorithm to actively estimate the parameters of a linear dynamical system.
Mosaic: A Sample-Based Database System for Open World Query Processing
Recently, with the increase in the number of public data repositories, sample data has become easier to access.
Sequential Experimental Design for Transductive Linear Bandits
Such a transductive setting naturally arises when the set of measurement vectors is limited due to factors such as availability or cost.
The True Sample Complexity of Identifying Good Arms
We consider two multi-armed bandit problems with $n$ arms: (i) given an $\epsilon > 0$, identify an arm with mean that is within $\epsilon$ of the largest mean and (ii) given a threshold $\mu_0$ and integer $k$, identify $k$ arms with means larger than $\mu_0$.
Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPs
This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Exploiting Reuse in Pipeline-Aware Hyperparameter Tuning
Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden.
Pure-Exploration for Infinite-Armed Bandits with General Arm Reservoirs
This paper considers a multi-armed bandit game where the number of arms is much larger than the maximum budget and is effectively infinite.
A System for Massively Parallel Hyperparameter Tuning
Modern learning models are characterized by large hyperparameter spaces and long training times.
A Bandit Approach to Multiple Testing with False Discovery Control
We propose an adaptive sampling approach for multiple testing which aims to maximize statistical power while ensuring anytime false discovery control.
Adaptive Sampling for Convex Regression
In this paper, we introduce the first principled adaptive-sampling procedure for learning a convex function in the $L_\infty$ norm, a problem that arises often in the behavioral and social sciences.
Firing Bandits: Optimizing Crowdfunding
In this paper, we model the problem of optimizing crowdfunding platforms, such as the non-profit Kiva or for-profit KickStarter, as a variant of the multi-armed bandit problem.
Massively Parallel Hyperparameter Tuning
Modern machine learning models are characterized by large hyperparameter search spaces and prohibitively expensive training costs.
A framework for Multi-A(rmed)/B(andit) testing with online FDR control
We propose an alternative framework to existing setups for controlling false alarms when multiple A/B tests are run over time.
Open Loop Hyperparameter Optimization and Determinantal Point Processes
Driven by the need for parallelizable hyperparameter optimization methods, this paper studies \emph{open loop} search methods: sequences that are predetermined and can be generated before a single configuration is evaluated.
The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime
Moreover, our lower bounds zero-in on the number of times each \emph{individual} arm needs to be pulled, uncovering new phenomena which are drowned out in the aggregate sample complexity.
Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations
Although policies learned with RC sampling can be superior to HC sampling for standard learning models such as linear SVMs, policies learned with HC sampling may be comparable with highly-expressive learning models such as deep learning and hyper-parametric decision trees, which have little model error.
Finite Sample Prediction and Recovery Bounds for Ordinal Embedding
First, we derive prediction error bounds for ordinal embedding with noise by exploiting the fact that the rank of a distance matrix of points in $\mathbb{R}^d$ is at most $d+2$.
On the Detection of Mixture Distributions with applications to the Most Biased Coin Problem
As a result, these bounds have surprising implications both for solutions to the most biased coin problem and for anomaly detection when only partial information about the parameters is known.
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters.
Best-of-K Bandits
This paper studies the Best-of-K Bandit game: At each time the player chooses a subset S among all N-choose-K possible options and observes reward max(X(i) : i in S) where X is a random vector drawn from a joint distribution.
Non-stochastic Best Arm Identification and Hyperparameter Optimization
Motivated by the task of hyperparameter optimization, we introduce the non-stochastic best-arm identification problem.
Sparse Dueling Bandits
We prove that in the absence of structural assumptions, the sample complexity of this problem is proportional to the sum of the inverse squared gaps between the Borda scores of each suboptimal arm and the best arm.
lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits
The paper proposes a novel upper confidence bound (UCB) procedure for identifying the arm with the largest mean in a multi-armed bandit game in the fixed confidence setting using a small number of total samples.
On Finding the Largest Mean Among Many
Motivated by large-scale applications, we are especially interested in identifying situations where the total number of samples that are necessary and sufficient to find the best arm scale linearly with the number of arms.
