Search Results for author:
Found 88 papers, 27 papers with code
Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learning
In this paper, we propose modeling offline MBRL as a Bayes Adaptive Markov Decision Process (BAMDP), which is a principled framework for addressing model uncertainty.
Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts
The traditional viewpoint on Sparse Mixture of Experts (MoE) models is that instead of training a single large expert, which is computationally expensive, we can train many small experts.
Assigning Credit with Partial Reward Decoupling in Multi-Agent Proximal Policy Optimization
We empirically demonstrate that our approach, PRD-MAPPO, decouples agents from teammates that do not influence their expected future reward, thereby streamlining credit assignment.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization
In this paper, we propose an enhancement to QMIX by incorporating an additional local Q-value learning method within the maximum entropy RL framework.
Planning with Adaptive World Models for Autonomous Driving
Motion planning is crucial for safe navigation in complex urban environments.
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
Large language models (LLMs) are trained on a vast amount of human-written data, but data providers often remain uncredited.
Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
Our main finding is that, in general, approaches that use on-policy sampling or attempt to push down the likelihood on certain responses (i. e., employ a "negative gradient") outperform offline and maximum likelihood objectives.
Full Shot Predictions for the DIII-D Tokamak via Deep Recurrent Networks
Although tokamaks are one of the most promising devices for realizing nuclear fusion as an energy source, there are still key obstacles when it comes to understanding the dynamics of the plasma and controlling it.
Tractable Joint Prediction and Planning over Discrete Behavior Modes for Urban Driving
Although these models have conventionally been evaluated for open-loop prediction, we show that they can be used to parameterize autoregressive closed-loop models without retraining.
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following
Diffusion-ES samples trajectories during evolutionary search from a diffusion model and scores them using a black-box reward function.
Decentralized Multi-Agent Active Search and Tracking when Targets Outnumber Agents
Multi-agent multi-target tracking has a wide range of applications, including wildlife patrolling, security surveillance or environment monitoring.
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving
Diffusion-ES samples trajectories during evolutionary search from a diffusion model and scores them using a black-box reward function.
Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration
Preference-based feedback is important for many applications in reinforcement learning where direct evaluation of a reward function is not feasible.
Reasoning with Latent Diffusion in Offline Reinforcement Learning
However, a key challenge in offline RL lies in effectively stitching portions of suboptimal trajectories from the static dataset while avoiding extrapolation errors arising due to a lack of support in the dataset.
Kernelized Offline Contextual Dueling Bandits
Preference-based feedback is important for many applications where direct evaluation of a reward function is not feasible.
Data Cross-Segmentation for Improved Generalization in Reinforcement Learning Based Algorithmic Trading
In a typical set-up, supervised learning is used to predict the future prices of assets, and those predictions drive a simple trading and execution strategy.
PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks
When this is the case, the policy needs to leverage the history of observations to infer the current state.
Enhancing Visual Domain Adaptation with Source Preparation
Our Almost Unsupervised Domain Adaptation (AUDA) framework, a label-efficient semi-supervised approach for robotic scenarios -- employs Source Preparation (SP), Unsupervised Domain Adaptation (UDA) and Supervised Alignment (SA) from limited labeled data.
GUTS: Generalized Uncertainty-Aware Thompson Sampling for Multi-Agent Active Search
We conduct field tests using our multi-robot system in an unstructured environment with a search area of approximately 75, 000 sq.
Near-optimal Policy Identification in Active Reinforcement Learning
Many real-world reinforcement learning tasks require control of complex dynamical systems that involve both costly data acquisition processes and large state spaces.
Exploration via Planning for Information about the Optimal Trajectory
In this work, we develop a method that allows us to plan for exploration while taking both the task and the current knowledge about the dynamics into account.
Cost Aware Asynchronous Multi-Agent Active Search
Multi-agent active search requires autonomous agents to choose sensing actions that efficiently locate targets.
Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning
Impressive results in natural language processing (NLP) based on the Transformer neural network architecture have inspired researchers to explore viewing offline reinforcement learning (RL) as a generic sequence modeling problem.
How Useful are Gradients for OOD Detection Really?
One critical challenge in deploying highly performant machine learning models in real-life applications is out of distribution (OOD) detection.
BATS: Best Action Trajectory Stitching
Past efforts for developing algorithms in this area have revolved around introducing constraints to online reinforcement learning algorithms to ensure the actions of the learned policy are constrained to the logged data.
Multi-Agent Active Search using Detection and Location Uncertainty
We then build a decision making algorithm on this inference method that uses Thompson sampling to enable decentralized multi-agent active search.
Robust Reinforcement Learning via Genetic Curriculum
Some of the state of the art approaches try to address the problem with adversarial agents, but these agents often require expert supervision to fine tune and prevent the adversary from becoming too challenging to the trainee agent.
Learning Cooperative Multi-Agent Policies with Partial Reward Decoupling
One of the preeminent obstacles to scaling multi-agent reinforcement learning to large numbers of agents is assigning credit to individual agents' actions.
An Experimental Design Perspective on Model-Based Reinforcement Learning
In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.
An Experimental Design Perspective on Exploration in Reinforcement Learning
In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.
Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty.
SBEVNet: End-to-End Deep Stereo Layout Estimation
Instead, the learning of a good internal bird's eye view feature representation is effective for layout estimation.
Computational catalyst discovery: Active classification through myopic multiscale sampling
We address this issue by relaxing the catalyst discovery goal into a classification problem: "What is the set of catalysts that is worth testing experimentally?"
Chemical Physics
Affordance-based Reinforcement Learning for Urban Driving
Traditional autonomous vehicle pipelines that follow a modular approach have been very successful in the past both in academia and industry, which has led to autonomy deployed on road.
Fine-Tuning Offline Reinforcement Learning with Model-Based Policy Optimization
Then, we can optionally enter a second stage where we fine-tune the policy using our novel Model-Based Behavior-Regularized Policy Optimization (MB2PO) algorithm.
Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification
However, this loss restricts the scope of applicable regression models, limits the ability to target many desirable properties (e. g. calibration, sharpness, centered intervals), and may produce poor conditional quantiles.
Multi-Agent Active Search using Realistic Depth-Aware Noise Model
The active search for objects of interest in an unknown environment has many robotics applications including search and rescue, detecting gas leaks or locating animal poachers.
Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning
In this work, we propose a behavior planning structure based on reinforcement learning (RL) which is capable of performing autonomous vehicle behavior planning with a hierarchical structure in simulated urban environments.
Interactive Visualization for Debugging RL
Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models.
Vizarel: A System to Help Better Understand RL Agents
Visualization tools for supervised learning have allowed users to interpret, introspect, and gain intuition for the successes and failures of their models.
Asynchronous Multi Agent Active Search
Active search refers to the problem of efficiently locating targets in an unknown environment by actively making data-collection decisions, and has many applications including detecting gas leaks, radiation sources or human survivors of disasters using aerial and/or ground robots (agents).
Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction
We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations.
Neural Dynamical Systems
We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models which incorporates prior knowledge in the form of systems of ordinary differential equations.
Offline Contextual Bayesian Optimization for Nuclear Fusion
One obstacle in utilizing fusion as a feasible energy source is the stability of the reaction.
Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning
In this work, we propose a hierarchical reinforcement learning (HRL) structure which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Human Driver Behavior Prediction based on UrbanFlow
How autonomous vehicles and human drivers share public transportation systems is an important problem, as fully automatic transportation environments are still a long way off.
ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations
In applications such as molecule design or drug discovery, it is desirable to have an algorithm which recommends new candidate molecules based on the results of past tests.
Deep Kinematic Models for Kinematically Feasible Vehicle Trajectory Predictions
Self-driving vehicles (SDVs) hold great potential for improving traffic safety and are poised to positively affect the quality of life of millions of people.
Predicting Motion of Vulnerable Road Users using High-Definition Maps and Efficient ConvNets
Following detection and tracking of traffic actors, prediction of their future motion is the next critical component of a self-driving vehicle (SDV) technology, allowing the SDV to operate safely and efficiently in its environment.
Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization
Motivated by our sales prediction problem, we propose a factor-based algorithm that is able to take time into account.
Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO.
ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum.
Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks
Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact.
Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
We address one of the crucial aspects necessary for safe and efficient operations of autonomous vehicles, namely predicting future state of traffic actors in the autonomous vehicle's surroundings.
Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming
We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal.
Neural Architecture Search with Bayesian Optimisation and Optimal Transport
A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model.
Transformation Autoregressive Networks
Further, through a comprehensive study over both real world and synthetic data, we show for that jointly leveraging transformations of variables and autoregressive conditional models, results in a considerable improvement in performance.
Ranked #1 on
Density Estimation
on BSDS300
Estimating Cosmological Parameters from the Dark Matter Distribution
A major approach to estimating the cosmological parameters is to use the large-scale matter distribution of the Universe.
Recurrent Estimation of Distributions
After, an RNN is used to compute the conditional distributions of the latent covariates.
Asynchronous Parallel Bayesian Optimisation via Thompson Sampling
We design and analyse variations of the classical Thompson sampling (TS) procedure for Bayesian optimisation (BO) in settings where function evaluations are expensive, but can be performed in parallel.
Scaling Active Search using Linear Similarity Functions
In this paper, we consider the problem of Active Search where we are given a similarity function between data points.
Multi-fidelity Bayesian Optimisation with Continuous Approximations
Bandit methods for black-box optimisation, such as Bayesian optimisation, are used in a variety of applications including hyper-parameter tuning and experiment design.
The Statistical Recurrent Unit
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications.
Equivariance Through Parameter-Sharing
We propose to study equivariance in deep neural networks through parameter symmetries.
Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learning
In this paper, we study active posterior estimation in a Bayesian setting when the likelihood is expensive to evaluate.
Active Search for Sparse Signals with Region Sensing
Autonomous systems can be used to search for sparse signals in a large space; e. g., aerial robots can be deployed to localize threats, detect gas leaks, or respond to distress calls.
Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations
However, in many cases, cheap approximations to $\func$ may be obtainable.
Deep Learning with Sets and Point Clouds
We introduce a simple permutation equivariant layer for deep learning with set structure. This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set.
The Multi-fidelity Multi-armed Bandit
We study a variant of the classical stochastic $K$-armed bandit where observing the outcome of each arm is expensive, but cheap approximations to this outcome are available.
Enabling Dark Energy Science with Deep Generative Models of Galaxy Images
To this end, we study the application of deep conditional generative models in generating realistic galaxy images.
Detecting Damped Lyman-$α$ Absorbers with Gaussian Processes
We develop an automated technique for detecting damped Lyman-$\alpha$ absorbers (DLAs) along spectroscopic lines of sight to quasi-stellar objects (QSOs or quasars).
Cosmology and Nongalactic Astrophysics Data Analysis, Statistics and Probability
Multi-fidelity Gaussian Process Bandit Optimisation
However, in many cases, cheap approximations to $f$ may be obtainable.
Stochastic Neural Networks with Monotonic Activation Functions
We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise.
Deep Mean Maps
The use of distributions and high-level features from deep architecture has become commonplace in modern computer vision.
Linear-time Learning on Distributions with Approximate Kernel Embeddings
This work develops the first random features for pdfs whose dot product approximates kernels using these non-Euclidean metrics, allowing estimators using such kernels to scale to large datasets by working in a primal space, without computing large Gram matrices.
Bayesian Nonparametric Kernel-Learning
In this paper we introduce Bayesian nonparmetric kernel-learning (BaNK), a generic, data-driven framework for scalable learning of kernels.
On the Error of Random Fourier Features
Kernel methods give powerful, flexible, and theoretically grounded approaches to solving many problems in machine learning.
High Dimensional Bayesian Optimisation and Bandits via Additive Models
We prove that, for additive functions the regret has only linear dependence on $D$ even though the function depends on all $D$ dimensions.
Flexible Transfer Learning under Support and Model Shift
Similarly, work on target/conditional shift focuses on matching marginal distributions on labels $Y$ and adjusting conditional distributions $P(X|Y)$, such that $P(X)$ can be matched across domains.
Fast Function to Function Regression
Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping between two separate types of distributions.
A Machine Learning Approach for Dynamical Mass Measurements of Galaxy Clusters
In the conventional method, we use a standard M(sigma_v) power law scaling relation to infer cluster mass, M, from line-of-sight (LOS) galaxy velocity dispersion, sigma_v.
Cosmology and Nongalactic Astrophysics
Σ-Optimality for Active Learning on Gaussian Random Fields
For active learning on GRFs, the commonly used V-optimality criterion queries nodes that reduce the L2 (regression) loss.
Learning Hidden Markov Models from Non-sequence Data via Tensor Decomposition
Under that framework, we identify reasonable assumptions on the generative process of non-sequence data, and propose learning algorithms based on the tensor decomposition method \cite{anandkumar2012tensor} to \textit{provably} recover first-order Markov models and hidden Markov models.
FuSSO: Functional Shrinkage and Selection Operator
We present the FuSSO, a functional analogue to the LASSO, that efficiently finds a sparse set of functional input covariates to regress a real-valued response against.
Fast Distribution To Real Regression
We study the problem of distribution to real-value regression, where one aims to regress a mapping $f$ that takes in a distribution input covariate $P\in \mathcal{I}$ (for a non-parametric family of distributions $\mathcal{I}$) and outputs a real-valued response $Y=f(P) + \epsilon$.
A First Look at creating mock catalogs with machine learning techniques
We investigate machine learning (ML) techniques for predicting the number of galaxies (N_gal) that occupy a halo, given the halo's properties.
Cosmology and Nongalactic Astrophysics
Bayesian Optimal Active Search and Surveying
In the second, active surveying, our goal is to actively query points to ultimately predict the proportion of a given class.
Kernels on Sample Sets via Nonparametric Divergence Estimates
Most machine learning algorithms, such as classification or regression, treat the individual data point as the object of interest.
