Search Results for author:
Found 119 papers, 51 papers with code
SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code
SceneCraft first models a scene graph as a blueprint, detailing the spatial relationships among assets in the scene.
Generalizability Under Sensor Failure: Tokenization + Transformers Enable More Robust Latent Spaces
A major goal in neuroscience is to discover neural data representations that generalize.
TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
To this end, we explore the impact of discrete, learnt, time series data representations that enable generalist, cross-domain training.
Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
We propose Stochastic Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions that can work with pre-trained diffusion models in a plug-and-play way, thus achieving training-free guidance for non-differentiable rules for the first time.
Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits
Central to our methodology is the application of robust regression, a distributionally robust technique tailored here to improve the estimation of conditional reward distribution from logging data.
SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning
To bridge these gaps, we introduce SciGLM, a suite of scientific language models able to conduct college-level scientific reasoning.
A Foundation Model for Cell Segmentation
Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.
Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation
Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods.
Automatic Gradient Descent: Deep Learning without Hyperparameters
Automatic gradient descent trains both fully-connected and convolutional networks out-of-the-box and at ImageNet scale.
Conformal Generative Modeling on Triangulated Surfaces
We propose conformal generative modeling, a framework for generative modeling on 2D surfaces approximated by discrete triangle meshes.
Eventual Discounting Temporal Logic Counterfactual Experience Replay
Linear temporal logic (LTL) offers a simplified way of specifying tasks for policy optimization that may otherwise be difficult to describe with scalar reward functions.
End-to-End Imitation Learning with Safety Guarantees using Control Barrier Functions
Imitation learning (IL) is a learning paradigm which can be used to synthesize controllers for complex systems that mimic behavior demonstrated by an expert (user or control algorithm).
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs
Forward invariance is a long-studied property in control theory that is used to certify that a dynamical system stays within some pre-specified set of states for all time, and also admits robustness guarantees (e. g., the certificate holds under perturbations).
Neurosymbolic Programming for Science
Neurosymbolic Programming (NP) techniques have the potential to accelerate scientific discovery.
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations.
Compactly Restrictable Metric Policy Optimization Problems
We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as Metric Policy Optimization Problems (MPOPs).
Policy Optimization with Linear Temporal Logic Constraints
We study the problem of policy optimization (PO) with linear temporal logic (LTL) constraints.
Deep Neural Imputation: A Framework for Recovering Incomplete Brain Recordings
Neuroscientists and neuroengineers have long relied on multielectrode neural recordings to study the brain.
Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds
Last, our control design extrapolates to unseen wind conditions, is shown to be effective for outdoor flights with only onboard sensors, and can transfer across drones with minimal performance degradation.
Investigating Generalization by Controlling Normalized Margin
Weight norm $\|w\|$ and margin $\gamma$ participate in learning theory via the normalized margin $\gamma/\|w\|$.
Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models
Existing design paradigms do not address the gap between theory (controller design with continuous time models) and practice (the discrete time sampled implementation of the resulting controllers); this can lead to poor performance and violations of safety for hardware instantiations.
MLNav: Learning to Safely Navigate on Martian Terrains
We present MLNav, a learning-enhanced path planning framework for safety-critical and resource-limited systems operating in complex environments, such as rovers navigating on Mars.
LyaNet: A Lyapunov Framework for Training Neural ODEs
Our approach, called LyaNet, is based on a novel Lyapunov loss formulation that encourages the inference dynamics to converge quickly to the correct prediction.
Self-Supervised Keypoint Discovery in Behavioral Videos
We propose a method for learning the posture and structure of agents from unlabelled behavioral videos.
Ranked #1 on
Unsupervised Human Pose Estimation
on Human3.6M
DeepGEM: Generalized Expectation-Maximization for Blind Inversion
In this paper, we study the problem of blind inversion: solving an inverse problem with unknown or imperfect knowledge of the forward model parameters.
Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis
We evaluate AutoSWAP in three behavior analysis domains and demonstrate that AutoSWAP outperforms existing approaches using only a fraction of the data.
Kernel Interpolation as a Bayes Point Machine
A Bayes point machine is a single classifier that approximates the majority decision of an ensemble of classifiers.
On the Implicit Biases of Architecture & Gradient Descent
Do neural networks generalise because of bias in the functions returned by gradient descent, or bias already present in the network architecture?
Cost-Sensitive Hierarchical Classification through Layer-wise Abstentions
For each layer, we also achieve higher accuracy when the overall accuracy is kept fixed across different methods.
Unsupervised Learning of Neurosymbolic Encoders
We present a framework for the unsupervised learning of neurosymbolic encoders, which are encoders obtained by composing neural networks with symbolic programs from a domain-specific language.
Meta-Adaptive Nonlinear Control: Theory and Algorithms
We provide instantiations of our approach under varying conditions, leading to the first non-asymptotic end-to-end convergence guarantee for multi-task nonlinear control.
Interpreting Expert Annotation Differences in Animal Behavior
Hand-annotated data can vary due to factors such as subjective differences, intra-rater variability, and differing annotator expertise.
Learning Pseudo-Backdoors for Mixed Integer Programs
In addition, we compare our learned approach against Gurobi, a state-of-the-art MIP solver, demonstrating that our method can be used to improve solver performance.
Fine-Grained System Identification of Nonlinear Neural Circuits
We study the problem of sparse nonlinear model recovery of high dimensional compositional functions.
End-to-End Sequential Sampling and Reconstruction for MRI
In this paper, we leverage the sequential nature of MRI measurements, and propose a fully differentiable framework that jointly learns a sequential sampling policy simultaneously with a reconstruction strategy.
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Multi-agent behavior modeling aims to understand the interactions that occur between agents.
Learning to Control an Unstable System with One Minute of Data: Leveraging Gaussian Process Differentiation in Predictive Control
We present a straightforward and efficient way to control unstable robotic systems using an estimated dynamics model.
Minimax Model Learning
We present a novel off-policy loss function for learning a transition model in model-based reinforcement learning.
Computing the Information Content of Trained Neural Networks
A simple resolution to this conundrum is that the number of weights is usually a bad proxy for the actual amount of information stored.
Learning Invariant Representation of Tasks for Robust Surgical State Estimation
The combination of high diversity and limited data calls for new learning methods that are robust and invariant to operating conditions and surgical techniques.
Learning by Turning: Neural Architecture Aware Optimisation
To address this problem, this paper conducts a combined study of neural architecture and optimisation, leading to a new optimiser called Nero: the neuronal rotator.
Disentangling Observed Causal Effects from Latent Confounders using Method of Moments
Thus, we provide a principled approach to tackling the joint problem of causal discovery and latent variable inference.
Learning to Make Decisions via Submodular Regularization
In particular, we focus on a class of combinatorial problems that can be solved via submodular maximization (either directly on the objective function or via submodular surrogates).
Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions
We present Neural-Swarm2, a learning-based method for motion planning and control that allows heterogeneous multirotors in a swarm to safely fly in close proximity.
The Power of Predictions in Online Control
We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics.
Task Programming: Learning Data Efficient Behavior Representations
The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.
Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty
Modern nonlinear control theory seeks to endow systems with properties such as stability and safety, and has been deployed successfully across various domains.
On the Benefits of Early Fusion in Multimodal Representation Learning
To facilitate the study of early multimodal fusion, we create a convolutional LSTM network architecture that simultaneously processes both audio and visual inputs, and allows us to select the layer at which audio and visual information combines.
Machine Learning Based Path Planning for Improved Rover Navigation (Pre-Print Version)
Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rover, sorts a list of candidate paths for the rover to traverse, then uses the Approximate Clearance Evaluation (ACE) algorithm to evaluate whether the most highly ranked paths are safe.
ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes
ROIAL learns Bayesian posteriors that predict each exoskeleton user's utility landscape across four exoskeleton gait parameters.
Architecture Agnostic Neural Networks
Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation.
Iterative Amortized Policy Optimization
Policy networks are a central feature of deep reinforcement learning (RL) algorithms for continuous control, enabling the estimation and sampling of high-value actions.
Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach
We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution.
Distributionally Robust Learning for Unsupervised Domain Adaptation
This formulation motivates the use of two neural networks that are jointly trained --- a discriminative network between the source and target domains for density-ratio estimation, in addition to the standard classification network.
Learning Differentiable Programs with Admissible Neural Heuristics
This relaxed program is differentiable and can be trained end-to-end, and the resulting training loss is an approximately admissible heuristic that can guide the combinatorial search.
Active Learning under Label Shift
We address the problem of active learning under label shift: when the class proportions of source and target domains differ.
Graph Neural Networks for the Prediction of Substrate-Specific Organic Reaction Conditions
We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions.
Deep Bayesian Quadrature Policy Optimization
On the other hand, more sample efficient alternatives like Bayesian quadrature methods have received little attention due to their high computational complexity.
Average-case Complexity of Teaching Convex Polytopes via Halfspace Queries
We investigate the average teaching complexity of the task, i. e., the minimal number of samples (halfspace queries) required by a teacher to help a version-space learner in locating a randomly selected target.
Learning compositional functions via multiplicative weight updates
This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional functions.
Competitive Policy Optimization
A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties.
Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems
The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints.
A General Large Neighborhood Search Framework for Solving Integer Linear Programs
This paper studies a strategy for data-driven algorithm design for large-scale combinatorial optimization problems that can leverage existing state-of-the-art solvers in general purpose ways.
Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits
Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space.
Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions
We design a stable nonlinear tracking controller using the learned model.
GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning
We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning.
Robotics
Online Optimization with Memory and Competitive Control
This paper presents competitive algorithms for a novel class of online optimization problems with memory.
Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis
Efficient and interpretable spatial analysis is crucial in many fields such as geology, sports, and climate science.
On the distance between two neural networks and the stability of learning
This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions.
Learning for Safety-Critical Control with Control Barrier Functions
Modern nonlinear control theory seeks to endow systems with properties of stability and safety, and have been deployed successfully in multiple domains.
Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications.
Triply Robust Off-Policy Evaluation
Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method.
Landmark Ordinal Embedding
In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k".
Learning Calibratable Policies using Programmatic Style-Consistency
We study the problem of controllable generation of long-term sequential behaviors, where the goal is to calibrate to multiple behavior styles simultaneously.
Dueling Posterior Sampling for Preference-Based Reinforcement Learning
In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback.
An Encoder-Decoder Based Approach for Anomaly Detection with Application in Additive Manufacturing
We show that the encoder-decoder model is able to identify the injected anomalies in a modern manufacturing process in an unsupervised fashion.
Imitation-Projected Programmatic Reinforcement Learning
First, we view our learning task as optimization in policy space, modulo the constraint that the desired policy has a programmatic representation, and solve this optimization problem using a form of mirror descent that takes a gradient step into the unconstrained policy space and then projects back onto the constrained space.
Co-training for Policy Learning
We study the problem of learning sequential decision-making policies in settings with multiple state-action representations.
Robust Regression for Safe Exploration in Control
To address this challenge, we present a deep robust regression model that is trained to directly predict the uncertainty bounds for safe exploration.
Control Regularization for Reduced Variance Reinforcement Learning
We show that functional regularization yields a bias-variance trade-off, and propose an adaptive tuning strategy to optimize this trade-off.
Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design
Importantly, we show that our objective function can be efficiently decomposed as a difference of submodular functions (DS), which allows us to employ DS optimization tools to greedily identify sets of constraints that increase the likelihood of finding items with high utility.
Batch Policy Learning under Constraints
When learning policies for real-world domains, two important questions arise: (i) how to efficiently use pre-collected off-policy, non-optimal behavior data; and (ii) how to mediate among different competing objectives and constraints.
A Control Lyapunov Perspective on Episodic Learning via Projection to State Stability
The goal of this paper is to understand the impact of learning on control synthesis from a Lyapunov function perspective.
Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
Many modern nonlinear control methods aim to endow systems with guaranteed properties, such as stability or safety, and have been successfully applied to the domain of robotics.
NAOMI: Non-Autoregressive Multiresolution Sequence Imputation
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems.
Ranked #1 on
Multivariate Time Series Imputation
on PEMS-SF
Neural Lander: Stable Drone Landing Control using Learned Dynamics
To the best of our knowledge, this is the first DNN-based nonlinear feedback controller with stability guarantees that can utilize arbitrarily large neural nets.
Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes
We apply numerical methods in combination with finite-difference-time-domain (FDTD) simulations to optimize transmission properties of plasmonic mirror color filters using a multi-objective figure of merit over a five-dimensional parameter space by utilizing novel multi-fidelity Gaussian processes approach.
A General Method for Amortizing Variational Filtering
We introduce the variational filtering EM algorithm, a simple, general-purpose method for performing variational inference in dynamical latent variable models using information from only past and present variables, i. e. filtering.
A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs?
Policy Gradient in Partially Observable Environments: Approximation and Convergence
Deploying these tools, we generalize a variety of existing theoretical guarantees, such as policy gradient and convergence theorems, to partially observable domains, those which also could be carried to more settings of interest.
PhaseLink: A Deep Learning Approach to Seismic Phase Association
For the examined datasets, PhaseLink can precisely associate P- and S-picks to events that are separated by ~12 seconds in origin time.
Iterative Amortized Inference
The failure of these models to reach fully optimized approximate posterior estimates results in an amortization gap.
Stagewise Safe Bayesian Optimization with Gaussian Processes
We provide theoretical guarantees for both the satisfaction of safety constraints as well as convergence to the optimal utility value.
Teaching Multiple Concepts to a Forgetful Learner
Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner.
Learning to Search via Retrospective Imitation
We study the problem of learning a good search policy for combinatorial search spaces.
Generating Multi-Agent Trajectories using Programmatic Weak Supervision
We study the problem of training sequential generative models for capturing coordinated multi-agent trajectory behavior, such as offensive basketball gameplay.
Detecting Adversarial Examples via Neural Fingerprinting
Deep neural networks are vulnerable to adversarial examples, which dramatically alter model output using small input changes.
Hierarchical Imitation and Reinforcement Learning
We study how to effectively leverage expert feedback to learn sequential decision-making policies.
Teaching Categories to Human Learners with Visual Explanations
We study the problem of computer-assisted teaching with explanations.
Multi-resolution Tensor Learning for Large-Scale Spatial Data
High-dimensional tensor models are notoriously computationally expensive to train.
Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners
We highlight that adaptivity does not speed up the teaching process when considering existing models of version space learners, such as "worst-case" (the learner picks the next hypothesis randomly from the version space) and "preference-based" (the learner picks hypothesis according to some global preference).
Learning to Infer
Inference models, which replace an optimization-based inference procedure with a learned model, have been fundamental in advancing Bayesian deep learning, the most notable example being variational auto-encoders (VAEs).
Structured Exploration via Hierarchical Variational Policy Networks
Reinforcement learning in environments with large state-action spaces is challenging, as exploration can be highly inefficient.
Long-term Forecasting using Tensor-Train RNNs
We present Tensor-Train RNN (TT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.
Long-term Forecasting using Higher Order Tensor RNNs
We present Higher-Order Tensor RNN (HOT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.
Correlational Dueling Bandits with Application to Clinical Treatment in Large Decision Spaces
This problem can be formulated as a $K$-armed Dueling Bandits problem where $K$ is the total number of decisions.
Factorized Variational Autoencoders for Modeling Audience Reactions to Movies
Matrix and tensor factorization methods are often used for finding underlying low-dimensional patterns from noisy data.
Generating Long-term Trajectories Using Deep Hierarchical Networks
We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations.
Multi-dueling Bandits with Dependent Arms
The dueling bandits problem is an online learning framework for learning from pairwise preference feedback, and is particularly well-suited for modeling settings that elicit subjective or implicit human feedback.
Coordinated Multi-Agent Imitation Learning
We study the problem of imitation learning from demonstrations of multiple coordinating agents.
Learning recurrent representations for hierarchical behavior modeling
We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network.
A Rotation Invariant Latent Factor Model for Moveme Discovery from Static Poses
We tackle the problem of learning a rotation invariant latent factor model when the training data is comprised of lower-dimensional projections of the original feature space.
Smooth Imitation Learning for Online Sequence Prediction
We study the problem of smooth imitation learning for online sequence prediction, where the goal is to train a policy that can smoothly imitate demonstrated behavior in a dynamic and continuous environment in response to online, sequential context input.
Learning Online Smooth Predictors for Realtime Camera Planning Using Recurrent Decision Trees
We study the problem of online prediction for realtime camera planning, where the goal is to predict smooth trajectories that correctly track and frame objects of interest (e. g., players in a basketball game).
Smooth Interactive Submodular Set Cover
Interactive submodular set cover is an interactive variant of submodular set cover over a hypothesis class of submodular functions, where the goal is to satisfy all sufficiently plausible submodular functions to a target threshold using as few (cost-weighted) actions as possible.
Knapsack Constrained Contextual Submodular List Prediction with Application to Multi-document Summarization
We study the problem of predicting a set or list of options under knapsack constraint.
Learning Policies for Contextual Submodular Prediction
Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options.
Linear Submodular Bandits and their Application to Diversified Retrieval
Diversified retrieval and online learning are two core research areas in the design of modern information retrieval systems. In this paper, we propose the linear submodular bandits problem, which is an online learning setting for optimizing a general class of feature-rich submodular utility models for diversified retrieval.
