Search Results for author: Yisong Yue

Found 133 papers, 61 papers with code

Practical Bayesian Algorithm Execution via Posterior Sampling

1 code implementation27 Oct 2024 Chu Xin Cheng, Raul Astudillo, Thomas Desautels, Yisong Yue

We consider Bayesian algorithm execution (BAX), a framework for efficiently selecting evaluation points of an expensive function to infer a property of interest encoded as the output of a base algorithm.

Unsupervised Representation Learning from Sparse Transformation Analysis

1 code implementation7 Oct 2024 Yue Song, Thomas Anderson Keller, Yisong Yue, Pietro Perona, Max Welling

In this paper we propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components.

Representation Learning

End-to-End Conformal Calibration for Optimization Under Uncertainty

1 code implementation30 Sep 2024 Christopher Yeh, Nicolas Christianson, Alan Wu, Adam Wierman, Yisong Yue

However, ensuring robustness guarantees requires well-calibrated uncertainty estimates, which can be difficult to achieve in high-capacity prediction models such as deep neural networks.

Conformal Prediction Decision Making +3

Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems

no code implementations30 Sep 2024 Hongkai Zheng, Wenda Chu, Austin Wang, Nikola Kovachki, Ricardo Baptista, Yisong Yue

This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications.

Constructive Nonlinear Control of Underactuated Systems via Zero Dynamics Policies

no code implementations27 Aug 2024 William Compton, Ivan Dario Jimenez Rodriguez, Noel Csomay-Shanklin, Yisong Yue, Aaron D. Ames

Stabilizing underactuated systems is an inherently challenging control task due to fundamental limitations on how the control input affects the unactuated dynamics.

Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search

1 code implementation20 Aug 2024 Jonathan Light, Min Cai, Weiqin Chen, Guanzhi Wang, Xiusi Chen, Wei Cheng, Yisong Yue, Ziniu Hu

In this paper, we propose a new method STRATEGIST that utilizes LLMs to acquire new skills for playing multi-agent games through a self-improvement process.

Decision Making Dialogue Generation

Beyond Numeric Awards: In-Context Dueling Bandits with LLM Agents

no code implementations2 Jul 2024 Fanzeng Xia, Hao liu, Yisong Yue, Tongxin Li

Our results reveal that LLMs, particularly GPT-4 Turbo, quickly identify the Condorcet winner, thus outperforming existing state-of-the-art algorithms in terms of weak regret.

Decision Making

Distributionally Robust Constrained Reinforcement Learning under Strong Duality

no code implementations22 Jun 2024 Zhengfei Zhang, Kishan Panaganti, Laixi Shi, Yanan Sui, Adam Wierman, Yisong Yue

We study the problem of Distributionally Robust Constrained RL (DRC-RL), where the goal is to maximize the expected reward subject to environmental distribution shifts and constraints.

Car Racing reinforcement-learning +1

Preferential Multi-Objective Bayesian Optimization

no code implementations20 Jun 2024 Raul Astudillo, Kejun Li, Maegan Tucker, Chu Xin Cheng, Aaron D. Ames, Yisong Yue

As a direct consequence, this result provides, to our knowledge, the first convergence guarantee for dueling Thompson sampling in the PBO setting.

Autonomous Driving Bayesian Optimization +1

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search

1 code implementation6 Jun 2024 Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, Jie Tang

We first show that the tree-search policy in ReST-MCTS* achieves higher accuracy compared with prior LLM reasoning baselines such as Best-of-N and Tree-of-Thought, within the same search budget.

Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller

1 code implementation4 Jun 2024 Min Cai, Yuchen Zhang, Shichang Zhang, Fan Yin, Dan Zhang, Difan Zou, Yisong Yue, Ziniu Hu

Given a desired behavior expressed in a natural language suffix string concatenated to the input prompt, SelfControl computes gradients of the LLM's self-evaluation of the suffix with respect to its latent representations.

Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors

1 code implementation29 May 2024 Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, Katherine L. Bouman

Diffusion models (DMs) have recently shown outstanding capabilities in modeling complex image distributions, making them expressive image priors for solving Bayesian inverse problems.

Image Deblurring Image Super-Resolution

TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis

1 code implementation26 Feb 2024 Sabera Talukder, Yisong Yue, Georgia Gkioxari

To this end, we explore the impact of discrete, learnt, time series data representations that enable generalist, cross-domain training.

Time Series Time Series Analysis

Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

1 code implementation22 Feb 2024 Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, Yisong Yue

We propose Stochastic Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions that can work with pre-trained diffusion models in a plug-and-play way, thus achieving training-free guidance for non-differentiable rules for the first time.

Music Generation

Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits

1 code implementation21 Jan 2024 Yihong Guo, Hao liu, Yisong Yue, Anqi Liu

Central to our methodology is the application of robust regression, a distributionally robust technique tailored here to improve the estimation of conditional reward distribution from logging data.

Multi-Armed Bandits regression

A Foundation Model for Cell Segmentation

no code implementations18 Nov 2023 Uriah Israel, Markus Marks, Rohit Dilip, Qilin Li, Morgan Schwartz, Elora Pradhan, Edward Pao, Shenyi Li, Alexander Pearson-Goulart, Pietro Perona, Georgia Gkioxari, Ross Barnowski, Yisong Yue, David Van Valen

Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.

Cell Segmentation Prompt Engineering +1

Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation

no code implementations25 Jul 2023 Fengxue Zhang, Jialin Song, James Bowden, Alexander Ladd, Yisong Yue, Thomas A. Desautels, Yuxin Chen

Our approach is easy to tune, and is able to focus on local region of the optimization space that can be tackled by existing BO methods.

Bayesian Optimization

Automatic Gradient Descent: Deep Learning without Hyperparameters

1 code implementation11 Apr 2023 Jeremy Bernstein, Chris Mingard, Kevin Huang, Navid Azizan, Yisong Yue

Automatic gradient descent trains both fully-connected and convolutional networks out-of-the-box and at ImageNet scale.

Deep Learning Second-order methods

Conformal Generative Modeling on Triangulated Surfaces

1 code implementation17 Mar 2023 Victor Dorobantu, Charlotte Borcherds, Yisong Yue

We propose conformal generative modeling, a framework for generative modeling on 2D surfaces approximated by discrete triangle meshes.

Eventual Discounting Temporal Logic Counterfactual Experience Replay

no code implementations3 Mar 2023 Cameron Voloshin, Abhinav Verma, Yisong Yue

Linear temporal logic (LTL) offers a simplified way of specifying tasks for policy optimization that may otherwise be difficult to describe with scalar reward functions.

counterfactual Counterfactual Reasoning

End-to-End Imitation Learning with Safety Guarantees using Control Barrier Functions

1 code implementation21 Dec 2022 Ryan K. Cosner, Yisong Yue, Aaron D. Ames

Imitation learning (IL) is a learning paradigm which can be used to synthesize controllers for complex systems that mimic behavior demonstrated by an expert (user or control algorithm).

Imitation Learning

BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos

1 code implementation CVPR 2023 Jennifer J. Sun, Lili Karashchuk, Amil Dravid, Serim Ryou, Sonia Fereidooni, John Tuthill, Aggelos Katsaggelos, Bingni W. Brunton, Georgia Gkioxari, Ann Kennedy, Yisong Yue, Pietro Perona

In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.

Decoder

FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs

1 code implementation30 Oct 2022 Yujia Huang, Ivan Dario Jimenez Rodriguez, huan zhang, Yuanyuan Shi, Yisong Yue

Forward invariance is a long-studied property in control theory that is used to certify that a dynamical system stays within some pre-specified set of states for all time, and also admits robustness guarantees (e. g., the certificate holds under perturbations).

Adversarial Robustness continuous-control +2

Compactly Restrictable Metric Policy Optimization Problems

no code implementations12 Jul 2022 Victor D. Dorobantu, Kamyar Azizzadenesheli, Yisong Yue

We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as Metric Policy Optimization Problems (MPOPs).

continuous-control Continuous Control

Policy Optimization with Linear Temporal Logic Constraints

no code implementations20 Jun 2022 Cameron Voloshin, Hoang M. Le, Swarat Chaudhuri, Yisong Yue

We study the problem of policy optimization (PO) with linear temporal logic (LTL) constraints.

Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds

1 code implementation13 May 2022 Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, Soon-Jo Chung

Last, our control design extrapolates to unseen wind conditions, is shown to be effective for outdoor flights with only onboard sensors, and can transfer across drones with minimal performance degradation.

Meta-Learning

Investigating Generalization by Controlling Normalized Margin

1 code implementation8 May 2022 Alexander R. Farhang, Jeremy Bernstein, Kushal Tirumala, Yang Liu, Yisong Yue

Weight norm $\|w\|$ and margin $\gamma$ participate in learning theory via the normalized margin $\gamma/\|w\|$.

Learning Theory

Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models

no code implementations22 Mar 2022 Andrew J. Taylor, Victor D. Dorobantu, Ryan K. Cosner, Yisong Yue, Aaron D. Ames

Existing design paradigms do not address the gap between theory (controller design with continuous time models) and practice (the discrete time sampled implementation of the resulting controllers); this can lead to poor performance and violations of safety for hardware instantiations.

MLNav: Learning to Safely Navigate on Martian Terrains

no code implementations9 Mar 2022 Shreyansh Daftry, Neil Abcouwer, Tyler del Sesto, Siddarth Venkatraman, Jialin Song, Lucas Igel, Amos Byon, Ugo Rosolia, Yisong Yue, Masahiro Ono

We present MLNav, a learning-enhanced path planning framework for safety-critical and resource-limited systems operating in complex environments, such as rovers navigating on Mars.

Navigate

LyaNet: A Lyapunov Framework for Training Neural ODEs

1 code implementation5 Feb 2022 Ivan Dario Jimenez Rodriguez, Aaron D. Ames, Yisong Yue

Our approach, called LyaNet, is based on a novel Lyapunov loss formulation that encourages the inference dynamics to converge quickly to the correct prediction.

Adversarial Robustness

DeepGEM: Generalized Expectation-Maximization for Blind Inversion

1 code implementation NeurIPS 2021 Angela Gao, Jorge Castellanos, Yisong Yue, Zachary Ross, Katherine Bouman

In this paper, we study the problem of blind inversion: solving an inverse problem with unknown or imperfect knowledge of the forward model parameters.

Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis

1 code implementation CVPR 2022 Albert Tseng, Jennifer J. Sun, Yisong Yue

We evaluate AutoSWAP in three behavior analysis domains and demonstrate that AutoSWAP outperforms existing approaches using only a fraction of the data.

Diversity Program Synthesis

Kernel Interpolation as a Bayes Point Machine

1 code implementation8 Oct 2021 Jeremy Bernstein, Alex Farhang, Yisong Yue

A Bayes point machine is a single classifier that approximates the majority decision of an ensemble of classifiers.

Bayesian Inference

On the Implicit Biases of Architecture & Gradient Descent

no code implementations29 Sep 2021 Jeremy Bernstein, Yisong Yue

Do neural networks generalise because of bias in the functions returned by gradient descent, or bias already present in the network architecture?

Bayesian Inference

Unsupervised Learning of Neurosymbolic Encoders

1 code implementation28 Jul 2021 Eric Zhan, Jennifer J. Sun, Ann Kennedy, Yisong Yue, Swarat Chaudhuri

We present a framework for the unsupervised learning of neurosymbolic encoders, which are encoders obtained by composing neural networks with symbolic programs from a domain-specific language.

Decoder Program Synthesis +1

Interpreting Expert Annotation Differences in Animal Behavior

no code implementations11 Jun 2021 Megan Tjandrasuwita, Jennifer J. Sun, Ann Kennedy, Swarat Chaudhuri, Yisong Yue

Hand-annotated data can vary due to factors such as subjective differences, intra-rater variability, and differing annotator expertise.

Program Synthesis

Meta-Adaptive Nonlinear Control: Theory and Algorithms

1 code implementation NeurIPS 2021 Guanya Shi, Kamyar Azizzadenesheli, Michael O'Connell, Soon-Jo Chung, Yisong Yue

We provide instantiations of our approach under varying conditions, leading to the first non-asymptotic end-to-end convergence guarantee for multi-task nonlinear control.

Multi-Task Learning Representation Learning

Fine-Grained System Identification of Nonlinear Neural Circuits

1 code implementation9 Jun 2021 Dawna Bagherian, James Gornet, Jeremy Bernstein, Yu-Li Ni, Yisong Yue, Markus Meister

We study the problem of sparse nonlinear model recovery of high dimensional compositional functions.

Learning Pseudo-Backdoors for Mixed Integer Programs

no code implementations9 Jun 2021 Aaron Ferber, Jialin Song, Bistra Dilkina, Yisong Yue

In addition, we compare our learned approach against Gurobi, a state-of-the-art MIP solver, demonstrating that our method can be used to improve solver performance.

Combinatorial Optimization

End-to-End Sequential Sampling and Reconstruction for MRI

1 code implementation13 May 2021 Tianwei Yin, Zihui Wu, He Sun, Adrian V. Dalca, Yisong Yue, Katherine L. Bouman

In this paper, we leverage the sequential nature of MRI measurements, and propose a fully differentiable framework that jointly learns a sequential sampling policy simultaneously with a reconstruction strategy.

Minimax Model Learning

no code implementations2 Mar 2021 Cameron Voloshin, Nan Jiang, Yisong Yue

We present a novel off-policy loss function for learning a transition model in model-based reinforcement learning.

Model-based Reinforcement Learning Off-policy evaluation +1

Computing the Information Content of Trained Neural Networks

1 code implementation1 Mar 2021 Jeremy Bernstein, Yisong Yue

A simple resolution to this conundrum is that the number of weights is usually a bad proxy for the actual amount of information stored.

Learning Invariant Representation of Tasks for Robust Surgical State Estimation

no code implementations18 Feb 2021 Yidan Qin, Max Allan, Yisong Yue, Joel W. Burdick, Mahdi Azizian

The combination of high diversity and limited data calls for new learning methods that are robust and invariant to operating conditions and surgical techniques.

Diversity

Learning by Turning: Neural Architecture Aware Optimisation

2 code implementations14 Feb 2021 Yang Liu, Jeremy Bernstein, Markus Meister, Yisong Yue

To address this problem, this paper conducts a combined study of neural architecture and optimisation, leading to a new optimiser called Nero: the neuronal rotator.

Learning to Make Decisions via Submodular Regularization

no code implementations ICLR 2021 Ayya Alieva, Aiden Aceves, Jialin Song, Stephen Mayo, Yisong Yue, Yuxin Chen

In particular, we focus on a class of combinatorial problems that can be solved via submodular maximization (either directly on the objective function or via submodular surrogates).

Active Learning Bayesian Optimization +2

Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions

no code implementations10 Dec 2020 Guanya Shi, Wolfgang Hönig, Xichen Shi, Yisong Yue, Soon-Jo Chung

We present Neural-Swarm2, a learning-based method for motion planning and control that allows heterogeneous multirotors in a swarm to safely fly in close proximity.

Motion Planning

The Power of Predictions in Online Control

no code implementations NeurIPS 2020 Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, Adam Wierman

We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics.

Task Programming: Learning Data Efficient Behavior Representations

1 code implementation CVPR 2021 Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona

The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.

Self-Supervised Learning

Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty

no code implementations21 Nov 2020 Andrew J. Taylor, Victor D. Dorobantu, Sarah Dean, Benjamin Recht, Yisong Yue, Aaron D. Ames

Modern nonlinear control theory seeks to endow systems with properties such as stability and safety, and has been deployed successfully across various domains.

On the Benefits of Early Fusion in Multimodal Representation Learning

no code implementations NeurIPS Workshop SVRHM 2020 George Barnum, Sabera Talukder, Yisong Yue

To facilitate the study of early multimodal fusion, we create a convolutional LSTM network architecture that simultaneously processes both audio and visual inputs, and allows us to select the layer at which audio and visual information combines.

Representation Learning

Machine Learning Based Path Planning for Improved Rover Navigation (Pre-Print Version)

no code implementations11 Nov 2020 Neil Abcouwer, Shreyansh Daftry, Siddarth Venkatraman, Tyler del Sesto, Olivier Toupet, Ravi Lanka, Jialin Song, Yisong Yue, Masahiro Ono

Enhanced AutoNav (ENav), the baseline surface navigation software for NASA's Perseverance rover, sorts a list of candidate paths for the rover to traverse, then uses the Approximate Clearance Evaluation (ACE) algorithm to evaluate whether the most highly ranked paths are safe.

BIG-bench Machine Learning

Architecture Agnostic Neural Networks

no code implementations5 Nov 2020 Sabera Talukder, Guruprasad Raghavan, Yisong Yue

Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation.

Iterative Amortized Policy Optimization

1 code implementation NeurIPS 2021 Joseph Marino, Alexandre Piché, Alessandro Davide Ialongo, Yisong Yue

Policy networks are a central feature of deep reinforcement learning (RL) algorithms for continuous control, enabling the estimation and sampling of high-value actions.

continuous-control Continuous Control +3

Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach

no code implementations8 Oct 2020 Haoxuan Wang, Zhiding Yu, Yisong Yue, Anima Anandkumar, Anqi Liu, Junchi Yan

We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution.

Density Ratio Estimation Unsupervised Domain Adaptation

Distributionally Robust Learning for Unsupervised Domain Adaptation

no code implementations28 Sep 2020 Haoxuan Wang, Anqi Liu, Zhiding Yu, Yisong Yue, Anima Anandkumar

This formulation motivates the use of two neural networks that are jointly trained --- a discriminative network between the source and target domains for density-ratio estimation, in addition to the standard classification network.

Density Ratio Estimation Unsupervised Domain Adaptation

Learning Differentiable Programs with Admissible Neural Heuristics

1 code implementation NeurIPS 2020 Ameesh Shah, Eric Zhan, Jennifer J. Sun, Abhinav Verma, Yisong Yue, Swarat Chaudhuri

This relaxed program is differentiable and can be trained end-to-end, and the resulting training loss is an approximately admissible heuristic that can guide the combinatorial search.

Active Learning under Label Shift

no code implementations16 Jul 2020 Eric Zhao, Anqi Liu, Animashree Anandkumar, Yisong Yue

We address the problem of active learning under label shift: when the class proportions of source and target domains differ.

Active Learning

Deep Bayesian Quadrature Policy Optimization

1 code implementation28 Jun 2020 Akella Ravi Tej, Kamyar Azizzadenesheli, Mohammad Ghavamzadeh, Anima Anandkumar, Yisong Yue

On the other hand, more sample efficient alternatives like Bayesian quadrature methods have received little attention due to their high computational complexity.

continuous-control Continuous Control +1

Average-case Complexity of Teaching Convex Polytopes via Halfspace Queries

no code implementations25 Jun 2020 Akash Kumar, Adish Singla, Yisong Yue, Yuxin Chen

We investigate the average teaching complexity of the task, i. e., the minimal number of samples (halfspace queries) required by a teacher to help a version-space learner in locating a randomly selected target.

Learning compositional functions via multiplicative weight updates

1 code implementation NeurIPS 2020 Jeremy Bernstein, Jia-Wei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, Yisong Yue

This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional functions.

LEMMA

Competitive Policy Optimization

4 code implementations18 Jun 2020 Manish Prajapat, Kamyar Azizzadenesheli, Alexander Liniger, Yisong Yue, Anima Anandkumar

A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties.

Policy Gradient Methods

Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

no code implementations9 May 2020 Yashwanth Kumar Nakka, Anqi Liu, Guanya Shi, Anima Anandkumar, Yisong Yue, Soon-Jo Chung

The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints.

Motion Planning Optimal Motion Planning +1

A General Large Neighborhood Search Framework for Solving Integer Linear Programs

no code implementations NeurIPS 2020 Jialin Song, Ravi Lanka, Yisong Yue, Bistra Dilkina

This paper studies a strategy for data-driven algorithm design for large-scale combinatorial optimization problems that can leverage existing state-of-the-art solvers in general purpose ways.

Combinatorial Optimization

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

1 code implementation13 Mar 2020 Maegan Tucker, Myra Cheng, Ellen Novoseller, Richard Cheng, Yisong Yue, Joel W. Burdick, Aaron D. Ames

Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space.

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning

1 code implementation26 Feb 2020 Benjamin Rivière, Wolfgang Hoenig, Yisong Yue, Soon-Jo Chung

We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning.

Robotics

Online Optimization with Memory and Competitive Control

1 code implementation NeurIPS 2020 Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, Adam Wierman

This paper presents competitive algorithms for a novel class of online optimization problems with memory.

Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis

1 code implementation ICML 2020 Jung Yeon Park, Kenneth Theo Carr, Stephan Zheng, Yisong Yue, Rose Yu

Efficient and interpretable spatial analysis is crucial in many fields such as geology, sports, and climate science.

On the distance between two neural networks and the stability of learning

2 code implementations NeurIPS 2020 Jeremy Bernstein, Arash Vahdat, Yisong Yue, Ming-Yu Liu

This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions.

LEMMA

Learning for Safety-Critical Control with Control Barrier Functions

no code implementations L4DC 2020 Andrew Taylor, Andrew Singletary, Yisong Yue, Aaron Ames

Modern nonlinear control theory seeks to endow systems with properties of stability and safety, and have been deployed successfully in multiple domains.

BIG-bench Machine Learning

Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning

3 code implementations15 Nov 2019 Cameron Voloshin, Hoang M. Le, Nan Jiang, Yisong Yue

We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications.

Benchmarking Diversity +4

Triply Robust Off-Policy Evaluation

no code implementations13 Nov 2019 Anqi Liu, Hao liu, Anima Anandkumar, Yisong Yue

Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method.

Multi-Armed Bandits Off-policy evaluation +1

Landmark Ordinal Embedding

no code implementations NeurIPS 2019 Nikhil Ghosh, Yuxin Chen, Yisong Yue

In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k".

Computational Efficiency

Learning Calibratable Policies using Programmatic Style-Consistency

2 code implementations ICML 2020 Eric Zhan, Albert Tseng, Yisong Yue, Adith Swaminathan, Matthew Hausknecht

We study the problem of controllable generation of long-term sequential behaviors, where the goal is to calibrate to multiple behavior styles simultaneously.

Imitation Learning

Dueling Posterior Sampling for Preference-Based Reinforcement Learning

1 code implementation4 Aug 2019 Ellen R. Novoseller, Yibing Wei, Yanan Sui, Yisong Yue, Joel W. Burdick

In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback.

reinforcement-learning Reinforcement Learning +1

Imitation-Projected Programmatic Reinforcement Learning

no code implementations NeurIPS 2019 Abhinav Verma, Hoang M. Le, Yisong Yue, Swarat Chaudhuri

First, we view our learning task as optimization in policy space, modulo the constraint that the desired policy has a programmatic representation, and solve this optimization problem using a form of mirror descent that takes a gradient step into the unconstrained policy space and then projects back onto the constrained space.

continuous-control Continuous Control +5

Co-training for Policy Learning

1 code implementation3 Jul 2019 Jialin Song, Ravi Lanka, Yisong Yue, Masahiro Ono

We study the problem of learning sequential decision-making policies in settings with multiple state-action representations.

Combinatorial Optimization continuous-control +3

Robust Regression for Safe Exploration in Control

no code implementations L4DC 2020 Anqi Liu, Guanya Shi, Soon-Jo Chung, Anima Anandkumar, Yisong Yue

To address this challenge, we present a deep robust regression model that is trained to directly predict the uncertainty bounds for safe exploration.

Generalization Bounds regression +1

Control Regularization for Reduced Variance Reinforcement Learning

1 code implementation14 May 2019 Richard Cheng, Abhinav Verma, Gabor Orosz, Swarat Chaudhuri, Yisong Yue, Joel W. Burdick

We show that functional regularization yields a bias-variance trade-off, and propose an adaptive tuning strategy to optimize this trade-off.

continuous-control Continuous Control +3

Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design

no code implementations17 Apr 2019 Kevin K. Yang, Yuxin Chen, Alycia Lee, Yisong Yue

Importantly, we show that our objective function can be efficiently decomposed as a difference of submodular functions (DS), which allows us to employ DS optimization tools to greedily identify sets of constraints that increase the likelihood of finding items with high utility.

Bayesian Optimization Experimental Design

Batch Policy Learning under Constraints

2 code implementations20 Mar 2019 Hoang M. Le, Cameron Voloshin, Yisong Yue

When learning policies for real-world domains, two important questions arise: (i) how to efficiently use pre-collected off-policy, non-optimal behavior data; and (ii) how to mediate among different competing objectives and constraints.

Reinforcement Learning

A Control Lyapunov Perspective on Episodic Learning via Projection to State Stability

no code implementations18 Mar 2019 Andrew J. Taylor, Victor D. Dorobantu, Meera Krishnamoorthy, Hoang M. Le, Yisong Yue, Aaron D. Ames

The goal of this paper is to understand the impact of learning on control synthesis from a Lyapunov function perspective.

Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems

no code implementations4 Mar 2019 Andrew J. Taylor, Victor D. Dorobantu, Hoang M. Le, Yisong Yue, Aaron D. Ames

Many modern nonlinear control methods aim to endow systems with guaranteed properties, such as stability or safety, and have been successfully applied to the domain of robotics.

Neural Lander: Stable Drone Landing Control using Learned Dynamics

2 code implementations19 Nov 2018 Guanya Shi, Xichen Shi, Michael O'Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, Soon-Jo Chung

To the best of our knowledge, this is the first DNN-based nonlinear feedback controller with stability guarantees that can utilize arbitrarily large neural nets.

Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes

no code implementations15 Nov 2018 Jialin Song, Yury S. Tokpanov, Yuxin Chen, Dagny Fleischman, Kate T. Fountaine, Harry A. Atwater, Yisong Yue

We apply numerical methods in combination with finite-difference-time-domain (FDTD) simulations to optimize transmission properties of plasmonic mirror color filters using a multi-objective figure of merit over a five-dimensional parameter space by utilizing novel multi-fidelity Gaussian processes approach.

Gaussian Processes

A General Method for Amortizing Variational Filtering

1 code implementation NeurIPS 2018 Joseph Marino, Milan Cvitkovic, Yisong Yue

We introduce the variational filtering EM algorithm, a simple, general-purpose method for performing variational inference in dynamical latent variable models using information from only past and present variables, i. e. filtering.

Inference Optimization Variational Inference

A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes

no code implementations2 Nov 2018 Jialin Song, Yuxin Chen, Yisong Yue

How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs?

Bayesian Optimization Gaussian Processes

Policy Gradient in Partially Observable Environments: Approximation and Convergence

no code implementations18 Oct 2018 Kamyar Azizzadenesheli, Yisong Yue, Animashree Anandkumar

Deploying these tools, we generalize a variety of existing theoretical guarantees, such as policy gradient and convergence theorems, to partially observable domains, those which also could be carried to more settings of interest.

Decision Making Policy Gradient Methods +1

PhaseLink: A Deep Learning Approach to Seismic Phase Association

no code implementations8 Sep 2018 Zachary E. Ross, Yisong Yue, Men-Andrin Meier, Egill Hauksson, Thomas H. Heaton

For the examined datasets, PhaseLink can precisely associate P- and S-picks to events that are separated by ~12 seconds in origin time.

Deep Learning

Iterative Amortized Inference

1 code implementation ICML 2018 Joseph Marino, Yisong Yue, Stephan Mandt

The failure of these models to reach fully optimized approximate posterior estimates results in an amortization gap.

Inference Optimization Variational Inference

Stagewise Safe Bayesian Optimization with Gaussian Processes

no code implementations ICML 2018 Yanan Sui, Vincent Zhuang, Joel W. Burdick, Yisong Yue

We provide theoretical guarantees for both the satisfaction of safety constraints as well as convergence to the optimal utility value.

Bayesian Optimization Decision Making +2

Teaching Multiple Concepts to a Forgetful Learner

no code implementations NeurIPS 2019 Anette Hunziker, Yuxin Chen, Oisin Mac Aodha, Manuel Gomez Rodriguez, Andreas Krause, Pietro Perona, Yisong Yue, Adish Singla

Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner.

Scheduling

Learning to Search via Retrospective Imitation

no code implementations3 Apr 2018 Jialin Song, Ravi Lanka, Albert Zhao, Aadyot Bhatnagar, Yisong Yue, Masahiro Ono

We study the problem of learning a good search policy for combinatorial search spaces.

Imitation Learning

Generating Multi-Agent Trajectories using Programmatic Weak Supervision

2 code implementations ICLR 2019 Eric Zhan, Stephan Zheng, Yisong Yue, Long Sha, Patrick Lucey

We study the problem of training sequential generative models for capturing coordinated multi-agent trajectory behavior, such as offensive basketball gameplay.

Imitation Learning

Detecting Adversarial Examples via Neural Fingerprinting

1 code implementation11 Mar 2018 Sumanth Dathathri, Stephan Zheng, Tianwei Yin, Richard M. Murray, Yisong Yue

Deep neural networks are vulnerable to adversarial examples, which dramatically alter model output using small input changes.

Multi-resolution Tensor Learning for Large-Scale Spatial Data

no code implementations19 Feb 2018 Stephan Zheng, Rose Yu, Yisong Yue

High-dimensional tensor models are notoriously computationally expensive to train.

Meta-Learning

Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners

no code implementations NeurIPS 2018 Yuxin Chen, Adish Singla, Oisin Mac Aodha, Pietro Perona, Yisong Yue

We highlight that adaptivity does not speed up the teaching process when considering existing models of version space learners, such as "worst-case" (the learner picks the next hypothesis randomly from the version space) and "preference-based" (the learner picks hypothesis according to some global preference).

Structured Exploration via Hierarchical Variational Policy Networks

no code implementations ICLR 2018 Stephan Zheng, Yisong Yue

Reinforcement learning in environments with large state-action spaces is challenging, as exploration can be highly inefficient.

Reinforcement Learning Variational Inference

Learning to Infer

no code implementations ICLR 2018 Joseph Marino, Yisong Yue, Stephan Mandt

Inference models, which replace an optimization-based inference procedure with a learned model, have been fundamental in advancing Bayesian deep learning, the most notable example being variational auto-encoders (VAEs).

Inference Optimization

Long-term Forecasting using Tensor-Train RNNs

no code implementations ICLR 2018 Rose Yu, Stephan Zheng, Anima Anandkumar, Yisong Yue

We present Tensor-Train RNN (TT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.

Long-term Forecasting using Higher Order Tensor RNNs

1 code implementation ICLR 2018 Rose Yu, Stephan Zheng, Anima Anandkumar, Yisong Yue

We present Higher-Order Tensor RNN (HOT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.

Time Series Time Series Analysis

Factorized Variational Autoencoders for Modeling Audience Reactions to Movies

no code implementations CVPR 2017 Zhiwei Deng, Rajitha Navarathna, Peter Carr, Stephan Mandt, Yisong Yue, Iain Matthews, Greg Mori

Matrix and tensor factorization methods are often used for finding underlying low-dimensional patterns from noisy data.

Generating Long-term Trajectories Using Deep Hierarchical Networks

no code implementations NeurIPS 2016 Stephan Zheng, Yisong Yue, Patrick Lucey

We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations.

Multi-dueling Bandits with Dependent Arms

no code implementations29 Apr 2017 Yanan Sui, Vincent Zhuang, Joel W. Burdick, Yisong Yue

The dueling bandits problem is an online learning framework for learning from pairwise preference feedback, and is particularly well-suited for modeling settings that elicit subjective or implicit human feedback.

Thompson Sampling

Coordinated Multi-Agent Imitation Learning

no code implementations ICML 2017 Hoang M. Le, Yisong Yue, Peter Carr, Patrick Lucey

We study the problem of imitation learning from demonstrations of multiple coordinating agents.

Imitation Learning

Learning recurrent representations for hierarchical behavior modeling

no code implementations1 Nov 2016 Eyrun Eyjolfsdottir, Kristin Branson, Yisong Yue, Pietro Perona

We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network.

Action Detection motion prediction

A Rotation Invariant Latent Factor Model for Moveme Discovery from Static Poses

no code implementations23 Sep 2016 Matteo Ruggero Ronchi, Joon Sik Kim, Yisong Yue

We tackle the problem of learning a rotation invariant latent factor model when the training data is comprised of lower-dimensional projections of the original feature space.

Smooth Imitation Learning for Online Sequence Prediction

2 code implementations3 Jun 2016 Hoang M. Le, Andrew Kang, Yisong Yue, Peter Carr

We study the problem of smooth imitation learning for online sequence prediction, where the goal is to train a policy that can smoothly imitate demonstrated behavior in a dynamic and continuous environment in response to online, sequential context input.

Imitation Learning regression

Learning Online Smooth Predictors for Realtime Camera Planning Using Recurrent Decision Trees

no code implementations CVPR 2016 Jianhui Chen, Hoang M. Le, Peter Carr, Yisong Yue, James J. Little

We study the problem of online prediction for realtime camera planning, where the goal is to predict smooth trajectories that correctly track and frame objects of interest (e. g., players in a basketball game).

Smooth Interactive Submodular Set Cover

no code implementations NeurIPS 2015 Bryan D. He, Yisong Yue

Interactive submodular set cover is an interactive variant of submodular set cover over a hypothesis class of submodular functions, where the goal is to satisfy all sufficiently plausible submodular functions to a target threshold using as few (cost-weighted) actions as possible.

Learning Policies for Contextual Submodular Prediction

no code implementations11 May 2013 Stephane Ross, Jiaji Zhou, Yisong Yue, Debadeepta Dey, J. Andrew Bagnell

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options.

Diversity Document Summarization +2

Linear Submodular Bandits and their Application to Diversified Retrieval

no code implementations NeurIPS 2011 Yisong Yue, Carlos Guestrin

Diversified retrieval and online learning are two core research areas in the design of modern information retrieval systems. In this paper, we propose the linear submodular bandits problem, which is an online learning setting for optimizing a general class of feature-rich submodular utility models for diversified retrieval.

Information Retrieval News Recommendation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.