Search Results for author: Anima Anandkumar

Found 135 papers, 48 papers with code

Langevin Monte Carlo for Contextual Bandits

1 code implementation22 Jun 2022 Pan Xu, Hongkai Zheng, Eric Mazumdar, Kamyar Azizzadenesheli, Anima Anandkumar

Existing Thompson sampling-based algorithms need to construct a Laplace approximation (i. e., a Gaussian distribution) of the posterior distribution, which is inefficient to sample in high dimensional applications for general covariance matrices.

Multi-Armed Bandits

Thompson Sampling Achieves $\tilde O(\sqrt{T})$ Regret in Linear Quadratic Control

no code implementations17 Jun 2022 Taylan Kargin, Sahin Lale, Kamyar Azizzadenesheli, Anima Anandkumar, Babak Hassibi

By carefully prescribing an early exploration strategy and a policy update rule, we show that TS achieves order-optimal regret in adaptive control of multidimensional stabilizable LQRs.

Decision Making Decision Making Under Uncertainty

Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Bandits

no code implementations7 Jun 2022 Tianyuan Jin, Pan Xu, Xiaokui Xiao, Anima Anandkumar

We study the regret of Thompson sampling (TS) algorithms for exponential family bandits, where the reward distribution is from a one-dimensional exponential family, which covers many common reward distributions including Bernoulli, Gaussian, Gamma, Exponential, etc.

Multi-Armed Bandits

KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems

no code implementations3 Jun 2022 Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli, Adam Wierman, Anima Anandkumar

However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems.


Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions

1 code implementation CVPR 2022 Huaizu Jiang, Xiaojian Ma, Weili Nie, Zhiding Yu, Yuke Zhu, Anima Anandkumar

A significant gap remains between today's visual pattern recognition models and human-level visual cognition especially when it comes to few-shot learning and compositional reasoning of novel concepts.

Few-Shot Image Classification Human-Object Interaction Detection +2

Diffusion Models for Adversarial Purification

no code implementations16 May 2022 Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, Anima Anandkumar

Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model.

Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds

1 code implementation13 May 2022 Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, Soon-Jo Chung

Last, our control design extrapolates to unseen wind conditions, is shown to be effective for outdoor flights with only onboard sensors, and can transfer across drones with minimal performance degradation.


Generative Adversarial Neural Operators

1 code implementation6 May 2022 Md Ashiqur Rahman, Manuel A. Florez, Anima Anandkumar, Zachary E. Ross, Kamyar Azizzadenesheli

We propose the generative adversarial neural operator (GANO), a generative model paradigm for learning probabilities on infinite-dimensional function spaces.

Hyperparameter Optimization

Understanding The Robustness in Vision Transformers

1 code implementation26 Apr 2022 Daquan Zhou, Zhiding Yu, Enze Xie, Chaowei Xiao, Anima Anandkumar, Jiashi Feng, Jose M. Alvarez

Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.

Ranked #2 on Domain Generalization on ImageNet-C (using extra training data)

Domain Generalization Image Classification +3

RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning

1 code implementation ICLR 2022 Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

This task remains challenging for current deep learning algorithms since it requires addressing three key technical problems jointly: 1) identifying object entities and their properties, 2) inferring semantic relations between pairs of entities, and 3) generalizing to novel object-relation combinations, i. e., systematic generalization.

Human-Object Interaction Detection Systematic Generalization +3

M$^2$BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation

no code implementations11 Apr 2022 Enze Xie, Zhiding Yu, Daquan Zhou, Jonah Philion, Anima Anandkumar, Sanja Fidler, Ping Luo, Jose M. Alvarez

In this paper, we propose M$^2$BEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inputs.

3D Object Detection object-detection

ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation

no code implementations14 Mar 2022 Bokui Shen, Zhenyu Jiang, Christopher Choy, Leonidas J. Guibas, Silvio Savarese, Anima Anandkumar, Yuke Zhu

Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, bring substantial challenges due to infinite shape variations, non-rigid motions, and partial observability.

Contrastive Learning Deformable Object Manipulation

FreeSOLO: Learning to Segment Objects without Annotations

1 code implementation CVPR 2022 Xinlong Wang, Zhiding Yu, Shalini De Mello, Jan Kautz, Anima Anandkumar, Chunhua Shen, Jose M. Alvarez

FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9. 8% AP when fine-tuning instance segmentation with only 5% COCO masks.

Instance Segmentation object-detection +2

Pre-Trained Language Models for Interactive Decision-Making

no code implementations3 Feb 2022 Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, Yuke Zhu

The agent iteratively learns by interacting with the environment, relabeling the language goal of past 'failed' experiences, and updating the policy in a self-supervised loop.

Decision Making Imitation Learning +1

Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases

no code implementations15 Dec 2021 Shrimai Prabhumoye, Rafal Kocielnik, Mohammad Shoeybi, Anima Anandkumar, Bryan Catanzaro

We then provide the LM with instruction that consists of this subset of labeled exemplars, the query text to be classified, a definition of bias, and prompt it to make a decision.

Pretrained Language Models

CEM-GD: Cross-Entropy Method with Gradient Descent Planner for Model-Based Reinforcement Learning

1 code implementation14 Dec 2021 Kevin Huang, Sahin Lale, Ugo Rosolia, Yuanyuan Shi, Anima Anandkumar

It then uses the top trajectories as initialization for gradient descent and applies gradient updates to each of these trajectories to find the optimal action sequence.

Continuous Control Model-based Reinforcement Learning +1

Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions

no code implementations NeurIPS 2021 Jiachen Sun, Yulong Cao, Christopher B. Choy, Zhiding Yu, Anima Anandkumar, Zhuoqing Morley Mao, Chaowei Xiao

In this paper, we systematically study the impact of various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds with adversarial training.

Adversarial Robustness Autonomous Driving +1

Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers

1 code implementation24 Nov 2021 John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, Bryan Catanzaro

AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.

Operator learning Representation Learning

Polymatrix Competitive Gradient Descent

no code implementations16 Nov 2021 Jeffrey Ma, Alistair Letcher, Florian Schäfer, Yuanyuan Shi, Anima Anandkumar

In this work we propose polymatrix competitive gradient descent (PCGD) as a method for solving general sum competitive optimization involving arbitrary numbers of agents.

Multi-agent Reinforcement Learning

Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization

no code implementations15 Nov 2021 Youngwoon Lee, Joseph J. Lim, Anima Anandkumar, Yuke Zhu

However, these approaches require larger state distributions to be covered as more policies are sequenced, and thus are limited to short skill sequences.

Physics-Informed Neural Operator for Learning Partial Differential Equations

2 code implementations6 Nov 2021 Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, Anima Anandkumar

The Physics-Informed Neural Network (PINN) is an example of the former while the Fourier neural operator (FNO) is an example of the latter.

Operator learning

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds

1 code implementation NeurIPS 2021 Yujia Huang, huan zhang, Yuanyuan Shi, J Zico Kolter, Anima Anandkumar

Certified robustness is a desirable property for deep neural networks in safety-critical applications, and popular training algorithms can certify robustness of a neural network by computing a global bound on its Lipschitz constant.

ZerO Initialization: Initializing Residual Networks with only Zeros and Ones

1 code implementation25 Oct 2021 Jiawei Zhao, Florian Schäfer, Anima Anandkumar

Deep neural networks are usually initialized with random weights, with adequately selected initial variance to ensure stable signal propagation during training.

Image Classification

OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

1 code implementation2 Oct 2021 Josiah Wong, Viktor Makoviychuk, Anima Anandkumar, Yuke Zhu

Operational Space Control (OSC) has been used as an effective task-space controller for manipulation.

Stability Constrained Reinforcement Learning for Real-Time Voltage Control

no code implementations30 Sep 2021 Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, Adam Wierman

Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems.


Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators

no code implementations ICLR 2022 John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, Bryan Catanzaro

AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.

Operator learning Representation Learning

Scaling Fair Learning to Hundreds of Intersectional Groups

no code implementations29 Sep 2021 Eric Zhao, De-An Huang, Hao liu, Zhiding Yu, Anqi Liu, Olga Russakovsky, Anima Anandkumar

In real-world applications, however, there are multiple protected attributes yielding a large number of intersectional protected groups.

Fairness Knowledge Distillation

Auditing AI models for Verified Deployment under Semantic Specifications

no code implementations25 Sep 2021 Homanga Bharadhwaj, De-An Huang, Chaowei Xiao, Anima Anandkumar, Animesh Garg

We enable such unit tests through variations in a semantically-interpretable latent space of a generative model.

Face Recognition

U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow

1 code implementation3 Sep 2021 Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, Sally M. Benson

Here we present U-FNO, a novel neural network architecture for solving multiphase flow problems with superior accuracy, speed, and data efficiency.

Decision Making

Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems

no code implementations26 Aug 2021 Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, Anima Anandkumar

Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or convex quadratic regulating costs.

Neural Operator: Learning Maps Between Function Spaces

no code implementations19 Aug 2021 Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

We propose a generalization of neural networks to learn operators that maps between infinite dimensional function spaces.

Operator learning

Tensor Methods in Computer Vision and Deep Learning

no code implementations7 Jul 2021 Yannis Panagakis, Jean Kossaifi, Grigorios G. Chrysos, James Oldfield, Mihalis A. Nicolaou, Anima Anandkumar, Stefanos Zafeiriou

Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions.

Representation Learning

Long-Short Transformer: Efficient Transformers for Language and Vision

3 code implementations NeurIPS 2021 Chen Zhu, Wei Ping, Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, Bryan Catanzaro

For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.

Language Modelling

Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update

no code implementations26 Jun 2021 Jiawei Zhao, Steve Dai, Rangharajan Venkatesan, Ming-Yu Liu, Brucek Khailany, Bill Dally, Anima Anandkumar

Representing deep neural networks (DNNs) in low-precision is a promising approach to enable efficient acceleration and memory reduction.


SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

no code implementations17 Jun 2021 Linxi Fan, Guanzhi Wang, De-An Huang, Zhiding Yu, Li Fei-Fei, Yuke Zhu, Anima Anandkumar

A student network then learns to mimic the expert policy by supervised learning with strong augmentations, making its representation more robust against visual variations compared to the expert.

Autonomous Driving Image Augmentation +1

Markov Neural Operators for Learning Chaotic Systems

no code implementations13 Jun 2021 Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

Experiments show neural operators are more accurate and stable compared to previous methods on chaotic systems such as the Kuramoto-Sivashinsky and Navier-Stokes equations.

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

10 code implementations NeurIPS 2021 Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, Ping Luo

We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders.

Semantic Segmentation Thermal Image Segmentation

Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry

no code implementations31 May 2021 Zhuoran Qiao, Anders S. Christensen, Matthew Welborn, Frederick R. Manby, Anima Anandkumar, Thomas F. Miller III

Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials.

Stable Online Control of Linear Time-Varying Systems

no code implementations29 Apr 2021 Guannan Qu, Yuanyuan Shi, Sahin Lale, Anima Anandkumar, Adam Wierman

In this work, we propose an efficient online control algorithm, COvariance Constrained Online Linear Quadratic (COCO-LQ) control, that guarantees input-to-state stability for a large class of LTV systems while also minimizing the control cost.

Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection

1 code implementation12 Apr 2021 Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Anima Anandkumar, Sanja Fidler, Jose M. Alvarez

As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level.

Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles

no code implementations12 Mar 2021 Zahra Ghodsi, Siva Kumar Sastry Hari, Iuri Frosio, Timothy Tsai, Alejandro Troccoli, Stephen W. Keckler, Siddharth Garg, Anima Anandkumar

Extracting interesting scenarios from real-world data as well as generating failure cases is important for the development and testing of autonomous systems.

Autonomous Vehicles

Dynamic Social Media Monitoring for Fast-Evolving Online Discussions

no code implementations24 Feb 2021 Maya Srikanth, Anqi Liu, Nicholas Adams-Cohen, Jian Cao, R. Michael Alvarez, Anima Anandkumar

However, collecting social media data using a static set of keywords fails to satisfy the growing need to monitor dynamic conversations and to study fast-changing topics.

Decision Making Time Series

Transferable Unsupervised Robust Representation Learning

no code implementations1 Jan 2021 De-An Huang, Zhiding Yu, Anima Anandkumar

We upend this view and show that URRL improves both the natural accuracy of unsupervised representation learning and its robustness to corruptions and adversarial noise.

Data Augmentation Representation Learning +1

A Coach-Player Framework for Dynamic Team Composition

no code implementations1 Jan 2021 Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Anima Anandkumar

The performance of our method is comparable or even better than the setting where all players have a full view of the environment, but no coach.

Stability and Identification of Random Asynchronous Linear Time-Invariant Systems

no code implementations8 Dec 2020 Sahin Lale, Oguzhan Teke, Babak Hassibi, Anima Anandkumar

In this model, each state variable is updated randomly and asynchronously with some probability according to the underlying system dynamics.

Distributionally Robust Learning for Uncertainty Calibration under Domain Shift

no code implementations8 Oct 2020 Haoxuan Wang, Anqi Liu, Zhiding Yu, Junchi Yan, Yisong Yue, Anima Anandkumar

We detect such domain shifts through the use of a binary domain classifier and integrate it with the task network and train them jointly end-to-end.

Density Ratio Estimation Unsupervised Domain Adaptation

Distributionally Robust Learning for Unsupervised Domain Adaptation

no code implementations28 Sep 2020 Haoxuan Wang, Anqi Liu, Zhiding Yu, Yisong Yue, Anima Anandkumar

This formulation motivates the use of two neural networks that are jointly trained --- a discriminative network between the source and target domains for density-ratio estimation, in addition to the standard classification network.

Density Ratio Estimation Unsupervised Domain Adaptation

OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation

1 code implementation17 Aug 2020 Hongyu Ren, Yuke Zhu, Jure Leskovec, Anima Anandkumar, Animesh Garg

We propose a variational inference framework OCEAN to perform online task inference for compositional tasks.

Variational Inference

Unsupervised Controllable Generation with Self-Training

no code implementations17 Jul 2020 Grigorios G. Chrysos, Jean Kossaifi, Zhiding Yu, Anima Anandkumar

Instead, we propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training.


Neural Networks with Recurrent Generative Feedback

1 code implementation NeurIPS 2020 Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Y. Tsao, Anima Anandkumar

This mechanism can be interpreted as a form of self-consistency between the maximum a posteriori (MAP) estimation of an internal generative model and the external environment.

Adversarial Robustness

Deep Bayesian Quadrature Policy Optimization

1 code implementation28 Jun 2020 Akella Ravi Tej, Kamyar Azizzadenesheli, Mohammad Ghavamzadeh, Anima Anandkumar, Yisong Yue

On the other hand, more sample efficient alternatives like Bayesian quadrature methods have received little attention due to their high computational complexity.

Continuous Control Policy Gradient Methods

Learning compositional functions via multiplicative weight updates

1 code implementation NeurIPS 2020 Jeremy Bernstein, Jia-Wei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, Yisong Yue

This paper proves that multiplicative weight updates satisfy a descent lemma tailored to compositional functions.

Competitive Policy Optimization

2 code implementations18 Jun 2020 Manish Prajapat, Kamyar Azizzadenesheli, Alexander Liniger, Yisong Yue, Anima Anandkumar

A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties.

Policy Gradient Methods

Competitive Mirror Descent

3 code implementations17 Jun 2020 Florian Schäfer, Anima Anandkumar, Houman Owhadi

Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential.

Multipole Graph Neural Operator for Parametric Partial Differential Equations

2 code implementations NeurIPS 2020 Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks.

Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

no code implementations9 May 2020 Yashwanth Kumar Nakka, Anqi Liu, Guanya Shi, Anima Anandkumar, Yisong Yue, Soon-Jo Chung

The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints.

Motion Planning Optimal Motion Planning +1

MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

1 code implementation1 May 2020 Chiyu Max Jiang, Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, Karthik Kashinath, Mustafa Mustafa, Hamdi A. Tchelepi, Philip Marcus, Prabhat, Anima Anandkumar

We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs.


Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting

no code implementations12 Mar 2020 Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, Anima Anandkumar

We study the problem of adaptive control in partially observable linear quadratic Gaussian control systems, where the model dynamics are unknown a priori.


Neural Operator: Graph Kernel Network for Partial Differential Equations

3 code implementations ICLR Workshop DeepDiffEq 2019 Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

The classical development of neural networks has been primarily for mappings between a finite-dimensional Euclidean space and a set of classes, or between two finite-dimensional Euclidean spaces.

Regret Minimization in Partially Observable Linear Quadratic Control

no code implementations31 Jan 2020 Sahin Lale, Kamyar Azizzadenesheli, Babak Hassibi, Anima Anandkumar

We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control.

InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers

no code implementations9 Dec 2019 Tan M. Nguyen, Animesh Garg, Richard G. Baraniuk, Anima Anandkumar

Continuous Normalizing Flows (CNFs) have emerged as promising deep generative models for a wide range of tasks thanks to their invertibility and exact likelihood estimation.

Conditional Image Generation Time Series

Triply Robust Off-Policy Evaluation

no code implementations13 Nov 2019 Anqi Liu, Hao liu, Anima Anandkumar, Yisong Yue

Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method.

Multi-Armed Bandits

Implicit competitive regularization in GANs

2 code implementations ICML 2020 Florian Schäfer, Hongkai Zheng, Anima Anandkumar

We show that opponent-aware modelling of generator and discriminator, as present in competitive gradient descent (CGD), can significantly strengthen ICR and thus stabilize GAN training without explicit regularization.

Image Generation

InfoCNF: Efficient Conditional Continuous Normalizing Flow Using Adaptive Solvers

no code implementations25 Sep 2019 Tan M. Nguyen, Animesh Garg, Richard G. Baraniuk, Anima Anandkumar

Continuous Normalizing Flows (CNFs) have emerged as promising deep generative models for a wide range of tasks thanks to their invertibility and exact likelihood estimation.

Conditional Image Generation Time Series

Multi Sense Embeddings from Topic Models

no code implementations WS 2019 Shobhit Jain, Sravan Babu Bodapati, Ramesh Nallapati, Anima Anandkumar

Distributed word embeddings have yielded state-of-the-art performance in many NLP tasks, mainly due to their success in capturing useful semantic information.

Topic Models Word Embeddings +1

Out-of-Distribution Detection Using Neural Rendering Generative Models

no code implementations10 Jul 2019 Yujia Huang, Sihui Dai, Tan Nguyen, Richard G. Baraniuk, Anima Anandkumar

Our results show that when trained on CIFAR-10, lower likelihood (of latent variables) is assigned to SVHN images.

Neural Rendering OOD Detection +1

Directivity Modes of Earthquake Populations with Unsupervised Learning

no code implementations30 Jun 2019 Zachary E. Ross, Daniel T. Trugman, Kamyar Azizzadenesheli, Anima Anandkumar

A seismic spectral decomposition technique is used to first produce relative measurements of radiated energy for earthquakes in a spatially-compact cluster.

Learning Causal State Representations of Partially Observable Environments

no code implementations25 Jun 2019 Amy Zhang, Zachary C. Lipton, Luis Pineda, Kamyar Azizzadenesheli, Anima Anandkumar, Laurent Itti, Joelle Pineau, Tommaso Furlanello

In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP).

Causal Inference

Robust Regression for Safe Exploration in Control

no code implementations L4DC 2020 Anqi Liu, Guanya Shi, Soon-Jo Chung, Anima Anandkumar, Yisong Yue

To address this challenge, we present a deep robust regression model that is trained to directly predict the uncertainty bounds for safe exploration.

Generalization Bounds Safe Exploration

Competitive Gradient Descent

7 code implementations NeurIPS 2019 Florian Schäfer, Anima Anandkumar

We introduce a new algorithm for the numerical computation of Nash equilibria of competitive two-player games.

Neural Rendering Model: Joint Generation and Prediction for Semi-Supervised Learning

no code implementations ICLR 2019 Nhat Ho, Tan Nguyen, Ankit B. Patel, Anima Anandkumar, Michael. I. Jordan, Richard G. Baraniuk

The conjugate prior yields a new regularizer for learning based on the paths rendered in the generative model for training CNNs–the Rendering Path Normalization (RPN).

Neural Rendering

Tensor Dropout for Robust Learning

no code implementations27 Feb 2019 Arinbjörn Kolbeinsson, Jean Kossaifi, Yannis Panagakis, Adrian Bulat, Anima Anandkumar, Ioanna Tzoulaki, Paul Matthews

CNNs achieve remarkable performance by leveraging deep, over-parametrized architectures, trained on large datasets.

Image Classification Inductive Bias

Stochastic Linear Bandits with Hidden Low Rank Structure

no code implementations28 Jan 2019 Sahin Lale, Kamyar Azizzadenesheli, Anima Anandkumar, Babak Hassibi

We modify the image classification task into the SLB setting and empirically show that, when a pre-trained DNN provides the high dimensional feature representations, deploying PSLB results in significant reduction of regret and faster convergence to an accurate model compared to state-of-art algorithm.

Decision Making Dimensionality Reduction +1

A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model

no code implementations1 Nov 2018 Tan Nguyen, Nhat Ho, Ankit Patel, Anima Anandkumar, Michael. I. Jordan, Richard G. Baraniuk

This conjugate prior yields a new regularizer based on paths rendered in the generative model for training CNNs-the Rendering Path Normalization (RPN).

Open Vocabulary Learning on Source Code with a Graph-Structured Cache

3 code implementations ICLR 2019 Milan Cvitkovic, Badal Singh, Anima Anandkumar

Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques.

Code Completion Natural Language Processing

signSGD with Majority Vote is Communication Efficient And Fault Tolerant

3 code implementations ICLR 2019 Jeremy Bernstein, Jia-Wei Zhao, Kamyar Azizzadenesheli, Anima Anandkumar

Workers transmit only the sign of their gradient vector to a server, and the overall update is decided by a majority vote.

Probabilistic FastText for Multi-Sense Word Embeddings

1 code implementation ACL 2018 Ben Athiwaratkun, Andrew Gordon Wilson, Anima Anandkumar

We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information.

Word Embeddings Word Similarity

Born Again Neural Networks

1 code implementation ICML 2018 Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, Anima Anandkumar

Knowledge distillation (KD) consists of transferring knowledge from one machine learning model (the teacher}) to another (the student).

Knowledge Distillation

Active Learning with Partial Feedback

1 code implementation ICLR 2019 Peiyun Hu, Zachary C. Lipton, Anima Anandkumar, Deva Ramanan

While many active learning papers assume that the learner can simply ask for a label and receive it, real annotation often presents a mismatch between the form of a label (say, one among many classes), and the form of an annotation (typically yes/no binary feedback).

Active Learning

signSGD: Compressed Optimisation for Non-Convex Problems

3 code implementations ICML 2018 Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, Anima Anandkumar

Using a theorem by Gauss we prove that majority vote can achieve the same reduction in variance as full precision distributed SGD.

Tensor Contraction & Regression Networks

no code implementations ICLR 2018 Jean Kossaifi, Zack Chase Lipton, Aran Khanna, Tommaso Furlanello, Anima Anandkumar

Second, we introduce tensor regression layers, which express the output of a neural network as a low-rank multi-linear mapping from a high-order activation tensor to the softmax layer.

Long-term Forecasting using Tensor-Train RNNs

no code implementations ICLR 2018 Rose Yu, Stephan Zheng, Anima Anandkumar, Yisong Yue

We present Tensor-Train RNN (TT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.

Learning From Noisy Singly-labeled Data

1 code implementation ICLR 2018 Ashish Khetan, Zachary C. Lipton, Anima Anandkumar

We propose a new algorithm for jointly modeling labels and worker quality from noisy crowd-sourced data.

StrassenNets: Deep Learning with a Multiplication Budget

1 code implementation ICML 2018 Michael Tschannen, Aran Khanna, Anima Anandkumar

A large fraction of the arithmetic operations required to evaluate deep neural networks (DNNs) consists of matrix multiplications, in both convolution and fully connected layers.

Knowledge Distillation Language Modelling +1

Long-term Forecasting using Higher Order Tensor RNNs

1 code implementation ICLR 2018 Rose Yu, Stephan Zheng, Anima Anandkumar, Yisong Yue

We present Higher-Order Tensor RNN (HOT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics.

Time Series

Tensor Regression Networks

no code implementations26 Jul 2017 Jean Kossaifi, Zachary C. Lipton, Arinbjorn Kolbeinsson, Aran Khanna, Tommaso Furlanello, Anima Anandkumar

First, we introduce Tensor Contraction Layers (TCLs) that reduce the dimensionality of their input while preserving their multilinear structure using tensor contraction.

Compact Tensor Pooling for Visual Question Answering

no code implementations20 Jun 2017 Yang Shi, Tommaso Furlanello, Anima Anandkumar

Performing high level cognitive tasks requires the integration of feature maps with drastically different structure.

Question Answering Visual Question Answering +1

Tensor Contraction Layers for Parsimonious Deep Nets

no code implementations1 Jun 2017 Jean Kossaifi, Aran Khanna, Zachary C. Lipton, Tommaso Furlanello, Anima Anandkumar

Specifically, we propose the Tensor Contraction Layer (TCL), the first attempt to incorporate tensor contractions as end-to-end trainable neural network layers.

Model Compression

TensorLy: Tensor Learning in Python

1 code implementation29 Oct 2016 Jean Kossaifi, Yannis Panagakis, Anima Anandkumar, Maja Pantic

In addition, using the deep-learning frameworks as backend allows users to easily design and train deep tensorized neural networks.

Homotopy Analysis for Tensor PCA

no code implementations28 Oct 2016 Anima Anandkumar, Yuan Deng, Rong Ge, Hossein Mobahi

For the challenging problem of tensor PCA, we prove global convergence of the homotopy method in the "high noise" regime.

Training Input-Output Recurrent Neural Networks through Spectral Methods

no code implementations3 Mar 2016 Hanie Sedghi, Anima Anandkumar

We consider the problem of training input-output recurrent neural networks (RNN) for sequence labeling tasks.


Efficient approaches for escaping higher order saddle points in non-convex optimization

no code implementations18 Feb 2016 Anima Anandkumar, Rong Ge

Local search heuristics for non-convex optimizations are popular in applied machine learning.

Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods

no code implementations28 Jun 2015 Majid Janzamin, Hanie Sedghi, Anima Anandkumar

We propose a novel algorithm based on tensor decomposition for guaranteed training of two-layer neural networks.

Tensor Decomposition

A Scale Mixture Perspective of Multiplicative Noise in Neural Networks

no code implementations10 Jun 2015 Eric Nalisnick, Anima Anandkumar, Padhraic Smyth

Corrupting the input and hidden layers of deep neural networks (DNNs) with multiplicative noise, often drawn from the Bernoulli distribution (or 'dropout'), provides regularization that has significantly contributed to deep learning's success.

Model Compression

Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models

no code implementations2 May 2015 Tejaswi Nimmagadda, Anima Anandkumar

We incorporate contextual information in natural images through a conditional latent tree probabilistic model (CLTM), where the object co-occurrences are conditioned on the extracted fc7 features from pre-trained Imagenet CNN as input.

Classification General Classification +2

Score Function Features for Discriminative Learning

no code implementations19 Dec 2014 Majid Janzamin, Hanie Sedghi, Anima Anandkumar

In this paper, we consider a novel class of matrix and tensor-valued features, which can be pre-trained using unlabeled samples.

Natural Language Processing

Score Function Features for Discriminative Learning: Matrix and Tensor Framework

no code implementations9 Dec 2014 Majid Janzamin, Hanie Sedghi, Anima Anandkumar

In this paper, we consider a novel class of matrix and tensor-valued features, which can be pre-trained using unlabeled samples.

Natural Language Processing

Provable Tensor Methods for Learning Mixtures of Generalized Linear Models

no code implementations9 Dec 2014 Hanie Sedghi, Majid Janzamin, Anima Anandkumar

In contrast, we present a tensor decomposition method which is guaranteed to correctly recover the parameters.

General Classification Tensor Decomposition

Provable Methods for Training Neural Networks with Sparse Connectivity

no code implementations8 Dec 2014 Hanie Sedghi, Anima Anandkumar

We provide novel guaranteed approaches for training feedforward neural networks with sparse connectivity.

Multi-Step Stochastic ADMM in High Dimensions: Applications to Sparse Optimization and Matrix Decomposition

no code implementations NeurIPS 2014 Hanie Sedghi, Anima Anandkumar, Edmond Jonckheere

We first analyze the simple setting, where the optimization problem consists of a loss function and a single regularizer (e. g. sparse optimization), and then extend to the multi-block setting with multiple regularizers and multiple variables (e. g. matrix decomposition into sparse and low rank components).

Analyzing Tensor Power Method Dynamics in Overcomplete Regime

no code implementations6 Nov 2014 Anima Anandkumar, Rong Ge, Majid Janzamin

We present a novel analysis of the dynamics of tensor power iterations in the overcomplete regime where the tensor CP rank is larger than the input dimension.

Guaranteed Scalable Learning of Latent Tree Models

no code implementations18 Jun 2014 Furong Huang, Niranjan U. N., Ioakeim Perros, Robert Chen, Jimeng Sun, Anima Anandkumar

We present an integrated approach for structure and parameter estimation in latent tree graphical models.

Multi-Step Stochastic ADMM in High Dimensions: Applications to Sparse Optimization and Noisy Matrix Decomposition

2 code implementations NeurIPS 2014 Hanie Sedghi, Anima Anandkumar, Edmond Jonckheere

For sparse optimization, we establish that the modified ADMM method has an optimal convergence rate of $\mathcal{O}(s\log d/T)$, where $s$ is the sparsity level, $d$ is the data dimension and $T$ is the number of steps.

A Tensor Approach to Learning Mixed Membership Community Models

no code implementations12 Feb 2013 Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade

We provide guaranteed recovery of community memberships and model parameters and present a careful finite sample analysis of our learning method.

Community Detection Stochastic Block Model

Learning Mixtures of Tree Graphical Models

no code implementations NeurIPS 2012 Anima Anandkumar, Daniel J. Hsu, Furong Huang, Sham M. Kakade

We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable is hidden and each mixture component can have a potentially different Markov graph structure and parameters over the observed variables.

A Spectral Algorithm for Latent Dirichlet Allocation

no code implementations NeurIPS 2012 Anima Anandkumar, Dean P. Foster, Daniel J. Hsu, Sham M. Kakade, Yi-Kai Liu

This work provides a simple and efficient learning procedure that is guaranteed to recover the parameters for a wide class of topic models, including Latent Dirichlet Allocation (LDA).

Topic Models

Tensor decompositions for learning latent variable models

no code implementations29 Oct 2012 Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, Matus Telgarsky

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order).

Cannot find the paper you are looking for? You can Submit a new open access paper.