Search Results for author:
Found 35 papers, 11 papers with code
Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
On complex downstream tasks with limited data, such as fluid flow simulations and fluid-structure interactions, we found CoDA-NO to outperform existing methods on the few-shot learning task by over $36\%$.
Equivariant Graph Neural Operator for Modeling 3D Dynamics
Modeling the complex three-dimensional (3D) dynamics of relational systems is an important problem in the natural sciences, with applications ranging from molecular simulations to particle mechanics.
Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs
Our contributions are threefold: i) we enable parallelization over input samples with a novel multi-grid-based domain decomposition, ii) we represent the parameters of the model in a high-order latent subspace of the Fourier domain, through a global tensor factorization, resulting in an extreme reduction in the number of parameters and improved generalization, and iii) we propose architectural improvements to the backbone FNO.
Neural Operators for Accelerating Scientific Simulations and Design
Scientific discovery and engineering design are currently limited by the time and cost of physical experiments, selected mostly through trial-and-error and intuition that require deep domain expertise.
Speeding up Fourier Neural Operators via Mixed Precision
In this work, we (i) profile memory and runtime for FNO with full and mixed-precision training, (ii) conduct a study on the numerical stability of mixed-precision training of FNO, and (iii) devise a training routine which substantially decreases training time and memory usage (up to 34%), with little or no reduction in accuracy, on the Navier-Stokes and Darcy flow equations.
Score-based Diffusion Models in Function Space
They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate samples by denoising.
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression
Transformers have attained superior performance in natural language processing and computer vision.
Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs
Fourier Neural Operators (FNO) offer a principled approach to solving challenging partial differential equations (PDE) such as turbulent flows.
Reinforcement Learning in Factored Action Spaces using Tensor Decompositions
We present an extended abstract for the previously published work TESSERACT [Mahajan et al., 2021], which proposes a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decompositions.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Defensive Tensorization
We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network.
AugMax: Adversarial Composition of Random Augmentations for Robust Training
Diversity and hardness are two complementary dimensions of data augmentation to achieve robustness.
Tensor Methods in Computer Vision and Deep Learning
Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions.
Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning
Algorithms derived from Tesseract decompose the Q-tensor across agents and utilise low-rank tensor approximations to model agent interactions relevant to the task.
Augmenting Deep Classifiers with Polynomial Neural Networks
The efficacy of the proposed models is evaluated on standard image and audio classification benchmarks.
Ranked #2 on
Audio Classification
on Speech Commands
Unsupervised Controllable Generation with Self-Training
Instead, we propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training.
Spectral Learning on Matrices and Tensors
PCA and other spectral techniques applied to matrices have several limitations.
Toward fast and accurate human pose estimation via soft-gated skip connections
In addition, with a reduction of 3x in model size and complexity, we show no decrease in performance when compared to the original HourGlass network.
Ranked #2 on
Pose Estimation
on Leeds Sports Poses
Convolutional Tensor-Train LSTM for Spatio-temporal Learning
Learning from spatio-temporal data has numerous applications such as human-behavior analysis, object tracking, video compression, and physics simulation. However, existing methods still perform poorly on challenging video tasks such as long-term forecasting.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Speech-driven facial animation using polynomial fusion of features
Speech-driven facial animation involves using a speech signal to generate realistic videos of talking faces.
Defensive Tensorization: Randomized Tensor Parametrization for Robust Neural Networks
As deep neural networks become widely adopted for solving most problems in computer vision and audio-understanding, there are rising concerns about their potential vulnerability.
Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimation
To alleviate this, one approach is to apply low-rank tensor decompositions to convolution kernels in order to compress the network and reduce its number of parameters.
Matrix and tensor decompositions for training binary neural networks
This paper is on improving the training of binary neural networks in which both activations and weights are binary.
Incremental multi-domain learning with network latent tensor factorization
Adapting the learned classification to new domains is a hard problem due to at least three reasons: (1) the new domains and the tasks might be drastically different; (2) there might be very limited amount of annotated data on the new domain and (3) full training of a new model for each new task is prohibitive in terms of computation and memory, due to the sheer number of parameters of deep CNNs.
Improved training of binary networks for human pose estimation and image recognition
Big neural networks trained on large datasets have advanced the state-of-the-art for a large variety of challenging problems, improving performance by a large margin.
T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensor
In this paper, we propose to fully parametrize Convolutional Neural Networks (CNNs) with a single high-order, low-rank tensor.
Ranked #35 on
Pose Estimation
on MPII Human Pose
Tensor Dropout for Robust Learning
CNNs achieve remarkable performance by leveraging deep, over-parametrized architectures, trained on large datasets.
SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild
Natural human-computer interaction and audio-visual human behaviour sensing systems, which would achieve robust performance in-the-wild are more needed than ever as digital devices are increasingly becoming an indispensable part of our life.
Robust Conditional Generative Adversarial Networks
Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision.
Stochastic Activation Pruning for Robust Adversarial Defense
Neural networks are known to be vulnerable to adversarial examples.
Tensor Contraction & Regression Networks
Second, we introduce tensor regression layers, which express the output of a neural network as a low-rank multi-linear mapping from a high-order activation tensor to the softmax layer.
GAGAN: Geometry-Aware Generative Adversarial Networks
Deep generative models learned through adversarial training have become increasingly popular for their ability to generate naturalistic image textures.
Tensor Regression Networks
First, we introduce Tensor Contraction Layers (TCLs) that reduce the dimensionality of their input while preserving their multilinear structure using tensor contraction.
Tensor Contraction Layers for Parsimonious Deep Nets
Specifically, we propose the Tensor Contraction Layer (TCL), the first attempt to incorporate tensor contractions as end-to-end trainable neural network layers.
TensorLy: Tensor Learning in Python
In addition, using the deep-learning frameworks as backend allows users to easily design and train deep tensorized neural networks.
Machine Learning for Neuroimaging with Scikit-Learn
Statistical machine learning methods are increasingly used for neuroimaging data analysis.
