Search Results for author:
Found 49 papers, 21 papers with code
σ-GPTs: A New Approach to Autoregressive Models
Autoregressive models, such as the GPT family, use a fixed order, usually left-to-right, to generate sequences.
Multi-Lattice Sampling of Quantum Field Theories via Neural Operator-based Flows
In particular, we propose to approximate a time-dependent operator $\mathcal V_t$ whose time integral provides a mapping between the functional distributions of the free theory $[\mathcal D\phi(x)] \mathcal Z_0^{-1} e^{-\mathcal S_{0}[\phi(x)]}$ and of the target theory $[\mathcal D\phi(x)]\mathcal Z^{-1}e^{-\mathcal S[\phi(x)]}$.
PAUMER: Patch Pausing Transformer for Semantic Segmentation
We study the problem of improving the efficiency of segmentation transformers by using disparate amounts of computation for different parts of the image.
Faster Causal Attention Over Large Sequences Through Sparse Flash Attention
While many works have proposed schemes to sparsify the attention patterns and reduce the computational overhead of self-attention, those are often limited by implementations concerns and end up imposing a simple and static structure over the attention matrix.
Graph Neural Networks Go Forward-Forward
We present the Graph Forward-Forward (GFF) algorithm, an extension of the Forward-Forward procedure to graphs, able to handle features distributed over a graph's nodes.
Learning Interpolations between Boltzmann Densities
We introduce a training objective for continuous normalizing flows that can be used in the absence of samples but in the presence of an energy function.
ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields
We present ESLAM, an efficient implicit neural representation method for Simultaneous Localization and Mapping (SLAM).
Deformations of Boltzmann Distributions
Consider a one-parameter family of Boltzmann distributions $p_t(x) = \tfrac{1}{Z_t}e^{-S_t(x)}$.
Inference from Real-World Sparse Measurements
Notably, our model reduces the root mean square error (RMSE) for wind nowcasting from 9. 24 to 7. 98 and for heat diffusion tasks from 0. 126 to 0. 084.
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models
In Multi-Task Learning (MTL), tasks may compete and limit the performance achieved on each other, rather than guiding the optimization to a solution, superior to all its single-task trained counterparts.
Transformers are Sample-Efficient World Models
Deep reinforcement learning agents are notoriously sample inefficient, which considerably limits their application to real-world problems.
Ranked #7 on
Atari Games 100k
on Atari 100k
Flowification: Everything is a Normalizing Flow
On the other hand, neural networks only perform a forward pass on the input, there is neither a notion of an inverse of a neural network nor is there one of its likelihood contribution.
The Theoretical Expressiveness of Maxpooling
Over the decade since deep neural networks became state of the art image classifiers there has been a tendency towards less use of max pooling: the function that takes the largest of nearby pixels in an image.
MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned
With this in mind, we hosted the third edition of the MineRL ObtainDiamond competition, MineRL Diamond 2021, with a separate track in which we permitted any solution to promote the participation of newcomers.
Borrowing from yourself: Faster future video segmentation with partial channel update
Semantic segmentation is a well-addressed topic in the computer vision literature, but the design of fast and accurate video processing networks remains challenging.
Agree to Disagree: Diversity through Disagreement for Better Transferability
This behavior can hinder the transferability of trained models by (i) favoring the learning of simpler but spurious features -- present in the training data but absent from the test data -- and (ii) by only leveraging a small subset of predictive features.
Efficient Wind Speed Nowcasting with GPU-Accelerated Nearest Neighbors Algorithm
This paper proposes a simple yet efficient high-altitude wind nowcasting pipeline.
GeoNeRF: Generalizing NeRF with Geometry Priors
To render a novel view, the geometry reasoner first constructs cascaded cost volumes for each nearby source view.
Test time Adaptation through Perturbation Robustness
Data samples generated by several real world processes are dynamic in nature \textit{i. e.}, their characteristics vary with time.
Speeding up PCA with priming
This algorithm first runs any approximate-PCA method to get an initial estimate of the principal components (priming), and then applies an exact PCA in the subspace they span.
Language Models are Few-Shot Butlers
Pretrained language models demonstrate strong performance in most NLP tasks when fine-tuned on small task-specific datasets.
Structural analysis of an all-purpose question answering model
Attention is a key component of the now ubiquitous pre-trained language models.
Unsupervised clustering of series using dynamic programming and neural processes
Following the work of arXiv:2101. 09512, we are interested in clustering a given multi-variate series in an unsupervised manner.
Unsupervised clustering of series using dynamic programming
We are interested in clustering parts of a given single multi-variate series in an unsupervised manner.
Computing Preimages of Deep Neural Networks with Applications to Safety
To apply an algorithm in a sensitive domain it is important to understand the set of input values that result in specific decisions.
On the importance of pre-training data volume for compact language models
Recent advances in language modeling have led to computationally intensive and resource-demanding state-of-the-art models.
Ranked #7 on
Question Answering
on FQuAD
Fast Transformers with Clustered Attention
This results in a model with linear complexity with respect to the sequence length for a fixed number of clusters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences.
Ranked #5 on
Offline RL
on D4RL
Multi-task Reinforcement Learning with a Planning Quasi-Metric
We introduce a new reinforcement learning approach combining a planning quasi-metric (PQM) that estimates the number of steps required to go from any state to another, with task-specific "aimers" that compute a target state to reach a given goal.
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning
The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration.
Processing Megapixel Images with Deep Attention-Sampling Models
We show that sampling from the attention distribution results in an unbiased estimator of the full model with minimal variance, and we derive an unbiased estimator of the gradient that we use to train our model end-to-end with a normal SGD procedure.
Reducing Noise in GAN Training with Variance Reduced Extragradient
We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges.
Not All Samples Are Created Equal: Deep Learning with Importance Sampling
Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored.
SGAN: An Alternative Training of Generative Adversarial Networks
The Generative Adversarial Networks (GANs) have demonstrated impressive performance for data synthesis, and are now used in a wide range of computer vision tasks.
The WILDTRACK Multi-Camera Person Dataset
People detection methods are highly sensitive to the perpetual occlusions among the targets.
Biased Importance Sampling for Deep Neural Network Training
Importance sampling has been successfully used to accelerate stochastic optimization in many convex problems.
Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection
People detection in single 2D images has improved greatly in recent years.
Ranked #8 on
Multiview Detection
on MultiviewX
Deep Multi-camera People Detection
The former does not exploit joint information, whereas the latter deals with ambiguous input due to the foreground blobs becoming more and more interconnected as the number of targets increases.
Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
We present a unified framework for understanding human social behaviors in raw image sequences.
Ranked #2 on
Action Recognition
on Volleyball
Multi-Modal Mean-Fields via Cardinality-Based Clamping
Mean Field inference is central to statistical physics.
Predicting the dynamics of 2d objects with a deep residual network
We investigate how a residual network can learn to predict the dynamics of interacting shapes purely as an image-to-image regression task.
K-Medoids For K-Means Seeding
We run experiments showing that algorithm clarans (Ng et al., 2005) finds better K-medoids solutions than the Voronoi iteration algorithm.
Data Structures and Algorithms
A Sub-Quadratic Exact Medoid Algorithm
We present a new algorithm, trimed, for obtaining the medoid of a set, that is the element of the set which minimises the mean distance to all other elements.
Nested Mini-Batch K-Means
A new algorithm is proposed which accelerates the mini-batch k-means algorithm of Sculley (2010) by using the distance bounding approach of Elkan (2003).
Fast K-Means with Accurate Bounds
We propose a novel accelerated exact k-means algorithm, which performs better than the current state-of-the-art low-dimensional algorithm in 18 of 22 experiments, running up to 3 times faster.
Kullback-Leibler Proximal Variational Inference
Secondly, we use the proximal framework to derive efficient variational algorithms for non-conjugate models.
Principled Parallel Mean-Field Inference for Discrete Random Fields
Mean-field variational inference is one of the most popular approaches to inference in discrete random fields.
A provably convergent alternating minimization method for mean field inference
Mean-Field is an efficient way to approximate a posterior distribution in complex graphical models and constitutes the most popular class of Bayesian variational approximation methods.
Reservoir Boosting : Between Online and Offline Ensemble Learning
We propose to train an ensemble with the help of a reservoir in which the learning algorithm can store a limited number of samples.
