Search Results for author:
Found 33 papers, 19 papers with code
Generic Deep Networks with Wavelet Scattering
We introduce a two-layer wavelet scattering network, for object classification.
Deep Roto-Translation Scattering for Object Classification
Dictionary learning algorithms or supervised deep convolution networks have considerably improved the efficiency of predefined feature representations such as SIFT.
Building a Regular Decision Boundary with Deep Networks
We show that increasing the width of our network permits being competitive with very deep networks.
Multiscale Hierarchical Convolutional Networks
Multiscale hierarchical convolutional networks are structured deep convolutional networks where layers are indexed by progressively higher dimensional attributes, which are learned from training data.
Scaling the Scattering Transform: Deep Hybrid Networks
Combining scattering networks with a modern ResNet, we achieve a single-crop top 5 error of 11. 4% on imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing only 10 layers.
Ranked #73 on
Image Classification
on STL-10
i-RevNet: Deep Invertible Networks
An analysis of i-RevNets learned representations suggests an alternative explanation for the success of deep networks by a progressive contraction and linear separation with depth.
Online Regularized Nonlinear Acceleration
Regularized nonlinear acceleration (RNA) estimates the minimum of a function by post-processing iterates from an algorithm such as the gradient method.
Nonlinear Acceleration of CNNs
The Regularized Nonlinear Acceleration (RNA) algorithm is an acceleration method capable of improving the rate of convergence of many optimization schemes such as gradient descend, SAGA or SVRG.
Scattering Networks for Hybrid Representation Learning
In particular, by working in scattering space, we achieve competitive results both for supervised and unsupervised learning tasks, while making progress towards constructing more interpretable CNNs.
Compressing the Input for CNNs with the First-Order Scattering Transform
We study the first-order scattering transform as a candidate for reducing the signal processed by a convolutional neural network (CNN).
Shallow Learning For Deep Networks
Here we use 1-hidden layer learning problems to sequentially build deep networks layer by layer, which can inherit properties from shallow networks.
On Lazy Training in Differentiable Programming
In a series of recent theoretical works, it was shown that strongly over-parameterized neural networks trained with gradient-based methods could converge exponentially fast to zero training loss, with their parameters hardly varying.
Kymatio: Scattering Transforms in Python
The wavelet scattering transform is an invariant signal representation suitable for many signal processing and machine learning applications.
Greedy Layerwise Learning Can Scale to ImageNet
Here we use 1-hidden layer learning problems to sequentially build deep networks layer by layer, which can inherit properties from shallow networks.
Decoupled Greedy Learning of CNNs
It is based on a greedy relaxation of the joint training objective, recently shown to be effective in the context of Convolutional Neural Networks (CNNs) on large-scale image classification.
Interferometric Graph Transform: a Deep Unsupervised Graph Representation
We propose the Interferometric Graph Transform (IGT), which is a new class of deep unsupervised graph convolutional neural network for building graph representations.
A spectral perspective on GCNs
In this work, we study the behavior of standard GCNs under spectral manipulations.
The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods.
A recent line of work showed that various forms of convolutional kernel methods can be competitive with standard supervised deep convolutional networks on datasets like CIFAR-10, obtaining accuracies in the range of 87-90% while being more amenable to theoretical analysis.
The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods
A recent line of work showed that various forms of convolutional kernel methods can be competitive with standard supervised deep convolutional networks on datasets like CIFAR-10, obtaining accuracies in the range of 87-90% while being more amenable to theoretical analysis.
$\texttt{DoStoVoQ}$: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning
The growing size of models and datasets have made distributed implementation of stochastic gradient descent (SGD) an active field of research.
Interferometric Graph Transform for Community Labeling
We present a new approach for learning unsupervised node representations in community graphs.
Low-Rank Projections of GCNs Laplacian
In this work, we study the behavior of standard models for community detection under spectral manipulations.
Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learning
With the use of a replay buffer we show that this approach can be extended to asynchronous settings, where modules can operate and continue to update with possibly large communication delays.
Deep Reinforcement Learning for L3 Slice Localization in Sarcopenia Assessment
In this paper, we propose a deep reinforcement learning method for accurate localization of the L3 CT slice.
Gradient Masked Averaging for Federated Learning
A major challenge in federated learning is the heterogeneity of data across client, which can degrade the performance of standard FL algorithms.
On Non-Linear operators for Geometric Deep Learning
In the context of Neural Networks defined over $\mathcal{M}$, it indicates that point-wise non-linear operators are the only universal family that commutes with any group of symmetries, and justifies their systematic use in combination with dedicated linear operators commuting with specific symmetries.
Why do tree-based models still outperform deep learning on tabular data?
While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear.
DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization
This work introduces DADAO: the first decentralized, accelerated, asynchronous, primal, first-order algorithm to minimize a sum of $L$-smooth and $\mu$-strongly convex functions distributed over a given network of size $n$.
Why do tree-based models still outperform deep learning on typical tabular data?
While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear.
Can Forward Gradient Match Backpropagation?
Forward Gradients - the idea of using directional derivatives in forward differentiation mode - have recently been shown to be utilizable for neural network training while avoiding problems generally associated with backpropagation gradient computation, such as locking and memorization requirements.
$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning
Distributed training of Deep Learning models has been critical to many recent successes in the field.
Vectorizing string entries for data processing on tables: when are larger language models better?
We study the benefits of language models in 14 analytical tasks on tables while varying the training size, as well as for a fuzzy join benchmark.
Cyclic Data Parallelism for Efficient Parallelism of Deep Neural Networks
Training large deep learning models requires parallelization techniques to scale.
