Search Results for author:
Found 10 papers, 4 papers with code
Deep Fried Convnets
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters.
Ranked #54 on
Image Classification
on MNIST
ADASECANT: Robust Adaptive Secant Method for Stochastic Gradient
The convergence of SGD depends on the careful choice of learning rate and the amount of the noise in stochastic estimates of the gradients.
ACDC: A Structured Efficient Linear Layer
Finally, this paper also provides a connection between structured linear transforms used in deep learning and the field of Fourier optics, illustrating how ACDC could in principle be implemented with lenses and diffractive elements.
A Controller-Recognizer Framework: How necessary is recognition for control?
Recently there has been growing interest in building active visual object recognizers, as opposed to the usual passive recognizers which classifies a given static image into a predefined set of object categories.
Noisy Activation Functions
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies that are not visible to vanilla-SGD (using first order gradients only).
Mollifying Networks
The optimization of deep neural networks can be more challenging than traditional convex optimization problems due to the highly non-convex nature of the loss function, e. g. it can involve pathological landscapes such as saddle-surfaces that can be difficult to escape for algorithms based on simple gradient descent.
A Robust Adaptive Stochastic Gradient Method for Deep Learning
The information about the element-wise curvature of the loss function is estimated from the local statistics of the stochastic first order gradients.
Contingency-Aware Exploration in Reinforcement Learning
This paper investigates whether learning contingency-awareness and controllable aspects of an environment can lead to better exploration in reinforcement learning.
Ranked #8 on
Atari Games
on Atari 2600 Montezuma's Revenge
Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards
Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow.
Self-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration Tasks
We propose a new method of learning a trajectory-conditioned policy to imitate diverse trajectories from the agent's own past experiences and show that such self-imitation helps avoid myopic behavior and increases the chance of finding a globally optimal solution for hard-exploration tasks, especially when there are misleading rewards.
