Search Results for author: Thomas George

Found 8 papers, 2 papers with code

Continual Learning in Deep Networks: an Analysis of the Last Layer

no code implementations3 Jun 2021 Timothée Lesort, Thomas George, Irina Rish

We study how different output layers in a deep neural network learn and forget in continual learning settings.

Continual Learning

NNGeometry: Easy and Fast Fisher Information Matrices and Neural Tangent Kernels in PyTorch

no code implementations1 Jan 2021 Thomas George

Fisher Information Matrices (FIM) and Neural Tangent Kernels (NTK) are useful tools in a number of diverse applications related to neural networks.

Revisiting Loss Modelling for Unstructured Pruning

1 code implementation22 Jun 2020 César Laurent, Camille Ballas, Thomas George, Nicolas Ballas, Pascal Vincent

By removing parameters from deep neural networks, unstructured pruning methods aim at cutting down memory footprint and computational cost, while maintaining prediction accuracy.

Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis

no code implementations NeurIPS 2018 Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, Pascal Vincent

Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions.

Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis

3 code implementations11 Jun 2018 Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, Pascal Vincent

Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions.

Cannot find the paper you are looking for? You can Submit a new open access paper.