Search Results for author:
Found 9 papers, 5 papers with code
Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis
Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions.
CLICKER: A Computational LInguistics Classification Scheme for Educational Resources
We propose a classification scheme -- CLICKER for CL/NLP based on the analysis of online lectures from 77 university courses on this subject.
Implicit Regularization via Neural Feature Alignment
We approach the problem of implicit regularization in deep learning from a geometrical viewpoint.
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty
Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called lazy training regime in which the network can be well approximated by its linearization around initialization.
Revisiting Loss Modelling for Unstructured Pruning
By removing parameters from deep neural networks, unstructured pruning methods aim at cutting down memory footprint and computational cost, while maintaining prediction accuracy.
Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis
Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions.
NNGeometry: Easy and Fast Fisher Information Matrices and Neural Tangent Kernels in PyTorch
Fisher Information Matrices (FIM) and Neural Tangent Kernels (NTK) are useful tools in a number of diverse applications related to neural networks.
Continual Learning in Deep Networks: an Analysis of the Last Layer
Our analysis and results shed light on the dynamics of the output layer in continual learning scenarios and suggest a way of selecting the best type of output layer for a given scenario.
A Transfer Learning Pipeline for Educational Resource Discovery with Application in Leading Paragraph Generation
In this paper, we propose the educational resource discovery (ERD) pipeline that automates web resource discovery for novel domains.
