Search Results for author:
Found 11 papers, 2 papers with code
Bayesian Robust Tensor Factorization for Incomplete Multiway Data
We propose a generative model for robust tensor factorization in the presence of both missing data and outliers.
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs).
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
The manifold of input signals is embedded in a higher dimensional manifold of the next layer as a curved submanifold, provided the number of neurons is larger than that of inputs.
Fisher Information and Natural Gradient Learning of Random Deep Networks
The natural gradient method uses the steepest descent direction in a Riemannian manifold, so it is effective in learning, avoiding plateaus.
Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
Comparing probability distributions is a fundamental problem in data sciences.
Statistics Theory Statistics Theory 62
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks
Thus, we can conclude that batch normalization in the last layer significantly contributes to decreasing the sharpness induced by the FIM.
Pathological spectra of the Fisher information metric and its variants in deep neural networks
The Fisher information matrix (FIM) plays an essential role in statistics and machine learning as a Riemannian metric tensor or a component of the Hessian matrix of loss functions.
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective
It is known that any target function is realized in a sufficiently small neighborhood of any randomly connected deep network, provided the width (the number of neurons in a layer) is sufficiently large.
When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) often speed up optimization, their effect on generalization has been called into question.
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
In this paper, we investigate such neural fields in a multilayer architecture to investigate the supervised learning of the fields.
Information Geometry of Wasserstein Statistics on Shapes and Affine Deformations
The shape of a probability distribution and its affine deformation are separated in the Wasserstein geometry, showing its robustness against the waveform perturbation in exchange for the loss in Fisher efficiency.
