Search Results for author:
Found 10 papers, 0 papers with code
Geometry of EM and related iterative algorithms
The Expectation--Maximization (EM) algorithm is a simple meta-algorithm that has been used for many years as a methodology for statistical inference when there are missing measurements in the observed data or when the data is composed of observables and unobservables.
Full-Span Log-Linear Model and Fast Learning Algorithm
The FSLL model is a "highest-order" Boltzmann machine; nevertheless, we can compute the dual parameters of the model distribution, which plays important roles in exponential families, in $O(|X|\log|X|)$ time.
Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting
Even for the same target, the trained model shows some transfer and forgetting depending on the sample size of each task.
Principal component analysis for Gaussian process posteriors
This paper proposes an extension of principal component analysis for Gaussian process (GP) posteriors, denoted by GP-PCA.
Reconsidering Dependency Networks from an Information Geometry Perspective
Like Bayesian networks, the structure of a dependency network is represented by a directed graph, and each node has a conditional probability table.
Pathological spectra of the Fisher information metric and its variants in deep neural networks
The Fisher information matrix (FIM) plays an essential role in statistics and machine learning as a Riemannian metric tensor or a component of the Hessian matrix of loss functions.
On a convergence property of a geometrical algorithm for statistical manifolds
In this paper, we examine a geometrical projection algorithm for statistical inference.
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks
Thus, we can conclude that batch normalization in the last layer significantly contributes to decreasing the sharpness induced by the FIM.
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs).
A kernel method for canonical correlation analysis
Canonical correlation analysis is a technique to extract common features from a pair of multivariate data.
