Search Results for author:
Found 22 papers, 2 papers with code
A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
For depth-2 fully-connected networks on a Euclidean space, the ridgelet transform has been discovered up to the closed-form expression, thus we could describe how the parameters are distributed.
A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guarantees
We study a primal-dual reinforcement learning (RL) algorithm for the online constrained Markov decision processes (CMDP) problem, wherein the agent explores an optimal policy that maximizes return while satisfying constraints.
Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks
We identify hidden layers inside a deep neural network (DNN) with group actions on the data domain, and formulate a formal deep network as a dual voice transform with respect to the Koopman operator, a linear representation of the group action.
Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks
The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less investigated.
LPML: LLM-Prompting Markup Language for Mathematical Reasoning
In this paper, we propose a novel framework that integrates the Chain-of-Thought (CoT) method with an external tool (Python REPL).
Koopman-based generalization bound: New aspect for full-rank weights
Our bound is tighter than existing norm-based bounds when the condition numbers of weight matrices are small.
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation
To address this problem, we develop a quantum ridgelet transform (QRT), which implements the ridgelet transform of a quantum state within a linear runtime $O(D)$ of quantum computation.
Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups
We show the universality of depth-2 group convolutional neural networks (GCNNs) in a unified and constructive manner based on the ridgelet theory.
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis
Based on the well-established framework of the Helgason-Fourier transform on the noncompact symmetric space, we present a fully-connected network and its associated ridgelet transform on the noncompact symmetric space, covering the hyperbolic neural network (HNN) and the SPDNet as special cases.
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
In this paper, we investigate such neural fields in a multilayer architecture to investigate the supervised learning of the fields.
Exponential Error Convergence in Data Classification with Optimized Random Features: Acceleration by Quantum Machine Learning
We prove that our algorithm can achieve the exponential error convergence under the low-noise condition even with optimized RFs; at the same time, our algorithm can exploit the advantage of the significant reduction of the number of features without the computational hardness owing to QML.
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space
In this paper, we present a structure theorem of the null space for a general class of neural networks.
How Powerful are Shallow Neural Networks with Bandlimited Random Weights?
Despite the well-known fact that a neural network is a universal approximator, in this study, we mathematically show that when hidden parameters are distributed in a bounded domain, the network may not achieve zero approximation error.
Ridge Regression with Over-Parametrized Two-Layer Networks Converge to Ridgelet Spectrum
We develop a new theory of ridgelet transform, a wavelet-like integral transform that provides a powerful and general framework for the theoretical study of neural networks involving not only the ReLU but general activation functions.
Learning with Optimized Random Features: Exponential Speedup by Quantum Machine Learning without Sparsity and Low-Rank Assumptions
Here, we develop a quantum algorithm for sampling from this optimized distribution over features, in runtime $O(D)$ that is linear in the dimension $D$ of the input data.
Fast Approximation and Estimation Bounds of Kernel Quadrature for Infinitely Wide Models
Kernel quadrature is a kernel-based numerical integration scheme developed for fast approximation of expectations $\int f(x) d p(x)$.
The global optimum of shallow neural network is attained by ridgelet transform
We prove that the global minimum of the backpropagation (BP) training problem of neural networks with an arbitrary nonlinear activation is given by the ridgelet transform.
Transportation analysis of denoising autoencoders: a novel method for analyzing deep neural networks
The feature map obtained from the denoising autoencoder (DAE) is investigated by determining transportation dynamics of the DAE, which is a cornerstone for deep learning.
Transport Analysis of Infinitely Deep Neural Network
Starting from the shallow DAE, this paper develops three topics: the transport map of the deep DAE, the equivalence between the stacked DAE and the composition of DAEs, and the development of the double continuum limit or the integral representation of the flow representation.
Neural Network with Unbounded Activation Functions is Universal Approximator
This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning.
Nonparametric Weight Initialization of Neural Networks via Integral Representation
A new initialization method for hidden parameters in a neural network is proposed.
