Search Results for author:
Found 14 papers, 3 papers with code
Toward High-Performance Energy and Power Battery Cells with Machine Learning-based Optimization of Electrode Manufacturing
This ML pipeline allows the inverse design of the process parameters to adopt in order to manufacture electrodes for energy or power applications.
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning
In this paper, we first present an explanation regarding the common occurrence of spikes in the training loss when neural networks are trained with stochastic gradient descent (SGD).
On Emergence of Clean-Priority Learning in Early Stopped Neural Networks
This behaviour is believed to be a result of neural networks learning the pattern of clean data first and fitting the noise later in the training, a phenomenon that we refer to as clean-priority learning.
ReLU soothes the NTK condition number and accelerates optimization for wide neural networks
Comparing with linear neural networks, we show that a ReLU activated wide neural network at random initialization has a larger angle separation for similar data in the feature space of model gradient, and has a smaller condition number for NTK.
Quadratic models for understanding neural network dynamics
While neural networks can be approximated by linear models as their width increases, certain properties of wide neural networks cannot be captured by linear models.
Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture
In this paper we show that feedforward neural networks corresponding to arbitrary directed acyclic graphs undergo transition to linearity as their "width" approaches infinity.
Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models
Wide neural networks with linear output layer have been shown to be near-linear, and to have near-constant neural tangent kernel (NTK), in a region containing the optimization path of gradient descent.
Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization
Hyper-parameter optimization is a crucial problem in machine learning as it aims to achieve the state-of-the-art performance in any model.
On the linearity of large non-linear models: when and why the tangent kernel is constant
We show that the transition to linearity of the model and, equivalently, constancy of the (neural) tangent kernel (NTK) result from the scaling properties of the norm of the Hessian matrix of the network as a function of the network width.
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
The success of deep learning is due, to a large extent, to the remarkable effectiveness of gradient-based optimization methods applied to large neural networks.
Accelerating SGD with momentum for over-parameterized learning
This is in contrast to the classical results in the deterministic scenario, where the same step size ensures accelerated convergence of the Nesterov's method over optimal gradient descent.
Parametrized Accelerated Methods Free of Condition Number
Analyses of accelerated (momentum-based) gradient descent usually assume bounded condition number to obtain exponential convergence rates.
Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI
Each network is trained using images of a single modality in a weakly-supervised manner by providing a set of prostate images with image-level labels indicating only the presence of PCa without priors of lesions’ locations.
Clustering with Bregman Divergences: an Asymptotic Analysis
Clustering, in particular $k$-means clustering, is a central topic in data analysis.
