Search Results for author:
Found 5 papers, 2 papers with code
A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis
In this paper, we generalize the SLTH to functions that preserve the action of the group $G$ -- i. e. $G$-equivariant network -- and prove, with high probability, that one can approximate any $G$-equivariant network of fixed width and depth by pruning a randomly initialized overparametrized $G$-equivariant network to a $G$-equivariant subnetwork.
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into $L$ simplices of $V$ dimensions each using a softmax operation.
Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
Flow-based models are powerful tools for designing probabilistic models with tractable density.
A Walk with SGD: How SGD Explores Regions of Deep Network Loss?
The non-convex nature of the loss landscape of deep neural networks (DNN) lends them the intuition that over the course of training, stochastic optimization algorithms explore different regions of the loss surface by entering and escaping many local minima due to the noise induced by mini-batches.
A Walk with SGD
Based on this and other metrics, we deduce that for most of the training update steps, SGD moves in valley like regions of the loss surface by jumping from one valley wall to another at a height above the valley floor.
