Search Results for author:
Found 20 papers, 11 papers with code
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).
Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
We call our method Lottery Pools.
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Transformers have quickly shined in the computer vision world since the emergence of Vision Transformers (ViTs).
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training
Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically sparse neural networks from scratch.
Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance
Even more unconventionally, our proposed method enables directly training sparse unbalanced GANs with an extremely sparse generator from scratch.
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training
In this paper, we focus on sparse training and highlight a perhaps counter-intuitive finding, that random pruning at initialization can be quite powerful for the sparse training of modern neural networks.
Achieving Personalized Federated Learning with Sparse Local Models
To counter this issue, personalized FL (PFL) was proposed to produce dedicated local models for each individual user.
On Heterogeneously Distributed Data, Sparsity Matters
Federated learning (FL) is particularly vulnerable to heterogeneously distributed data, since a common global model in FL may not adapt to the heterogeneous data distribution of each user.
Sparse Unbalanced GAN Training with In-Time Over-Parameterization
Perhaps most importantly, we find instead of inheriting parameters from expensive pre-trained GANs, directly training sparse GANs from scratch can be a much more efficient solution.
Hierarchical Semantic Segmentation using Psychometric Learning
One of the major challenges in the supervised learning approaches is expressing and collecting the rich knowledge that experts have with respect to the meaning present in the image data.
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Our framework, FreeTickets, is defined as the ensemble of these relatively cheap sparse subnetworks.
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at initialization).
Ranked #3 on
Sparse Learning
on ImageNet
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
By starting from a random sparse network and continuously exploring sparse connectivities during training, we can perform an Over-Parameterization in the space-time manifold, closing the gap in the expressibility between sparse training and dense training.
Ranked #4 on
Sparse Learning
on ImageNet
Mitigating the effect of atmospheric turbulence on orbital angular momentum-based quantum key distribution using real-time adaptive optics with phase unwrapping
Quantum key distribution (QKD) employed orbital angular momentum (OAM) for high-dimensional encoding enhances the system security and information capacity between two communication parties.
Quantum Physics
Selfish Sparse RNN Training
Sparse neural networks have been widely applied to reduce the computational demands of training and deploying over-parameterized deep neural networks.
Topological Insights into Sparse Neural Networks
However, comparing different sparse topologies and determining how sparse topologies evolve during training, especially for the situation in which the sparse structure optimization is involved, remain as challenging open questions.
On improving deep learning generalization with adaptive sparse connectivity
Large neural networks are very successful in various tasks.
A Brain-inspired Algorithm for Training Highly Sparse Neural Networks
Concretely, by exploiting the cosine similarity metric to measure the importance of the connections, our proposed method, Cosine similarity-based and Random Topology Exploration (CTRE), evolves the topology of sparse neural networks by adding the most important connections to the network without calculating dense gradient in the backward.
Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware
Despite the success of ANNs, it is challenging to train and deploy modern ANNs on commodity hardware due to the ever-increasing model size and the unprecedented growth in the data volumes.
Intrinsically Sparse Long Short-Term Memory Networks
However, LSTMs are prone to be memory-bandwidth limited in realistic applications and need an unbearable period of training and inference time as the model size is ever-increasing.
