Search Results for author:
Found 15 papers, 12 papers with code
Are Large Kernels Better Teachers than Transformers for ConvNets?
We hereby carry out a first-of-its-kind study unveiling that modern large-kernel ConvNets, a compelling competitor to Vision Transformers, are remarkably more effective teachers for small-kernel ConvNets, due to more similar architectures.
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!
In pursuit of a more general evaluation and unveiling the true potential of sparse algorithms, we introduce "Sparsity May Cry" Benchmark (SMC-Bench), a collection of carefully-curated 4 diverse tasks with 10 datasets, that accounts for capturing a wide range of domain-specific and sophisticated knowledge.
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).
Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
We call our method Lottery Pools.
Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
Although various approaches have been proposed to solve this problem, two major limitations exist: (1) unsupervised approaches usually work much less efficiently due to the lack of supervisory signal, and (2) existing anomaly detection methods only use local contextual information to detect anomalous nodes, e. g., one- or two-hop information, but ignore the global contextual information.
Self-Supervised Anomaly Detection
Supervised Anomaly Detection
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
The rapid development of large-scale deep learning models questions the affordability of hardware platforms, which necessitates the pruning to reduce their computational and memory footprints.
Direction-Aggregated Attack for Transferable Adversarial Examples
Deep neural networks are vulnerable to adversarial examples that are crafted by imposing imperceptible changes to the inputs.
On Generalization of Graph Autoencoders with Adversarial Training
Adversarial training is an approach for increasing model's resilience against adversarial perturbations.
Enhancing Adversarial Training via Reweighting Optimization Trajectory
Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training suffers from daunting robust overfitting, resulting in unsatisfactory robust generalization.
ResGCN: Attention-based Deep Residual Modeling for Anomaly Detection on Attributed Networks
Effectively detecting anomalous nodes in attributed networks is crucial for the success of many real-world applications such as fraud and intrusion detection.
Bridging the Performance Gap between FGSM and PGD Adversarial Training
In addition, it achieves comparable performance of adversarial robustness on MNIST dataset under white-box attack, and it achieves better performance than adv. PGD under white-box attack and effectively defends the transferable adversarial attack on CIFAR-10 dataset.
Calibrated Adversarial Training
In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training.
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training
Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically sparse neural networks from scratch.
Heterophily-Based Graph Neural Network for Imbalanced Classification
This leads to suboptimal performance of standard GNNs on imbalanced graphs.
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need
Therefore, it remains unclear whether the self-attention operation is crucial for the recent advances in SSL - or CNNs can deliver the same excellence with more advanced designs, too?
