Search Results for author:
Found 23 papers, 17 papers with code
Benchmarking Audio Deepfake Detection Robustness in Real-world Communication Scenarios
We introduced ADD-C, a new test dataset to evaluate the robustness of ADD systems under diverse communication conditions, including different combinations of audio codecs for compression and Packet Loss Rates (PLR).
Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness of Smoothed Classifiers
While prior work has attempted to address this issue in adversarial robustness, the study of worst-class certified robustness for smoothed classifiers remains unexplored.
Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam
This paper comprehensively evaluates several recently proposed optimizers for 4-bit training, revealing that low-bit precision amplifies sensitivity to learning rates and often causes unstable gradient norms, leading to divergence at higher learning rates.
Enhancing Robust Fairness via Confusional Spectral Regularization
Our analysis shows that the worst-class robust error is influenced by two main factors: the spectral norm of the empirical robust confusion matrix and the information embedded in the model and training set.
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks, yet their training remains highly resource-intensive and susceptible to critical challenges such as training instability.
Are Sparse Neural Networks Better Hard Sample Learners?
These hard samples play a crucial role in the optimal performance of deep neural networks.
(PASS) Visual Prompt Locates Good Structure Sparsity through a Recurrent HyperNetwork
In this paper, we investigate a charming possibility - \textit{leveraging visual prompts to capture the channel importance and derive high-quality structural sparsity}.
Composable Interventions for Language Models
Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining.
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need
Therefore, it remains unclear whether the self-attention operation is crucial for the recent advances in SSL - or CNNs can deliver the same excellence with more advanced designs, too?
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
The rapid development of large-scale deep learning models questions the affordability of hardware platforms, which necessitates the pruning to reduce their computational and memory footprints.
Heterophily-Based Graph Neural Network for Imbalanced Classification
This leads to suboptimal performance of standard GNNs on imbalanced graphs.
Enhancing Adversarial Training via Reweighting Optimization Trajectory
Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training suffers from daunting robust overfitting, resulting in unsatisfactory robust generalization.
Are Large Kernels Better Teachers than Transformers for ConvNets?
We hereby carry out a first-of-its-kind study unveiling that modern large-kernel ConvNets, a compelling competitor to Vision Transformers, are remarkably more effective teachers for small-kernel ConvNets, due to more similar architectures.
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!
In pursuit of a more general evaluation and unveiling the true potential of sparse algorithms, we introduce "Sparsity May Cry" Benchmark (SMC-Bench), a collection of carefully-curated 4 diverse tasks with 10 datasets, that accounts for capturing a wide range of domain-specific and sophisticated knowledge.
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).
Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
We call our method Lottery Pools.
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training
Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically sparse neural networks from scratch.
Calibrated Adversarial Training
In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training.
On Generalization of Graph Autoencoders with Adversarial Training
Adversarial training is an approach for increasing model's resilience against adversarial perturbations.
Direction-Aggregated Attack for Transferable Adversarial Examples
Deep neural networks are vulnerable to adversarial examples that are crafted by imposing imperceptible changes to the inputs.
Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
Although various approaches have been proposed to solve this problem, two major limitations exist: (1) unsupervised approaches usually work much less efficiently due to the lack of supervisory signal, and (2) existing anomaly detection methods only use local contextual information to detect anomalous nodes, e. g., one- or two-hop information, but ignore the global contextual information.
Self-Supervised Anomaly Detection
Supervised Anomaly Detection
Bridging the Performance Gap between FGSM and PGD Adversarial Training
In addition, it achieves comparable performance of adversarial robustness on MNIST dataset under white-box attack, and it achieves better performance than adv. PGD under white-box attack and effectively defends the transferable adversarial attack on CIFAR-10 dataset.
ResGCN: Attention-based Deep Residual Modeling for Anomaly Detection on Attributed Networks
Effectively detecting anomalous nodes in attributed networks is crucial for the success of many real-world applications such as fraud and intrusion detection.
