Search Results for author:
Found 118 papers, 64 papers with code
Projection Head is Secretly an Information Bottleneck
In this paper, we develop an in-depth theoretical understanding of the projection head from the information-theoretic perspective.
When More is Less: Understanding Chain-of-Thought Length in LLMs
We theoretically prove the existence of an optimal CoT length and derive a scaling law for this optimal length based on model capability and task difficulty.
Three-dimensional signal processing: a new approach in dynamical sampling via tensor products
Multidimensional signal recovery has also been studied, but primarily in scenarios where the driving operator is a convolution operator.
SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation
As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e. g., LoRA) will become essential due to their efficiency in reducing computation costs.
Understanding Difficult-to-learn Examples in Contrastive Learning: A Theoretical Framework for Spectral Contrastive Learning
Unsupervised contrastive learning has shown significant performance improvements in recent years, often approaching or even rivaling supervised learning in various tasks.
An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise
In particular, we for the first time derive a general robust condition for arbitrary contrastive losses, which serves as a criterion to verify the theoretical robustness of a supervised contrastive loss against label noise.
MADE: Graph Backdoor Defense with Masked Unlearning
Graph Neural Networks (GNNs) have garnered significant attention from researchers due to their outstanding performance in handling graph-related tasks, such as social network analysis, protein design, and so on.
Understanding the Role of Equivariance in Self-supervised Learning
Given this limitation, there has been a surge of interest in equivariant self-supervised learning (E-SSL) that learns features to be augmentation-aware.
Dissecting the Failure of Invariant Learning on Graphs
Enhancing node-level Out-Of-Distribution (OOD) generalization on graphs remains a crucial area of research.
What is Wrong with Perplexity for Long-context Language Modeling?
To address this, we propose \textbf{LongPPL}, a novel metric that focuses on key tokens by employing a long-short context contrastive method to identify them.
Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness
In this work, we challenge the prevailing belief of the accuracy-interpretability tradeoff, showing that monosemantic features not only enhance interpretability but also bring concrete gains in model performance.
Can In-context Learning Really Generalize to Out-of-distribution Tasks?
To achieve this, we conduct synthetic experiments where the objective is to learn OOD mathematical functions through ICL using a GPT-2 model.
On the Adversarial Transferability of Generalized "Skip Connections"
Specifically, in ResNet-like models (with skip connections), we find that using more gradients from the skip connections rather than the residual modules according to a decay factor during backpropagation allows one to craft adversarial examples with high transferability.
AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation
This paper studies the vulnerabilities of transformer-based Large Language Models (LLMs) to jailbreaking attacks, focusing specifically on the optimization-based Greedy Coordinate Gradient (GCG) strategy.
Incorporating Arbitrary Matrix Group Equivariance into KANs
Kolmogorov-Arnold Networks (KANs) have seen great success in scientific domains thanks to spline activation functions, becoming an alternative to Multi-Layer Perceptrons (MLPs).
TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors
Additionally, with the reversed trigger, we propose backdoor detection from the noise space, introducing the first backdoor input detection approach for diffusion models and a novel model detection algorithm that calculates the KL divergence between reversed and benign distributions.
CycleHOI: Improving Human-Object Interaction Detection with Cycle Consistency of Detection and Generation
Recognition and generation are two fundamental tasks in computer vision, which are often investigated separately in the exiting literature.
On the Role of Discrete Tokenization in Visual Representation Learning
Furthermore, we propose a novel metric named TCAS, which is specifically designed to assess the effectiveness of discrete tokens within the MIM framework.
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
In recent years, the rise of generative self-supervised learning (SSL) paradigms has exhibited impressive performance across visual, language, and multi-modal domains.
PID: Prompt-Independent Data Protection Against Latent Diffusion Models
Drawing on these insights, we propose a simple yet effective method called \textbf{Prompt-Independent Defense (PID)} to safeguard privacy against LDMs.
A Theoretical Understanding of Self-Correction through In-context Alignment
Going beyond mimicking limited human experiences, recent studies show initial evidence that, like humans, large language models (LLMs) are capable of improving their abilities purely by self-correction, i. e., correcting previous responses through self-examination, in certain circumstances.
A Canonicalization Perspective on Invariant and Equivariant Learning
In this work, we introduce a canonicalization perspective that provides an essential and complete view of the design of frames.
How to Craft Backdoors with Unlabeled Data Alone?
Relying only on unlabeled data, Self-supervised learning (SSL) can learn rich features in an economical and scalable way.
FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs
Despite the remarkable performance of video-based large language models (LLMs), their adversarial threat remains unexplored.
Non-negative Contrastive Learning
In this paper, we propose Non-negative Contrastive Learning (NCL), a renaissance of Non-negative Matrix Factorization (NMF) aimed at deriving interpretable features.
Do Generated Data Always Help Contrastive Learning?
Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augmentations.
Fight Back Against Jailbreaking via Prompt Adversarial Tuning
To achieve our defense goal whilst maintaining natural performance, we optimize the control prompt with both adversarial and benign prompts.
Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning
With the prosperity of contrastive learning for visual representation learning (VCL), it is also adapted to the graph domain and yields promising performance.
Adversarial Examples Are Not Real Features
A well-known theory by \citet{ilyas2019adversarial} explains adversarial vulnerability from a data perspective by showing that one can extract non-robust features from adversarial examples and these features alone are useful for classification.
Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding
However, from a theoretical perspective, the universal expressive power of spectral embedding comes at the price of losing two important invariance properties of graphs, sign and basis invariance, which also limits its effectiveness on graph data.
Towards Control-Centric Representations in Reinforcement Learning from Images
Image-based Reinforcement Learning is a practical yet challenging task.
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations
Large Language Models (LLMs) have shown remarkable success in various tasks, yet their safety and the risk of generating harmful content remain pressing concerns.
Robust Long-Tailed Learning via Label-Aware Bounded CVaR
Data in the real-world classification problems are always imbalanced or long-tailed, wherein the majority classes have the most of the samples that dominate the model training.
Rethinking Weak Supervision in Helping Contrastive Learning
Therefore, to explore the mechanical differences between semi-supervised and noisy-labeled information in helping contrastive learning, we establish a unified theoretical framework of contrastive learning under weak supervision.
On the Generalization of Multi-modal Contrastive Learning
Multi-modal contrastive learning (MMCL) has recently garnered considerable interest due to its superior performance in visual tasks, achieved by embedding multi-modal data, such as visual-language pairs.
CFA: Class-wise Calibrated Fair Adversarial Training
Adversarial training has been widely acknowledged as the most effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs).
Generalist: Decoupling Natural and Robust Generalization
The parameters of base learners are collected and combined to form a global learner at intervals during the training process.
ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond
Instead of characterizing oversmoothing from the view of complete collapse in which representations converge to a single point, we dive into a more general perspective of dimensional collapse in which representations lie in a narrow cone.
Ranked #8 on
Node Property Prediction
on ogbn-arxiv
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics.
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism
In this work, we propose a unified theoretical understanding for existing variants of non-contrastive learning.
Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning
Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap.
ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
Moreover, we show that contrastive learning fails to learn domain-invariant features, which limits its transferability.
Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks
In particular, our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.
SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
In this paper, we study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the recently proposed training method, implicit differentiation on the equilibrium state (IDE), for supervised learning with purely spike-based computation, which demonstrates the potential for energy-efficient training of SNNs.
On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders
Masked Autoencoders (MAE) based on a reconstruction task have risen to be a promising paradigm for self-supervised learning (SSL) and achieve state-of-the-art performance across different benchmark datasets.
When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs.
Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance.
Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferability
This paper summarizes the common mechanism shared by twelve previous transferability-boosting methods in a unified view, i. e., these methods all reduce game-theoretic interactions between regional adversarial perturbations.
GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling
Semantic segmentation of point clouds, aiming to assign each point a semantic category, is critical to 3D scene understanding. Despite of significant advances in recent years, most of existing methods still suffer from either the object-level misclassification or the boundary-level ambiguity.
Optimization-Induced Graph Implicit Nonlinear Diffusion
Moreover, we show that the optimization-induced variants of our models can boost the performance and improve training stability and efficiency as well.
Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation
In this paper, we propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance that is competitive to ANNs yet with low latency.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap
Our theory suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and the overlapped augmented views (i. e., the chaos) create a ladder for contrastive learning to gradually learn class-separated representations.
A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training
On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM.
Self-Ensemble Adversarial Training for Improved Robustness
In this work, we are dedicated to the weight states of models through the training process and devise a simple but powerful \emph{Self-Ensemble Adversarial Training} (SEAT) method for yielding a robust classifier by averaging weights of history models.
On the Convergence and Robustness of Adversarial Training
In this paper, we propose such a criterion, namely First-Order Stationary Condition for constrained optimization (FOSC), to quantitatively evaluate the convergence quality of adversarial examples found in the inner maximization.
Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Efficient Equivariant Network
Group equivariant CNNs (G-CNNs) that incorporate more equivariance can significantly improve the performance of conventional CNNs.
Morié Attack (MA): A New Potential Risk of Screen Photos
Images, captured by a camera, play a critical role in training Deep Neural Networks (DNNs).
Gauge Equivariant Transformer
Attention mechanism has shown great performance and efficiency in a lot of deep learning models, in which relative position encoding plays a crucial role.
Clustering Effect of Adversarial Robust Models
Adversarial robustness has received increasing attention along with the study of adversarial examples.
Clustering Effect of (Linearized) Adversarial Robust Models
Adversarial robustness has received increasing attention along with the study of adversarial examples.
Fooling Adversarial Training with Inducing Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks
In this paper, we propose a novel geometric-based approach called Tangent Attack (TA), which identifies an optimal tangent point of a virtual hemisphere located on the decision boundary to reduce the distortion of the attack.
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i. e.} the view of multi-order interactions between input variables of DNNs.
Residual Relaxation for Multi-view Representation Learning
Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation.
Adversarial Neuron Pruning Purifies Backdoored Deep Models
As deep neural networks (DNNs) are growing larger, their requirements for computational resources become huge, which makes outsourcing training more popular.
Moiré Attack (MA): A New Potential Risk of Screen Photos
Images, captured by a camera, play a critical role in training Deep Neural Networks (DNNs).
Generalization in Deep RL for TSP Problems via Equivariance and Local Search
Since training on large instances is impractical, we design a novel deep RL approach with a focus on generalizability.
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness.
Improving Generalization of Deep Reinforcement Learning-based TSP Solvers
In this work, we propose a novel approach named MAGIC that includes a deep learning architecture and a DRL training method.
Dissecting Local Properties of Adversarial Examples
Adversarial examples have attracted significant attention over the years, yet a sufficient understanding is in lack, especially when analyzing their performances in combination with adversarial training.
Optimization inspired Multi-Branch Equilibrium Models
Works have shown the strong connections between some implicit models and optimization problems.
Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State
In this work, we consider feedback spiking neural networks, which are more brain-like, and propose a novel training method that does not rely on the exact reverse of the forward computation.
Fooling Adversarial Training with Induction Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Towards Understanding Catastrophic Overfitting in Fast Adversarial Training
After adversarial training was proposed, a series of works focus on improving the compunational efficiency of adversarial training for deep neural networks (DNNs).
Certified Adversarial Robustness Under the Bounded Support Set
In this paper, we introduce a framework that is able to deal with robustness properties of arbitrary smoothing measures including those with bounded support set by using Wasserstein distance as well as total variation distance.
Understanding Graph Learning with Local Intrinsic Dimensionality
Many real-world problems can be formulated as graphs and solved by graph learning techniques.
Chaos is a Ladder: A New Understanding of Contrastive Learning
Our work suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and it is the overlapping augmented views (i. e., the chaos) that create a ladder for contrastive learning to gradually learn class-separated representations.
Domain-wise Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove the domain-varying features and improve OOD generalization on both correlation shift and diversity shift tasks.
Reparameterized Sampling for Generative Adversarial Networks
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
Demystifying Adversarial Training via A Unified Probabilistic Framework
Based on these, we propose principled adversarial sampling algorithms in both supervised and unsupervised scenarios.
Adversarial Interaction Attacks: Fooling AI to Misinterpret Human Intentions
Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life.
Leveraged Weighted Loss for Partial Label Learning
As an important branch of weakly supervised learning, partial label learning deals with data where each instance is assigned with a set of candidate labels, whereas only one of them is true.
GBHT: Gradient Boosting Histogram Transform for Density Estimation
In this paper, we propose a density estimation algorithm called \textit{Gradient Boosting Histogram Transform} (GBHT), where we adopt the \textit{Negative Log Likelihood} as the loss function to make the boosting procedure available for the unsupervised tasks.
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
Can models with particular structure avoid being biased towards spurious correlation in out-of-distribution (OOD) generalization?
Analysis and Applications of Class-wise Robustness in Adversarial Training
Adversarial training is one of the most effective approaches to improve model robustness against adversarial examples.
Optimization Induced Equilibrium Networks
In this paper, we investigate an emerging question: can an implicit equilibrium model's equilibrium point be regarded as the solution of an optimization problem?
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Improving Adversarial Robustness via Channel-wise Activation Suppressing
The study of adversarial examples and their activation has attracted significant attention for secure and robust learning with deep neural networks (DNNs).
Dissecting the Diffusion Process in Linear Graph Convolutional Networks
Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years.
What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space
In this paper, we focus on image classification and propose a method to visualize and understand the class-wise knowledge (patterns) learned by DNNs under three different settings including natural, backdoor and adversarial.
Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions
Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life.
Unlearnable Examples: Making Personal Data Unexploitable
This paper raises the question: \emph{can data be made unlearnable for deep learning models?}
Towards A Unified Understanding and Improving of Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
Efficient Sampling for Generative Adversarial Networks with Coupling Markov Chains
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
Intriguing class-wise properties of adversarial training
Adversarial training is one of the most effective approaches to improve model robustness against adversarial examples.
A Unified Approach to Interpreting and Boosting Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
Improving Query Efficiency of Black-box Adversarial Attack
Deep neural networks (DNNs) have demonstrated excellent performance on various tasks, however they are under the risk of adversarial examples that can be easily generated when the target model is accessible to an attacker (white-box setting).
Temporal Calibrated Regularization for Robust Noisy Label Learning
Deep neural networks (DNNs) exhibit great success on many tasks with the help of large-scale well annotated datasets.
Normalized Loss Functions for Deep Learning with Noisy Labels
However, in practice, simply being robust is not sufficient for a loss function to train accurate DNNs.
Ranked #32 on
Image Classification
on mini WebVision 1.0
(ImageNet Top-1 Accuracy metric)
Improving Adversarial Robustness Requires Revisiting Misclassified Examples
In this paper, we investigate the distinctive influence of misclassified and correctly classified examples on the final robustness of adversarial training.
Adversarial Weight Perturbation Helps Robust Generalization
The study on improving the robustness of deep neural networks against adversarial examples grows rapidly in recent years.
Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples.
Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
We find that using more gradients from the skip connections rather than the residual modules according to a decay factor, allows one to craft adversarial examples with high transferability.
Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation
Based on which, we further find that there is redundancy among the dimensions of latent variable, and the lengths and sentence patterns of the responses can be strongly correlated to each dimension of the latent variable.
Symmetric Cross Entropy for Robust Learning with Noisy Labels
In this paper, we show that DNN learning with Cross Entropy (CE) exhibits overfitting to noisy labels on some classes ("easy" classes), but more surprisingly, it also suffers from significant under learning on some other classes ("hard" classes).
Ranked #43 on
Image Classification
on Clothing1M
Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
This raises safety concerns about the deployment of these systems in clinical settings.
Joint Semantic Domain Alignment and Target Classifier Learning for Unsupervised Domain Adaptation
Unsupervised domain adaptation aims to transfer the classifier learned from the source domain to the target domain in an unsupervised manner.
Learning Deep Hidden Nonlinear Dynamics from Aggregate Data
Learning nonlinear dynamics from diffusion data is a challenging problem since the individuals observed may be different at different time points, generally following an aggregate behaviour.
Dimensionality-Driven Learning with Noisy Labels
Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs).
Ranked #41 on
Image Classification
on mini WebVision 1.0
Decoupled Networks
Inner product-based convolution has been a central component of convolutional neural networks (CNNs) and the key to learning visual representations.
Iterative Learning with Open-set Noisy Labels
We refer to this more complex scenario as the \textbf{open-set noisy label} problem and show that it is nontrivial in order to make accurate predictions.
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction.
Residual Convolutional CTC Networks for Automatic Speech Recognition
Furthermore, we introduce a CTC-based system combination, which is different from the conventional frame-wise senone-based one.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Unifying Decision Trees Split Criteria Using Tsallis Entropy
In this paper, a Tsallis Entropy Criterion (TEC) algorithm is proposed to unify Shannon entropy, Gain Ratio and Gini index, which generalizes the split criteria of decision trees.
