Search Results for author:
Found 71 papers, 31 papers with code
How to Craft Backdoors with Unlabeled Data Alone?
Relying only on unlabeled data, Self-supervised learning (SSL) can learn rich features in an economical and scalable way.
SM2C: Boost the Semi-supervised Segmentation for Medical Image by using Meta Pseudo Labels and Mixed Images
This method uses three strategies - scaling-up image size, multi-class mixing, and object shape jittering - to improve the ability to learn semantic features within medical images.
Do Generated Data Always Help Contrastive Learning?
Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augmentations.
Non-negative Contrastive Learning
In this paper, we propose Non-negative Contrastive Learning (NCL), a renaissance of Non-negative Matrix Factorization (NMF) aimed at deriving interpretable features.
A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Features
We prove that training neural networks on 1-D data is equivalent to solving a convex Lasso problem with a fixed, explicitly defined dictionary matrix of features.
On the Duality Between Sharpness-Aware Minimization and Adversarial Training
Instead of perturbing the samples, Sharpness-Aware Minimization (SAM) perturbs the model weights during training to find a more flat loss landscape and improve generalization.
Federated learning-outcome prediction with multi-layer privacy protection
To this end, this study proposes a distributed grade prediction model, dubbed FecMap, by exploiting the federated learning (FL) framework that preserves the private data of local clients and communicates with others through a global generalized model.
Erasing Self-Supervised Learning Backdoor by Cluster Activation Masking
After obtaining the threat model trained on the poisoned dataset, our method can precisely detect poisonous samples based on the assumption that masking the backdoor trigger can effectively change the activation of a downstream clustering model.
STF: Spatial Temporal Fusion for Trajectory Prediction
The main reason is that the trajectory is a kind of complex data, including spatial and temporal information, which is crucial for accurate prediction.
Polynomial-Time Solutions for ReLU Network Training: A Complexity Classification via Max-Cut and Zonotopes
Using this convex formulation, we prove that the hardness of approximation of ReLU networks not only mirrors the complexity of the Max-Cut problem but also, in certain special cases, exactly corresponds to it.
Asymmetric Contrastive Multimodal Learning for Advancing Chemical Understanding
Through practical tasks such as isomer discrimination and uncovering crucial chemical properties for drug discovery, ACML exhibits its capability to revolutionize chemical research and applications, providing a deeper understanding of chemical semantics of different modalities.
Adversarial Examples Are Not Real Features
A well-known theory by \citet{ilyas2019adversarial} explains adversarial vulnerability from a data perspective by showing that one can extract non-robust features from adversarial examples and these features alone are useful for classification.
Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding
However, from a theoretical perspective, the universal expressive power of spectral embedding comes at the price of losing two important invariance properties of graphs, sign and basis invariance, which also limits its effectiveness on graph data.
Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey
This survey presents a comprehensive overview of natural language interfaces for tabular data querying and visualization, which allow users to interact with data using natural language queries.
Towards Control-Centric Representations in Reinforcement Learning from Images
Image-based Reinforcement Learning is a practical yet challenging task.
OODRobustBench: benchmarking and analyzing adversarial robustness under distribution shift
Based on this, we are able to predict the upper limit of OOD robustness for existing robust training schemes.
CSG: Curriculum Representation Learning for Signed Graph
Signed graphs are valuable for modeling complex relationships with positive and negative connections, and Signed Graph Neural Networks (SGNNs) have become crucial tools for their analysis.
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations
Large Language Models (LLMs) have shown remarkable success in various tasks, but concerns about their safety and the potential for generating malicious content have emerged.
Robust Long-Tailed Learning via Label-Aware Bounded CVaR
Data in the real-world classification problems are always imbalanced or long-tailed, wherein the majority classes have the most of the samples that dominate the model training.
Rethinking Weak Supervision in Helping Contrastive Learning
Therefore, to explore the mechanical differences between semi-supervised and noisy-labeled information in helping contrastive learning, we establish a unified theoretical framework of contrastive learning under weak supervision.
On the Generalization of Multi-modal Contrastive Learning
Multi-modal contrastive learning (MMCL) has recently garnered considerable interest due to its superior performance in visual tasks, achieved by embedding multi-modal data, such as visual-language pairs.
Counterpart Fairness -- Addressing Systematic between-group Differences in Fairness Evaluation
The confounding factors, which are non-sensitive variables but manifest systematic differences, can significantly affect fairness evaluation.
Contrastive Label Enhancement
Label distribution learning (LDL) is a new machine learning paradigm for solving label ambiguity.
CFA: Class-wise Calibrated Fair Adversarial Training
Adversarial training has been widely acknowledged as the most effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs).
ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond
Instead of characterizing oversmoothing from the view of complete collapse in which representations converge to a single point, we dive into a more general perspective of dimensional collapse in which representations lie in a narrow cone.
Ranked #8 on
Node Property Prediction
on ogbn-arxiv
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics.
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism
In this work, we propose a unified theoretical understanding for existing variants of non-contrastive learning.
Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning
Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap.
Multi-view Semantic Consistency based Information Bottleneck for Clustering
In this way, the learned semantic consistency from multi-view data can improve the information bottleneck to more exactly distinguish the consistent information and learn a unified feature representation with more discriminative consistent information for clustering.
MCoCo: Multi-level Consistency Collaborative Multi-view Clustering
Further, we designed a multi-level consistency collaboration strategy, which utilizes the consistent information of semantic space as a self-supervised signal to collaborate with the cluster assignments in feature space.
Ranked #1 on
Multiview Clustering
on Fashion-MNIST
USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
To this end, we propose USER, an unsupervised robust version of graph neural networks that is based on structural entropy.
On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.
Biomedical image analysis competitions: The state of current participation practice
Of these, 84% were based on standard architectures.
Beyond the Best: Estimating Distribution Functionals in Infinite-Armed Bandits
In the infinite-armed bandit problem, each arm's average reward is sampled from an unknown distribution, and each arm can be sampled further to obtain noisy estimates of the average reward of that arm.
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders
Masked Autoencoders (MAE) based on a reconstruction task have risen to be a promising paradigm for self-supervised learning (SSL) and achieve state-of-the-art performance across different benchmark datasets.
When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs.
Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance.
Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery
For randomly generated data, we show the existence of a phase transition in recovering planted neural network models, which is easy to describe: whenever the ratio between the number of samples and the dimension exceeds a numerical threshold, the recovery succeeds with high probability; otherwise, it fails with high probability.
Motif-based Graph Representation Learning with Application to Chemical Molecules
MCM builds a motif vocabulary in an unsupervised way and deploys a novel motif convolution operation to extract the local structural context of individual nodes, which is then used to learn higher-level node representations via multilayer perceptron and/or message passing in graph neural networks.
Optimization-Induced Graph Implicit Nonlinear Diffusion
Moreover, we show that the optimization-induced variants of our models can boost the performance and improve training stability and efficiency as well.
A DTCWT-SVD Based Video Watermarking resistant to frame rate conversion
To address this issue, we present a new video watermarking based on joint Dual-Tree Cosine Wavelet Transformation (DTCWT) and Singular Value Decomposition (SVD), which is resistant to frame rate conversion.
Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization
We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation.
A stochastic Stein Variational Newton method
Meanwhile, Stein variational Newton (SVN), a Newton-like extension of SVGD, dramatically accelerates the convergence of SVGD by incorporating Hessian information into the dynamics, but also produces biased samples.
Knowledgebra: An Algebraic Learning Framework for Knowledge Graph
As far as we know, by applying abstract algebra in statistical learning, this work develops the first formal language for general knowledge graphs, and also sheds light on the problem of neural-symbolic integration from an algebraic perspective.
A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training
On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM.
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap
Our theory suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and the overlapped augmented views (i. e., the chaos) create a ladder for contrastive learning to gradually learn class-separated representations.
Fooling Adversarial Training with Inducing Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Residual Relaxation for Multi-view Representation Learning
Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation.
Parallel Deep Neural Networks Have Zero Duality Gap
Recent work has proven that the strong duality holds (which means zero duality gap) for regularized finite-width two-layer ReLU networks and consequently provided an equivalent convex training problem.
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program
We then show that the limit points of non-convex subgradient flows can be identified via primal-dual correspondence in this convex optimization problem.
Predicting the Stereoselectivity of Chemical Transformations by Machine Learning
Since late 1960s, there have been numerous successes in the exciting new frontier of asymmetric catalysis.
Fooling Adversarial Training with Induction Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Chaos is a Ladder: A New Understanding of Contrastive Learning
Our work suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and it is the overlapping augmented views (i. e., the chaos) that create a ladder for contrastive learning to gradually learn class-separated representations.
The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions
As additional consequences of our convex perspective, (i) we establish that Clarke stationary points found by stochastic gradient descent correspond to the global optimum of a subsampled convex problem (ii) we provide a polynomial-time algorithm for checking if a neural network is a global minimum of the training loss (iii) we provide an explicit construction of a continuous path between any neural network and the global minimum of its sublevel set and (iv) characterize the minimal size of the hidden layer so that the neural network optimization landscape has no spurious valleys.
Domain-wise Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove the domain-varying features and improve OOD generalization on both correlation shift and diversity shift tasks.
Reparameterized Sampling for Generative Adversarial Networks
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
Demystifying Adversarial Training via A Unified Probabilistic Framework
Based on these, we propose principled adversarial sampling algorithms in both supervised and unsupervised scenarios.
Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionality
Our first contribution is to show that, at each iteration, the embedding dimension (or sketch size) can be as small as the effective dimension of the Hessian matrix.
Dissecting the Diffusion Process in Linear Graph Convolutional Networks
Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years.
Projected Wasserstein gradient descent for high-dimensional Bayesian inference
We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems.
Efficient Sampling for Generative Adversarial Networks with Coupling Markov Chains
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
On Dyadic Fairness: Exploring and Mitigating Bias in Graph Connections
Disparate impact has raised serious concerns in machine learning applications and its societal impacts.
Dynamic Context-Sensitive Filtering Network for Video Salient Object Detection
Our bidirectional dynamic fusion strategy encourages the interaction of spatial and temporal information in a dynamic manner.
Ranked #12 on
Video Polyp Segmentation
on SUN-SEG-Easy (Unseen)
Train Once, and Decode As You Like
In this paper we propose a unified approach for supporting different generation manners of machine translation, including autoregressive, semi-autoregressive, and refinement-based non-autoregressive models.
Optimizing AD Pruning of Sponsored Search with Reinforcement Learning
Industrial sponsored search system (SSS) can be logically divided into three modules: keywords matching, ad retrieving, and ranking.
Decoder-free Robustness Disentanglement without (Additional) Supervision
Adversarial Training (AT) is proposed to alleviate the adversarial vulnerability of machine learning models by extracting only robust features from the input, which, however, inevitably leads to severe accuracy reduction as it discards the non-robust yet useful features.
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions
As additional consequences of our convex perspective, (i) we establish that Clarke stationary points found by stochastic gradient descent correspond to the global optimum of a subsampled convex problem (ii) we provide a polynomial-time algorithm for checking if a neural network is a global minimum of the training loss (iii) we provide an explicit construction of a continuous path between any neural network and the global minimum of its sublevel set and (iv) characterize the minimal size of the hidden layer so that the neural network optimization landscape has no spurious valleys.
Information Newton's flow: second-order optimization method in probability space
We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows.
Regularized Non-negative Spectral Embedding for Clustering
Spectral Clustering is a popular technique to split data points into groups, especially for complex datasets.
Accelerated Information Gradient flow
We present a framework for Nesterov's accelerated gradient flows in probability space to design efficient mean-field Markov chain Monte Carlo (MCMC) algorithms for Bayesian inverse problems.
Deep Domain Adaptation by Geodesic Distance Minimization
Our work builds on the recently proposed Deep CORAL method, which proposed to train a convolutional neural network and simultaneously minimize the Euclidean distance of convariance matrices between the source and target domains.
