Search Results for author:
Found 50 papers, 22 papers with code
Counterpart Fairness -- Addressing Systematic between-group Differences in Fairness Evaluation
The confounding factors, which are non-sensitive variables but manifest systematic differences, can significantly affect fairness evaluation.
Contrastive Label Enhancement
Label distribution learning (LDL) is a new machine learning paradigm for solving label ambiguity.
CFA: Class-wise Calibrated Fair Adversarial Training
Adversarial training has been widely acknowledged as the most effective method to improve the adversarial robustness against adversarial examples for Deep Neural Networks (DNNs).
ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond
Instead of characterizing oversmoothing from the view of complete collapse in which representations converge to a single point, we dive into a more general perspective of dimensional collapse in which representations lie in a narrow cone.
Ranked #4 on
Node Property Prediction
on ogbn-arxiv
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics.
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism
In this work, we propose a unified theoretical understanding for existing variants of non-contrastive learning.
Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning
Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap.
Multi-view Semantic Consistency based Information Bottleneck for Clustering
In this way, the learned semantic consistency from multi-view data can improve the information bottleneck to more exactly distinguish the consistent information and learn a unified feature representation with more discriminative consistent information for clustering.
MCoCo: Multi-level Consistency Collaborative Multi-view Clustering
Further, we designed a multi-level consistency collaboration strategy, which utilizes the consistent information of semantic space as a self-supervised signal to collaborate with the cluster assignments in feature space.
Ranked #1 on
Multiview Clustering
on Fashion-MNIST
USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
To this end, we propose USER, an unsupervised robust version of graph neural networks that is based on structural entropy.
On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.
Biomedical image analysis competitions: The state of current participation practice
Of these, 84% were based on standard architectures.
Beyond the Best: Estimating Distribution Functionals in Infinite-Armed Bandits
In the infinite-armed bandit problem, each arm's average reward is sampled from an unknown distribution, and each arm can be sampled further to obtain noisy estimates of the average reward of that arm.
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders
Masked Autoencoders (MAE) based on a reconstruction task have risen to be a promising paradigm for self-supervised learning (SSL) and achieve state-of-the-art performance across different benchmark datasets.
When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs.
Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance.
Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery
For randomly generated data, we show the existence of a phase transition in recovering planted neural network models, which is easy to describe: whenever the ratio between the number of samples and the dimension exceeds a numerical threshold, the recovery succeeds with high probability; otherwise, it fails with high probability.
Motif-based Graph Representation Learning with Application to Chemical Molecules
MCM builds a motif vocabulary in an unsupervised way and deploys a novel motif convolution operation to extract the local structural context of individual nodes, which is then used to learn higher-level node representations via multilayer perceptron and/or message passing in graph neural networks.
Optimization-Induced Graph Implicit Nonlinear Diffusion
Moreover, we show that the optimization-induced variants of our models can boost the performance and improve training stability and efficiency as well.
A DTCWT-SVD Based Video Watermarking resistant to frame rate conversion
To address this issue, we present a new video watermarking based on joint Dual-Tree Cosine Wavelet Transformation (DTCWT) and Singular Value Decomposition (SVD), which is resistant to frame rate conversion.
Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization
We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation.
A stochastic Stein Variational Newton method
Meanwhile, Stein variational Newton (SVN), a Newton-like extension of SVGD, dramatically accelerates the convergence of SVGD by incorporating Hessian information into the dynamics, but also produces biased samples.
Knowledgebra: An Algebraic Learning Framework for Knowledge Graph
As far as we know, by applying abstract algebra in statistical learning, this work develops the first formal language for general knowledge graphs, and also sheds light on the problem of neural-symbolic integration from an algebraic perspective.
A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training
On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM.
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap
Our theory suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and the overlapped augmented views (i. e., the chaos) create a ladder for contrastive learning to gradually learn class-separated representations.
Fooling Adversarial Training with Inducing Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Residual Relaxation for Multi-view Representation Learning
Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation.
Parallel Deep Neural Networks Have Zero Duality Gap
Recent work has proven that the strong duality holds (which means zero duality gap) for regularized finite-width two-layer ReLU networks and consequently provided an equivalent convex training problem.
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program
We then show that the limit points of non-convex subgradient flows can be identified via primal-dual correspondence in this convex optimization problem.
Predicting the Stereoselectivity of Chemical Transformations by Machine Learning
Since late 1960s, there have been numerous successes in the exciting new frontier of asymmetric catalysis.
The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions
As additional consequences of our convex perspective, (i) we establish that Clarke stationary points found by stochastic gradient descent correspond to the global optimum of a subsampled convex problem (ii) we provide a polynomial-time algorithm for checking if a neural network is a global minimum of the training loss (iii) we provide an explicit construction of a continuous path between any neural network and the global minimum of its sublevel set and (iv) characterize the minimal size of the hidden layer so that the neural network optimization landscape has no spurious valleys.
Fooling Adversarial Training with Induction Noise
Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack.
Chaos is a Ladder: A New Understanding of Contrastive Learning
Our work suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and it is the overlapping augmented views (i. e., the chaos) that create a ladder for contrastive learning to gradually learn class-separated representations.
Domain-wise Adversarial Training for Out-of-Distribution Generalization
Extensive experiments show that our proposed DAT can effectively remove the domain-varying features and improve OOD generalization on both correlation shift and diversity shift tasks.
Reparameterized Sampling for Generative Adversarial Networks
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
Demystifying Adversarial Training via A Unified Probabilistic Framework
Based on these, we propose principled adversarial sampling algorithms in both supervised and unsupervised scenarios.
Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionality
Our first contribution is to show that, at each iteration, the embedding dimension (or sketch size) can be as small as the effective dimension of the Hessian matrix.
Dissecting the Diffusion Process in Linear Graph Convolutional Networks
Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years.
Projected Wasserstein gradient descent for high-dimensional Bayesian inference
We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems.
Dynamic Context-Sensitive Filtering Network for Video Salient Object Detection
Our bidirectional dynamic fusion strategy encourages the interaction of spatial and temporal information in a dynamic manner.
Ranked #11 on
Video Polyp Segmentation
on SUN-SEG-Easy (Unseen)
On Dyadic Fairness: Exploring and Mitigating Bias in Graph Connections
Disparate impact has raised serious concerns in machine learning applications and its societal impacts.
Efficient Sampling for Generative Adversarial Networks with Coupling Markov Chains
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
Train Once, and Decode As You Like
In this paper we propose a unified approach for supporting different generation manners of machine translation, including autoregressive, semi-autoregressive, and refinement-based non-autoregressive models.
Optimizing AD Pruning of Sponsored Search with Reinforcement Learning
Industrial sponsored search system (SSS) can be logically divided into three modules: keywords matching, ad retrieving, and ranking.
Decoder-free Robustness Disentanglement without (Additional) Supervision
Adversarial Training (AT) is proposed to alleviate the adversarial vulnerability of machine learning models by extracting only robust features from the input, which, however, inevitably leads to severe accuracy reduction as it discards the non-robust yet useful features.
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions
As additional consequences of our convex perspective, (i) we establish that Clarke stationary points found by stochastic gradient descent correspond to the global optimum of a subsampled convex problem (ii) we provide a polynomial-time algorithm for checking if a neural network is a global minimum of the training loss (iii) we provide an explicit construction of a continuous path between any neural network and the global minimum of its sublevel set and (iv) characterize the minimal size of the hidden layer so that the neural network optimization landscape has no spurious valleys.
Information Newton's flow: second-order optimization method in probability space
We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows.
Regularized Non-negative Spectral Embedding for Clustering
Spectral Clustering is a popular technique to split data points into groups, especially for complex datasets.
Accelerated Information Gradient flow
We present a framework for Nesterov's accelerated gradient flows in probability space to design efficient mean-field Markov chain Monte Carlo (MCMC) algorithms for Bayesian inverse problems.
Deep Domain Adaptation by Geodesic Distance Minimization
Our work builds on the recently proposed Deep CORAL method, which proposed to train a convolutional neural network and simultaneously minimize the Euclidean distance of convariance matrices between the source and target domains.
