Search Results for author:
Found 66 papers, 21 papers with code
On Breaking Deep Generative Model-based Defenses and Beyond
The idea is to view the inversion phase as a dynamical system, through which we extract the gradient with respect to the input by tracing its recent trajectory.
Functional Bayesian Neural Networks for Model Uncertainty Quantification
In this paper, we extend the Bayesian neural network to functional Bayesian neural network with functional Monte Carlo methods that use the samples of functionals instead of samples of networks' parameters for inference to overcome the curse of dimensionality for uncertainty quantification.
Unified Spatial-Temporal Edge-Enhanced Graph Networks for Pedestrian Trajectory Prediction
We propose the Edge-to-Edge-Node-to-Node Graph Convolution (E2E-N2N-GCN), a novel dual-graph network that jointly models explicit N2N social interactions among pedestrians and implicit E2E influence propagation across these interaction patterns.
SynthFormer: Equivariant Pharmacophore-based Generation of Synthesizable Molecules for Ligand-Based Drug Design
Drug discovery is a complex, resource-intensive process requiring significant time and cost to bring new medicines to patients.
Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
Bi-level optimization (BO) has become a fundamental mathematical framework for addressing hierarchical machine learning problems.
Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability
While previous studies primarily focus on the affects of label noises to the performance of learning, our work intends to investigate the implicit regularization effects of the label noises, under mini-batch sampling settings of stochastic gradient descent (SGD), with assumptions that label noises are unbiased.
MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows
The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator learns to minimize this divergence.
Fine-grained differentiable physics: a yarn-level model for fabrics
To this end, we propose several differentiable forces, whose counterparts in empirical physics are indifferentiable, to facilitate gradient-based learning.
Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay
Specifically, 1) we introduce the assumptions that can lead to equilibrium state in SMD, and prove equilibrium can be reached in a linear rate regime under given assumptions; 2) we propose ``angular update" as a substitute for effective learning rate to depict the state of SMD, and derive the theoretical value of angular update in equilibrium state; 3) we verify our assumptions and theoretical results on various large-scale computer vision tasks including ImageNet and MSCOCO with standard settings.
Implicit Bias of Adversarial Training for Deep Neural Networks
Furthermore, we generalize this result to the case of adversarial training for non-linear homogeneous deep neural networks without the linear separability of the dataset.
Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI
This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI.
Adversarial Invariant Learning
To the best of our knowledge, this is one of the first to adopt differentiable environment splitting method to enable stable predictions across environments without environment index information, which achieves the state-of-the-art performance on datasets with strong spurious correlation, such as Colored MNIST.
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of deep networks.
A frequency domain analysis of gradient-based adversarial examples
And the log-spectrum difference of the adversarial examples and clean image is more concentrated in the high-frequency part than the low-frequency part.
Implicit Regularization Effects of Unbiased Random Label Noises with SGD
Random label noises (or observational noises) widely exist in practical machinelearning settings.
Neural Approximate Sufficient Statistics for Likelihood-free Inference
We consider the fundamental problem of how to automatically construct summary statistics for likelihood-free inference where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible.
Can We Use Gradient Norm as a Measure of Generalization Error for Model Selection in Practice?
The recent theoretical investigation (Li et al., 2020) on the upper bound of generalization error of deep neural networks (DNNs) demonstrates the potential of using the gradient norm as a measure that complements validation accuracy for model selection in practice.
Robust Multi-Agent Reinforcement Learning Driven by Correlated Equilibrium
Therefore, to achieve robust CMARL, we introduce novel strategies to encourage agents to learn correlated equilibrium while maximally preserving the convenience of the decentralized execution.
Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting
SFTGNN could effectively learn hidden spatial-temporal dependencies by a novel fusion operation of various spatial and temporal graphs, which is generated by a data-driven method.
Ranked #4 on
Traffic Prediction
on BJTaxi
Amata: An Annealing Mechanism for Adversarial Training Acceleration
However, conducting adversarial training brings much computational overhead compared with standard training.
Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher
In this paper, we theoretically analyze the knowledge distillation of a wide neural network.
Neural Approximate Sufficient Statistics for Implicit Models
We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of the likelihood function is intractable, but sampling data from the model is possible.
Automatic Data Augmentation for 3D Medical Image Segmentation
Data augmentation is an effective and universal technique for improving generalization performance of deep neural networks.
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
Spherical Motion Dynamics: Learning Dynamics of Neural Network with Normalization, Weight Decay, and SGD
In this work, we comprehensively reveal the learning dynamics of neural network with normalization, weight decay (WD), and SGD (with momentum), named as Spherical Motion Dynamics (SMD).
Classify and Generate Reciprocally: Simultaneous Positive-Unlabelled Learning and Conditional Generation with Extra Data
The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems.
Global Robustness Verification Networks
The wide deployment of deep neural networks, though achieving great success in many domains, has severe safety and reliability concerns.
Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy
Regularization plays a crucial role in machine learning models, especially for deep neural networks.
Towards Making Deep Transfer Learning Never Hurt
The empirical results show that the proposed descent direction estimation strategy DTNH can always improve the performance of deep transfer learning tasks based on all above regularizers, even when transferring pre-trained weights from inappropriate networks.
Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling
In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications.
AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models
The design of deep graph models still remains to be investigated and the crucial part is how to explore and exploit the knowledge from different hops of neighbors in an efficient way.
Ranked #2 on
Node Classification
on MS ACADEMIC
On the Noisy Gradient Descent that Generalizes as SGD
The gradient noise of SGD is considered to play a central role in the observed strong generalization abilities of deep learning.
Efficient Neural Architecture Search via Proximal Iterations
Recently, DARTS, which constructs a differentiable search space and then optimizes it by gradient descent, can obtain high-performance architecture and reduces the search time to several days.
On the Learning Dynamics of Two-layer Nonlinear Convolutional Neural Networks
Convolutional neural networks (CNNs) have achieved remarkable performance in various fields, particularly in the domain of computer vision.
Interpreting Adversarially Trained Convolutional Neural Networks
Our findings shed some light on why AT-CNNs are more robust than those normally trained ones and contribute to a better understanding of adversarial training over CNNs from an interpretation perspective.
Bayesian Optimized Continual Learning with Attention Mechanism
Though neural networks have achieved much progress in various applications, it is still highly challenging for them to learn from a continuous stream of tasks without forgetting.
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks.
Exploring and Enhancing the Transferability of Adversarial Examples
State-of-the-art deep neural networks are vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs.
A Guider Network for Multi-Dual Learning
A large amount of parallel data is needed to train a strong neural machine translation (NMT) system.
Learning to Search Efficient DenseNet with Layer-wise Pruning
Deep neural networks have achieved outstanding performance in many real-world applications with the expense of huge computational resources.
SHE2: Stochastic Hamiltonian Exploration and Exploitation for Derivative-Free Optimization
Derivative-free optimization (DFO) using trust region methods is frequently used for machine learning applications, such as (hyper-)parameter optimization without the derivatives of objective functions known.
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Minima and Regularization Effects
Along this line, we theoretically study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics.
ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling
In this U-shaped network, a paired sampling operation is proposed in spacetime domain accordingly: the pooling (ST-Pool) coarsens the input graph in spatial from its deterministic partition while abstracts multi-resolution temporal dependencies through dilated recurrent skip connections; based on previous settings in the downsampling, the unpooling (ST-Unpool) restores the original structure of spatio-temporal graphs and resumes regular intervals within graph sequences.
Ranked #1 on
Traffic Prediction
on PeMS-M
3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting
(2) We propose an original 3D graph convolution model to model the spatio-temporal data more accurately.
Ranked #2 on
Traffic Prediction
on PeMS-M
Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
In this paper, we present a systematic study on adversarial examples from three aspects: the amount of training data, task-dependent and model-specific factors.
Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN
In this work, we propose a novel defense mechanism called Boundary Conditional GAN to enhance the robustness of deep neural networks against adversarial examples.
Virtual Adversarial Training on Graph Convolutional Networks in Node Classification
The effectiveness of Graph Convolutional Networks (GCNs) has been demonstrated in a wide range of graph-based machine learning tasks.
Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels
In this paper, we propose a novel training algorithm for Graph Convolutional Network, called Multi-Stage Self-Supervised(M3S) Training Algorithm, combined with self-supervised learning approach, focusing on improving the generalization performance of GCNs on graphs with few labeled nodes.
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent
We show in this case that the quasi-potential function is related to the noise covariance structure of SGD via a partial differential equation of Hamilton-Jacobi type.
Bayesian Adversarial Learning
In this work, a novel robust training framework is proposed to alleviate this issue, Bayesian Robust Learning, in which a distribution is put on the adversarial data-generating distribution to account for the uncertainty of the adversarial data-generating process.
Tangent-Normal Adversarial Regularization for Semi-supervised Learning
The proposed TNAR is composed by two complementary parts, the tangent adversarial regularization (TAR) and the normal adversarial regularization (NAR).
Neural Control Variates for Variance Reduction
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications.
Reinforced Continual Learning
In this work, a novel approach for continual learning is proposed, which searches for the best neural architecture for each coming task via sophisticatedly designed reinforcement learning strategies.
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects
Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics.
Understanding and Enhancing the Transferability of Adversarial Examples
State-of-the-art deep neural networks are known to be vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs.
Enhancing the Transferability of Adversarial Examples with Noise Reduced Gradient
Deep neural networks provide state-of-the-art performance for many applications of interest.
Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions.
Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
Timely accurate traffic forecast is crucial for urban traffic control and guidance.
Ranked #2 on
Time Series Forecasting
on PeMSD7
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
It is widely observed that deep learning models with learned parameters generalize well, even with much more model parameters than the number of training samples.
Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix
We show that the dynamic transition matrix can effectively characterize the noise in the training data built by distant supervision.
Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks
Minimizing non-convex and high-dimensional objective functions is challenging, especially when training modern deep neural networks.
Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems
We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions.
Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling
Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning.
Adaptive Stochastic Primal-Dual Coordinate Descent for Separable Saddle Point Problems
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications.
