Search Results for author:
Found 41 papers, 16 papers with code
Learning to Control the Smoothness of Graph Convolutional Network Features
(2) Building on our geometric insights, we augment the message-passing process of graph convolutional layers (GCLs) with a learnable term to modulate the smoothness of node features with computational efficiency.
Deep Learning with Data Privacy via Residual Perturbation
Protecting data privacy in deep learning (DL) is of crucial importance.
Adaptive and Implicit Regularization for Matrix Completion
Theoretically, we show that the adaptive regularization of \ReTwo{AIR} enhances the implicit regularization and vanishes at the end of training.
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization
Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the input sequence.
Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs
Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers.
Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs
We compare HBNODE with other popular ROMs on several complex dynamical systems, including the von K\'{a}rm\'{a}n Street flow, the Kurganov-Petrova-Popov equation, and the one-dimensional Euler equations for fluids modeling.
glassoformer: a query-sparse transformer for post-fault power grid voltage prediction
Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers.
Efficient and Reliable Overlay Networks for Decentralized Federated Learning
As such, our proposed overlay networks accelerate convergence, improve generalization, and enhance robustness to clients failures in DFL with theoretical guarantees.
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
In this paper, to eliminate the effort for tuning the momentum-related hyperparameter, we propose a new adaptive momentum inspired by the optimal choice of the heavy ball momentum for quadratic optimization.
How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies
As case studies, we consider how momentum can improve the architecture design for recurrent neural networks (RNNs), neural ordinary differential equations (ODEs), and transformers.
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion
Theoretically, we show that the adaptive regularization of AIR enhances the implicit regularization and vanishes at the end of training.
Heavy Ball Neural Ordinary Differential Equations
We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the continuous limit of the classical momentum accelerated gradient descent, to improve neural ODEs (NODEs) training and inference.
GRAND++: Graph Neural Diffusion with A Source Term
We propose GRAph Neural Diffusion with a source term (GRAND++) for graph deep learning with a limited number of labeled nodes, i. e., low-labeling rate.
FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention
For instance, FMMformers achieve an average classification accuracy of $60. 74\%$ over the five Long Range Arena tasks, which is significantly better than the standard transformer's average accuracy of $58. 70\%$.
Decentralized Federated Averaging
In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients.
Robust Certification for Laplace Learning on Geometric Graphs
Graph Laplacian (GL)-based semi-supervised learning is one of the most used approaches for classifying nodes in a graph.
Stability and Generalization of the Decentralized Stochastic Gradient Descent
The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models.
Certifying Robustness of Graph Laplacian Based Semi-Supervised Learning
Within a certain adversarial perturbation regime, we prove that GL with a $k$-nearest neighbor graph is intrinsically more robust than the $k$-nearest neighbor classifier.
Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum
In this paper, we propose a novel \emph{adaptive momentum} for improving DNNs training; this adaptive momentum, with no momentum related hyperparameter required, is motivated by the nonlinear conjugate gradient (NCG) method.
Deep Interactive Denoiser (DID) for X-Ray Computed Tomography
However, there exists two challenges regarding the DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs which sometimes are needed for different clinical tasks; 2) the model generalizability might be an issue when the noise level in the testing images is different from that in the training dataset.
An Integrated Approach to Produce Robust Models with High Efficiency
To solve the problems that adversarial training jeopardizes DNNs' accuracy on clean images and the struture of sparsity, we design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity.
MomentumRNN: Integrating Momentum into Recurrent Neural Networks
Designing deep neural networks is an art that often involves an expensive search over candidate architectures.
Reflections in the Sky: Joint Trajectory and Passive Beamforming Design for Secure UAV Networks with Reconfigurable Intelligent Surface
This paper investigates the problem of secure energy efficiency maximization for a reconfigurable intelligent surface (RIS) assisted uplink wireless communication system, where an unmanned aerial vehicle (UAV) equipped with an RIS works as a mobile relay between the base station (BS) and a group of users.
Differentially Private Federated Learning with Laplacian Smoothing
Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users.
Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets
In this paper, we focus on a co-design of efficient DNN compression algorithms and sparse neural architectures for robust and accurate deep learning.
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent
Nesterov accelerated gradient (NAG) improves the convergence rate of gradient descent (GD) for convex optimization using a specially designed momentum; however, it accumulates error when an inexact gradient is used (such as in SGD), slowing convergence at best and diverging at worst.
Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo
As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling.
Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning.
DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM
At the core of DP-LSSGD is the Laplacian smoothing, which smooths out the Gaussian noise used in the Gaussian mechanism.
A Study on Graph-Structured Recurrent Neural Networks and Sparsification with Application to Epidemic Forecasting
We study epidemic forecasting on real-world health data by a graph-structured recurrent neural network (GSRNN).
A Deterministic Gradient-Based Approach to Avoid Saddle Points
First-order methods such as gradient descent are usually the methods of choice for training machine learning models.
ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from satisfactory.
Mathematical Analysis of Adversarial Attacks
In this paper, we analyze efficacy of the fast gradient sign method (FGSM) and the Carlini-Wagner's L2 (CW-L2) attack.
Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation.
Laplacian Smoothing Gradient Descent
We propose a class of very simple modifications of gradient descent and stochastic gradient descent.
Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning
Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.
Graph-Based Deep Modeling and Real Time Forecasting of Sparse Spatio-Temporal Data
We present a generic framework for spatio-temporal (ST) data modeling, analysis, and forecasting, with a special focus on data that is sparse in both space and time.
Deep Neural Nets with Interpolating Function as Output Activation
We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function.
Deep Learning for Real-Time Crime Forecasting and its Ternarization
In this work, we first present a proper representation of crime data.
Deep Learning for Real Time Crime Forecasting
Experiments over a half year period in Los Angeles reveal highly accurate predictive power of our models.
Feature functional theory - binding predictor (FFT-BP) for the blind prediction of binding free energies
The underpinning assumptions of FFT-BP are as follows: i) representability: there exists a microscopic feature vector that can uniquely characterize and distinguish one protein-ligand complex from another; ii) feature-function relationship: the macroscopic features, including binding free energy, of a complex is a functional of microscopic feature vectors; and iii) similarity: molecules with similar microscopic features have similar macroscopic features, such as binding affinity.
