Search Results for author:
Found 40 papers, 6 papers with code
Exploring Example Influence in Continual Learning
Continual Learning (CL) sequentially learns new tasks like human beings, with the goal to achieve better Stability (S, remembering past tasks) and Plasticity (P, adapting to new tasks).
Rethinking Rehearsal in Lifelong Learning: Does An Example Contribute the Plasticity or Stability?
Traditionally, the primary goal of LL is to achieve the trade-off between the Stability (remembering past tasks) and Plasticity (adapting to new tasks).
Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning
Unlike existing FL methods, our IGFL can be applied to both client and server optimization.
Learned Interpretable Residual Extragradient ISTA for Sparse Coding
Recently, the study on learned iterative shrinkage thresholding algorithm (LISTA) has attracted increasing attentions.
Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration
The training of deep neural networks (DNNs) always requires intensive resources for both computation and data storage.
Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling
In this paper, we propose a novel deep neural network with Dual Subnet and Multi-stage Communicated Upsampling (DSMC) for super-resolution of videos with large motion.
MWQ: Multiscale Wavelet Quantized Neural Networks
Inspired by the characteristics of images in the frequency domain, we propose a novel multiscale wavelet quantization (MWQ) method.
Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
Since model quantization helps to reduce the model size and computation latency, it has been successfully applied in many applications of mobile phones, embedded devices and smart chips.
Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks
For layer pruning, we convert convolutional layers of network into ResConv with a layer scaling factor.
Differentially Private ADMM Algorithms for Machine Learning
For smooth convex loss functions with (non)-smooth regularization, we propose the first differentially private ADMM (DP-ADMM) algorithm with performance guarantee of $(\epsilon,\delta)$-differential privacy ($(\epsilon,\delta)$-DP).
Boosting Gradient for White-Box Adversarial Attacks
Existing white-box attack algorithms can generate powerful adversarial examples.
A Single Frame and Multi-Frame Joint Network for 360-degree Panorama Video Super-Resolution
Spherical videos, also known as \ang{360} (panorama) videos, can be viewed with various virtual reality devices such as computers and head-mounted displays.
Video Super Resolution Based on Deep Learning: A Comprehensive Survey
To the best of our knowledge, this work is the first systematic review on VSR tasks, and it is expected to make a contribution to the development of recent studies in this area and potentially deepen our understanding to the VSR techniques based on deep learning.
Data Augmentation Imbalance For Imbalanced Attribute Classification
Pedestrian attribute recognition is an important multi-label classification problem.
Deep Residual-Dense Lattice Network for Speech Enhancement
Motivated by this, we propose the residual-dense lattice network (RDL-Net), which is a new CNN for speech enhancement that employs both residual and dense aggregations without over-allocating parameters for feature re-usage.
Ranked #11 on
Speech Enhancement
on VoiceBank + DEMAND
Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization
Large-scale non-convex sparsity-constrained problems have recently gained extensive attention.
Efficient High-Dimensional Data Representation Learning via Semi-Stochastic Block Coordinate Descent Methods
To address this issue, we propose a novel hard thresholding algorithm, called Semi-stochastic Block Coordinate Descent Hard Thresholding Pursuit (SBCD-HTP).
Accelerated Variance Reduced Stochastic Extragradient Method for Sparse Machine Learning Problems
Moreover, our theoretical analysis shows that AVR-SExtraGD enjoys the best-known convergence rates and oracle complexities of stochastic first-order algorithms such as Katyusha for both strongly convex and non-strongly convex problems.
signADAM: Learning Confidences for Deep Neural Networks
In this paper, we propose a new first-order gradient-based algorithm to train deep neural networks.
CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentation
This paper proposes a novel cascaded U-Net for brain tumor segmentation.
Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1
The training of deep neural networks (DNNs) requires intensive resources both for computation and for storage performance.
Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications
The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment.
Efficient Computation of Quantized Neural Networks by {−1, +1} Encoding Decomposition
Deep neural networks require extensive computing resources, and can not be efficiently applied to embedded devices such as mobile phones, which seriously limits their applicability.
ASVRG: Accelerated Proximal SVRG
This paper proposes an accelerated proximal stochastic variance reduced gradient (ASVRG) method, in which we design a simple and effective momentum acceleration trick.
A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks
To address this issue, various approximation techniques have been investigated, which seek for a light weighted network with little performance degradation in exchange of smaller model size or faster inference.
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates
Recent years have witnessed exciting progress in the study of stochastic variance reduced gradient methods (e. g., SVRG, SAGA), their accelerated variants (e. g, Katyusha) and their extensions in many different settings (e. g., online, sparse, asynchronous, distributed).
Tractable and Scalable Schatten Quasi-Norm Approximations for Rank Minimization
The Schatten quasi-norm was introduced to bridge the gap between the trace norm and rank function.
VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning
In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD).
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization
In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of stochastic variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct.
Accelerated First-order Methods for Geodesically Convex Optimization on Riemannian Manifolds
In this paper, we propose an accelerated first-order method for geodesically convex optimization, which is the generalization of the standard Nesterov's accelerated method from Euclidean space to nonlinear Riemannian space.
Accelerated Variance Reduced Stochastic ADMM
Besides having a low per-iteration complexity as existing stochastic ADMM methods, ASVRG-ADMM improves the convergence rate on general convex problems from O(1/T) to O(1/T^2).
Larger is Better: The Effect of Learning Rates Enjoyed by Stochastic Optimization with Progressive Variance Reduction
This setting allows us to use much larger learning rates or step sizes than SVRG, e. g., 3/(7L) for VR-SGD vs 1/(10L) for SVRG, and also makes our convergence analysis more challenging.
Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning
Recently, research on accelerated stochastic gradient descent methods (e. g., SVRG) has made exciting progress (e. g., linear convergence for strongly convex problems).
Guaranteed Sufficient Decrease for Variance Reduced Stochastic Gradient Descent
In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct.
Scalable Algorithms for Tractable Schatten Quasi-Norm Minimization
In this paper, we first define two tractable Schatten quasi-norms, i. e., the Frobenius/nuclear hybrid and bi-nuclear quasi-norms, and then prove that they are in essence the Schatten-2/3 and 1/2 quasi-norms, respectively, which lead to the design of very efficient algorithms that only need to update two much smaller factor matrices.
Unified Scalable Equivalent Formulations for Schatten Quasi-Norms
In this paper, we rigorously prove that for any p, p1, p2>0 satisfying 1/p=1/p1+1/p2, the Schatten-p quasi-norm of any matrix is equivalent to minimizing the product of the Schatten-p1 norm (or quasi-norm) and Schatten-p2 norm (or quasi-norm) of its two factor matrices.
Regularized Orthogonal Tensor Decompositions for Multi-Relational Learning
We first induce the equivalence relation of the Schatten p-norm (0<p<\infty) of a low multi-linear rank tensor and its core tensor.
Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion
Then the Schatten 1-norm of the core tensor is used to replace that of the whole tensor, which leads to a much smaller-scale matrix SNM problem.
Structured Low-Rank Matrix Factorization with Missing and Grossly Corrupted Observations
In this paper, we propose a scalable, provable structured low-rank matrix factorization method to recover low-rank and sparse matrices from missing and grossly corrupted data, i. e., robust matrix completion (RMC) problems, or incomplete and grossly corrupted measurements, i. e., compressive principal component pursuit (CPCP) problems.
Generalized Higher-Order Tensor Decomposition via Parallel ADMM
To address these problems, we first propose a parallel trace norm regularized tensor decomposition method, and formulate it as a convex optimization problem.
