Search Results for author:
Found 93 papers, 13 papers with code
Projection-free Distributed Online Convex Optimization with $O(\sqrt{T})$ Communication Complexity
To deal with complicated constraints via locally light computation in distributed online learning, recent study has presented a projection-free algorithm called distributed online conditional gradient (D-OCG), and achieved an $O(T^{3/4})$ regret bound, where $T$ is the number of prediction rounds.
Structural Properties of Invariant Dual Subspaces of Boolean Networks
With the help of equitable partitions of an STG, we study the structural properties of the smallest invariant dual subspaces containing a number of Boolean functions.
Safety Analysis of Autonomous Driving Systems Based on Model Learning
The safety properties proved in the resulting surrogate model apply to the original ADS with a probabilistic guarantee.
Break the Wall Between Homophily and Heterophily for Graph Representation Learning
Homophily and heterophily are intrinsic properties of graphs that describe whether two linked nodes share similar properties.
Ranked #3 on
Node Classification
on arXiv-year
ECSAS: Exploring Critical Scenarios from Action Sequence in Autonomous Driving
How to model action sequences so that one can further consider the effects of different action parameters in the scenario is the bottleneck of the problem.
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization
The key issue is to track and estimate a sequence of $\mathbf g(\mathbf{w})=(g_1(\mathbf{w}), \ldots, g_m(\mathbf{w}))$ across iterations, where $\mathbf g(\mathbf{w})$ has $m$ blocks and it is only allowed to probe $\mathcal{O}(1)$ blocks to attain their stochastic values and Jacobians.
Smoothed Online Convex Optimization Based on Discounted-Normal-Predictor
In this paper, we investigate an online prediction strategy named as Discounted-Normal-Predictor (Kapralov and Panigrahy, 2010) for smoothed online convex optimization (SOCO), in which the learner needs to minimize not only the hitting cost but also the switching cost.
Online Frank-Wolfe with Unknown Delays
The online Frank-Wolfe (OFW) method has gained much popularity for online convex optimization due to its projection-free property.
A Tree-Structured Multi-Task Model Recommender
Tree-structured multi-task architectures have been employed to jointly tackle multiple vision tasks in the context of multi-task learning (MTL).
Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence
To the best of our knowledge, this is the first time that stochastic algorithms are proposed to optimize NDCG with a provable convergence guarantee.
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance
In this paper, we study contrastive learning from an optimization perspective, aiming to analyze and address a fundamental issue of existing contrastive learning methods that either rely on a large batch size or a large dictionary of feature vectors.
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization
To address these limitations, we propose a Stochastic Multi-level Variance Reduction method (SMVR), which achieves the optimal sample complexity of $\mathcal{O}\left(1 / \epsilon^{3}\right)$ to find an $\epsilon$-stationary point for non-convex objectives.
Weight Expansion: A New Perspective on Dropout and Generalization
While dropout is known to be a successful regularization technique, insights into the mechanisms that lead to this success are still lacking.
Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization
We investigate online convex optimization in non-stationary environments and choose the \emph{dynamic regret} as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence.
Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression
This paper investigates deep neural network (DNN) compression from the perspective of compactly representing and storing trained parameters.
AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning
The first challenge is to determine what parameters to share across tasks to optimize for both memory efficiency and task accuracy.
HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration
Extensive experiments are conducted on two large-scale outdoor LiDAR point cloud datasets to demonstrate the high accuracy and efficiency of the proposed HRegNet.
Rethinking Hard-Parameter Sharing in Multi-Domain Learning
One common sharing practice is to share the bottom layers of a deep neural network among domains while using separate top layers for each domain.
Probabilistic Verification of Neural Networks Against Group Fairness
Our approach has been evaluated with multiple models trained on benchmark datasets and the experiment results show that our approach is effective and efficient.
Momentum Accelerates the Convergence of Stochastic AUPRC Maximization
In this paper, we further improve the stochastic optimization of AURPC by (i) developing novel stochastic momentum methods with a better iteration complexity of $O(1/\epsilon^4)$ for finding an $\epsilon$-stationary solution; and (ii) designing a novel family of stochastic adaptive methods with the same iteration complexity, which enjoy faster convergence in practice.
Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness
In this paper, we propose a framework of filter-based ensemble of deep neuralnetworks (DNNs) to defend against adversarial attacks.
A Simple yet Universal Strategy for Online Convex Optimization
In this paper, we propose a simple strategy for universal online convex optimization, which avoids these limitations.
Randomized Stochastic Variance-Reduced Methods for Multi-Task Stochastic Bilevel Optimization
Although numerous studies have proposed stochastic algorithms for solving these problems, they are limited in two perspectives: (i) their sample complexities are high, which do not match the state-of-the-art result for non-convex stochastic optimization; (ii) their algorithms are tailored to problems with only one lower-level problem.
Invariant Subspace Approach to Boolean (Control) Networks
Then the invariant subspace of Boolean control network (BCN) is also proposed.
ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization
Traditional methods mainly focus on foreground and background frames separation with only a single attention branch and class activation sequence.
Weakly Supervised Action Localization
Weakly-supervised Temporal Action Localization
+1
Online Convex Optimization with Continuous Switching Constraint
To control the switching cost, we introduce the problem of online convex optimization with continuous switching constraint, where the goal is to achieve a small regret given a budget on the \emph{overall} switching cost.
Online Strongly Convex Optimization with Unknown Delays
Specifically, we first extend the delayed variant of OGD for strongly convex functions, and establish a better regret bound of $O(d\log T)$, where $d$ is the maximum delay.
Projection-free Distributed Online Learning with Sublinear Communication Complexity
In this paper, we propose an improved variant of D-OCG, namely D-BOCG, which can attain the same $O(T^{3/4})$ regret bound with only $O(\sqrt{T})$ communication rounds for convex losses, and a better regret bound of $O(T^{2/3}(\log T)^{1/3})$ with fewer $O(T^{1/3}(\log T)^{2/3})$ communication rounds for strongly convex losses.
Non-stationary Linear Bandits Revisited
Existing studies develop various algorithms and show that they enjoy an $\widetilde{O}(T^{2/3}(1+P_T)^{1/3})$ dynamic regret, where $T$ is the time horizon and $P_T$ is the path-length that measures the fluctuation of the evolving unknown parameter.
Revisiting Smoothed Online Learning
Moreover, when the hitting cost is both convex and $\lambda$-quadratic growth, we reduce the competitive ratio to $1 + \frac{2}{\sqrt{\lambda}}$ by minimizing the weighted sum of the hitting cost and the switching cost.
NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge.
Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning
It is shown that DeepPAC outperforms the state-of-the-art statistical method PROVERO, and it achieves more practical robustness analysis than the formal verification tool ERAN.
SID-NISM: A Self-supervised Low-light Image Enhancement Framework
When capturing images in low-light conditions, the images often suffer from low visibility, which not only degrades the visual aesthetics of images, but also significantly degenerates the performance of many computer vision algorithms.
Adaptive Deep Learning for Entity Resolution by Risk Analysis
Built on the recent advances on risk analysis for ER, the proposed approach first trains a deep model on labeled training data, and then fine-tunes it by minimizing its estimated misprediction risk on unlabeled target data.
How does Weight Correlation Affect Generalisation Ability of Deep Neural Networks?
This paper studies the novel concept of weight correlation in deep neural networks and discusses its impact on the networks' generalisation ability.
Projection-free Online Learning over Strongly Convex Sets
In this paper, we study the special case of online learning over strongly convex sets, for which we first prove that OFW enjoys a better regret bound of $O(T^{2/3})$ for general convex losses.
Improving Neural Network Verification through Spurious Region Guided Refinement
The core idea is to make use of the obtained constraints of the abstraction to infer new bounds for the neurons.
How does Weight Correlation Affect the Generalisation Ability of Deep Neural Networks
This paper studies the novel concept of weight correlation in deep neural networks and discusses its impact on the networks' generalisation ability.
Approximate Multiplication of Sparse Matrices with Limited Space
In this paper, we propose to reduce the time complexity by exploiting the sparsity of the input matrices.
Dynamic Regret of Convex and Smooth Functions
We investigate online convex optimization in non-stationary environments and choose the dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence.
Proving Non-Inclusion of Büchi Automata based on Monte Carlo Sampling
While this is well-understood in the termination analysis of programs, this is not the case for the language inclusion analysis of B\"uchi automata, where research mainly focused on improving algorithms for proving language inclusion, with the search for counterexamples left to the expensive complementation operation.
Improved Analysis for Dynamic Regret of Strongly Convex and Smooth Functions
In this paper, we present an improved analysis for dynamic regret of strongly convex and smooth functions.
On the Power of Unambiguity in Büchi Complementation
In this work, we exploit the power of \emph{unambiguity} for the complementation problem of B\"uchi automata by utilizing reduced run directed acyclic graphs (DAGs) over infinite words, in which each vertex has at most one predecessor.
Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffs
In this paper, we study the problem of stochastic linear bandits with finite action sets.
Minimizing Dynamic Regret and Adaptive Regret Simultaneously
To address this limitation, new performance measures, including dynamic regret and adaptive regret have been proposed to guide the design of online algorithms.
Multi-attention Networks for Temporal Localization of Video-level Labels
The model performance is further improved by constructing multiple sets of attention networks.
Adaptive and Efficient Algorithms for Tracking the Best Expert
The first algorithm achieves a second-order tracking regret bound, which improves existing first-order bounds.
Stochastic Optimization for Non-convex Inf-Projection Problems
In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable.
Bandit Convex Optimization in Non-stationary Environments
In this paper, we investigate BCO in non-stationary environments and choose the \emph{dynamic regret} as the performance measure, which is defined as the difference between the cumulative loss incurred by the algorithm and that of any feasible comparator sequence.
Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive Regret of Convex Functions
Along this line of research, this paper presents the first universal algorithm for minimizing the adaptive regret of convex functions.
Multi-Objective Generalized Linear Bandits
In this paper, we study the multi-objective bandits (MOB) problem, where a learner repeatedly selects one arm to play and then receives a reward vector consisting of multiple objectives.
Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss
In this paper, we improve the robustness of DNNs by utilizing techniques of Distance Metric Learning.
Adaptivity and Optimality: A Universal Algorithm for Online Convex Optimization
In this paper, we study adaptive online convex optimization, and aim to design a universal algorithm that achieves optimal regret bounds for multiple common types of loss functions.
SAdam: A Variant of Adam for Strongly Convex Functions
In this paper, we give an affirmative answer by developing a variant of Adam (referred to as SAdam) which achieves a data-dependant $O(\log T)$ regret bound for strongly convex functions.
Prediction with Unpredictable Feature Evolution
Learning with feature evolution studies the scenario where the features of the data streams can evolve, i. e., old features vanish and new features emerge.
Adaptive Regret of Convex and Smooth Functions
We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure.
Stochastic Primal-Dual Algorithms with Faster Convergence than $O(1/\sqrt{T})$ for Problems without Bilinear Structure
The main contribution of this paper is the design and analysis of new stochastic primal-dual algorithms that use a mixture of stochastic gradient updates and a logarithmic number of deterministic dual updates for solving a family of convex-concave problems with no bilinear structure assumed.
Analyzing Deep Neural Networks with Symbolic Propagation: Towards Higher Precision and Faster Verification
Several verification approaches have been developed to automatically prove or disprove safety properties of DNNs.
Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the $O(1/T)$ Convergence Rate
Finally, we emphasize that our proof is constructive and each risk bound is equipped with an efficient stochastic algorithm attaining that bound.
A Wasserstein GAN model with the total variational regularization
It is well known that the generative adversarial nets (GANs) are remarkably difficult to train.
\ell_1-regression with Heavy-tailed Distributions
In this paper, we consider the problem of linear regression with heavy-tailed distributions.
Adaptive Online Learning in Dynamic Environments
In this paper, we study online convex optimization in dynamic environments, and aim to bound the dynamic regret with respect to any sequence of comparators.
Query-Efficient Black-Box Attack by Active Learning
To conduct black-box attack, a popular approach aims to train a substitute model based on the information queried from the target DNN.
Matrix Completion from Non-Uniformly Sampled Entries
Then, for each partially observed column, we recover it by finding a vector which lies in the recovered column space and consists of the observed entries.
Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions
Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set.
An Image dehazing approach based on the airlight field estimation
This paper proposes a scheme for single image haze removal based on the airlight field (ALF) estimation.
$\ell_1$-regression with Heavy-tailed Distributions
In this paper, we consider the problem of linear regression with heavy-tailed distributions.
VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning
In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD).
A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer
In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization.
Learning with Feature Evolvable Streams
To benefit from the recovered features, we develop two ensemble methods.
Scalable Demand-Aware Recommendation
In particular for durable goods, time utility is a function of inter-purchase duration within product category because consumers are unlikely to purchase two items in the same category in close temporal succession.
Empirical Risk Minimization for Stochastic Convex Optimization: $O(1/n)$- and $O(1/n^2)$-type of Risk Bounds
First, we establish an $\widetilde{O}(d/n + \sqrt{F_*/n})$ risk bound when the random function is nonnegative, convex and smooth, and the expected function is Lipschitz continuous, where $d$ is the dimensionality of the problem, $n$ is the number of samples, and $F_*$ is the minimal risk.
Dynamic Regret of Strongly Adaptive Methods
To cope with changing environments, recent developments in online learning have introduced the concepts of adaptive regret and dynamic regret independently.
Efficient Non-oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee
Previously, oblivious random projection based approaches that project high dimensional features onto a random subspace have been used in practice for tackling high-dimensionality challenge in machine learning.
Improved Dynamic Regret for Non-degenerate Functions
When multiple gradients are accessible to the learner, we first demonstrate that the dynamic regret of strongly convex functions can be upper bounded by the minimum of the path-length and the squared path-length.
A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates
In this paper, we develop projection reduced optimization algorithms for both smooth and non-smooth optimization with improved convergence rates under a certain regularity condition of the constraint function.
Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient
Secondly, we present a lower bound with noisy gradient feedback and then show that we can achieve optimal dynamic regrets under a stochastic gradient feedback and two-point bandit feedback.
Sparse Learning for Large-scale and High-dimensional Data: A Randomized Convex-concave Optimization Approach
In this paper, we develop a randomized algorithm and theory for learning a sparse model from large-scale and high-dimensional data, which is usually formulated as an empirical risk minimization problem with a sparsity-inducing regularizer.
Stochastic Proximal Gradient Descent for Nuclear Norm Regularization
In this paper, we utilize stochastic optimization to reduce the space complexity of convex composite optimization with a nuclear norm regularizer, where the variable is a matrix of size $m \times n$.
Online Stochastic Linear Optimization under One-bit Feedback
In this paper, we study a special bandit setting of online stochastic linear optimization, where only one-bit of information is revealed to the learner at each round.
Towards Making High Dimensional Distance Metric Learning Practical
In this work, we present a dual random projection frame for DML with high dimensional data that explicitly addresses the limitation of dimensionality reduction for DML.
Fast Sparse Least-Squares Regression with Non-Asymptotic Guarantees
In this paper, we study a fast approximation method for {\it large-scale high-dimensional} sparse least-squares regression problem by exploiting the Johnson-Lindenstrauss (JL) transforms, which embed a set of high-dimensional vectors into a low-dimensional space.
An Explicit Sampling Dependent Spectral Error Bound for Column Subset Selection
In this paper, we consider the problem of column subset selection.
Analysis of Nuclear Norm Regularization for Full-rank Matrix Completion
To the best of our knowledge, this is first time such a relative bound is proved for the regularized formulation of matrix completion.
Theory of Dual-sparse Regularized Randomized Reduction
In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e. g., random projection, random hashing), for large-scale high-dimensional classification.
Binary Excess Risk for Smooth Convex Surrogates
In statistical learning theory, convex surrogates of the 0-1 loss are highly preferred because of the computational and theoretical virtues that convexity brings in.
Mixed Optimization for Smooth Functions
It is well known that the optimal convergence rate for stochastic optimization of smooth functions is $[O(1/\sqrt{T})]$, which is same as stochastic optimization of Lipschitz continuous convex functions.
Linear Convergence with Condition Number Independent Access of Full Gradients
For smooth and strongly convex optimization, the optimal iteration complexity of the gradient-based algorithm is $O(\sqrt{\kappa}\log 1/\epsilon)$, where $\kappa$ is the conditional number.
Beating the Minimax Rate of Active Learning with Prior Knowledge
Under the assumption that the norm of the optimal classifier that minimizes the convex risk is available, our analysis shows that the introduction of the convex surrogate loss yields an exponential reduction in the label complexity even when the parameter $\kappa$ of the Tsybakov noise is larger than $1$.
Optimal Stochastic Strongly Convex Optimization with a Logarithmic Number of Projections
We consider stochastic strongly convex optimization with a complex inequality constraint.
Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD)
Although stochastic gradient descent (SGD) has been successfully applied to improve the efficiency of DML, it can still be computationally expensive because in order to ensure that the solution is a PSD matrix, it has to, at every iteration, project the updated distance metric onto the PSD cone, an expensive operation.
O(logT) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions
Traditional algorithms for stochastic optimization require projecting the solution at each iteration into a given domain to ensure its feasibility.
Recovering the Optimal Solution by Dual Random Projection
Random projection has been widely used in data classification.
