Search Results for author:
Found 139 papers, 33 papers with code
Making Vision Transformers Efficient from A Token Sparsification View
In this work, we propose a novel Semantic Token ViT (STViT), for efficient global and local vision transformers, which can also be revised to serve as backbone for downstream tasks.
Power Time Series Forecasting by Pretrained LM
The diversity and domain dependence of time series data pose significant challenges in transferring learning to time series forecasting.
Free Lunch for Domain Adversarial Training: Environment Label Smoothing
A fundamental challenge for machine learning models is how to generalize learned models for out-of-distribution (OOD) data.
Ranked #2 on
Domain Adaptation
on Office-Home
HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition
To be specific, HyRSM++ consists of two key components, a hybrid relation module and a temporal set matching metric.
GLINKX: A Scalable Unified Framework For Homophilous and Heterophilous Graphs
On the other hand, shallow (or node-level) models using ego features and adjacency embeddings work well in heterophilous graphs.
FeDXL: Provable Federated Learning for Deep X-Risk Optimization
This problem has important applications in machine learning, e. g., AUROC maximization with a pairwise loss, and partial AUROC maximization with a compositional loss.
Grow and Merge: A Unified Framework for Continuous Categories Discovery
Although a number of studies are devoted to novel category discovery, most of them assume a static setting where both labeled and unlabeled data are given at once for finding new categories.
Robust Graph Structure Learning via Multiple Statistical Tests
A natural way to construct a graph among images is to treat each image as a node and assign pairwise image similarities as weights to corresponding edges.
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
While significant theoretical progress has been achieved, unveiling the generalization mystery of overparameterized neural networks still remains largely elusive.
TreeDRNet:A Robust Deep Model for Long Term Time Series Forecasting
Various deep learning models, especially some latest Transformer-based approaches, have greatly improved the state-of-art performance for long-term time series forecasting. However, those transformer-based models suffer a severe deterioration performance with prolonged input length, which prohibits them from using extended historical info. Moreover, these methods tend to handle complex examples in long-term forecasting with increased model complexity, which often leads to a significant increase in computation and less robustness in performance(e. g., overfitting).
An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation
With our empirical result obtained from 1, 330 models, we provide the following main observations: 1) ERM combined with data augmentation can achieve state-of-the-art performance if we choose a proper pre-trained model respecting the data property; 2) specialized algorithms further improve the robustness on top of ERM when handling a specific type of distribution shift, e. g., GroupDRO for spurious correlation and CORAL for large-scale out-of-distribution data; 3) Comparing different pre-training modes, architectures and data sizes, we provide novel observations about pre-training on distribution shift, which sheds light on designing or selecting pre-training strategy for different kinds of distribution shifts.
FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting
Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information.
Ranked #2 on
Time Series Forecasting
on ETTh1 (720)
Hybrid Relation Guided Set Matching for Few-shot Action Recognition
To overcome the two limitations, we propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components: hybrid relation module and set matching metric.
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation
Although data augmentation is widely used to enrich the training data, conventional methods with discrete manipulations fail to generate diverse and faithful training samples.
Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency
In this work, we aim to learn representations by leveraging more abundant information in untrimmed videos.
CHEX: CHannel EXploration for CNN Model Compression
However, conventional pruning methods have limitations in that: they are restricted to pruning process only, and they require a fully pre-trained large model.
Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks
Deep neural networks (DNNs) are known to be vulnerable to both backdoor attacks as well as adversarial attacks.
Entroformer: A Transformer-based Entropy Model for Learned Image Compression
One critical component in lossy deep image compression is the entropy model, which predicts the probability distribution of the quantized latent representation in the encoding and decoding modules.
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e. g. overall trend).
ELSA: Enhanced Local Self-Attention for Vision Transformer
Self-attention is powerful in modeling long-range dependencies, but it is weak in local finer-level feature learning.
Ranked #37 on
Instance Segmentation
on COCO minival
Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition
Decoupling spatiotemporal representation refers to decomposing the spatial and temporal features into dimension-independent factors.
A Novel Convergence Analysis for Algorithms of the Adam Family
Although rigorous convergence analysis exists for Adam, they impose specific requirements on the update of the adaptive step size, which are not generic enough to cover many other variants of Adam.
TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation
Unsupervised semantic segmentation aims to obtain high-level semantic representation on low-level visual features without manual annotations.
Ranked #1 on
Unsupervised Semantic Segmentation
on COCO-Stuff-171
(using extra training data)
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection
Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS).
Ranked #74 on
Object Detection
on COCO minival
Improved Fine-Tuning by Better Leveraging Pre-Training Data
The generalization result of using pre-training data shows that the excess risk bound on a target task can be improved when the appropriate pre-training data is included in fine-tuning.
Self-Supervised Pre-Training for Transformer-Based Person Re-Identification
We first investigate self-supervised learning (SSL) methods with Vision Transformer (ViT) pretrained on unlabelled person images (the LUPerson dataset), and empirically find it significantly surpasses ImageNet supervised pre-training models on ReID tasks.
Ranked #1 on
Unsupervised Person Re-Identification
on Market-1501
(using extra training data)
Achieving Human Parity on Visual Question Answering
The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image.
Ranked #11 on
Visual Question Answering (VQA)
on VQA v2 test-dev
BINAS: Bilinear Interpretable Neural Architecture Search
Practical use of neural networks often involves requirements on latency, energy and memory among others.
Rethinking Supervised Pre-training for Better Downstream Transferring
Though for most cases, the pre-training stage is conducted based on supervised methods, recent works on self-supervised pre-training have shown powerful transferability and even outperform supervised pre-training on multiple downstream tasks.
Unsupervised Domain Adaptation By Optimal Transportation Of Clusters Between Domains
Unsupervised domain adaptation (UDA) aims to transfer the knowledge from a labeled source domain to an unlabeled target domain.
Hierarchical Cross Contrastive Learning of Visual Representations
In this work, we propose a novel approach termed Hierarchical Cross Contrastive Learning(HCCL) to further distill the information mismatched by the conventional contrastive loss.
ZenDet: Revisiting Efficient Object Detection Backbones from Zero-Shot Neural Architecture Search
Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS).
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment, respectively.
Ranked #1 on
Domain Adaptation
on Office-31
Scaled ReLU Matters for Training Vision Transformers
In this paper, we further investigate this problem and extend the above conclusion: only early convolutions do not help for stable training, but the scaled ReLU operation in the \textit{convolutional stem} (\textit{conv-stem}) matters.
Dash: Semi-Supervised Learning with Dynamic Thresholding
In this work we develop a simple yet powerful framework, whose key idea is to select a subset of training examples from the unlabeled data when performing existing SSL methods so that only the unlabeled examples with pseudo labels related to the labeled data will be used to train models.
ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
The visualizations show that ParamCrop adaptively controls the center distance and the IoU between two augmented views, and the learned change in the disparity along the training process is beneficial to learning a strong representation.
Graph Convolution for Re-ranking in Person Re-identification
In this work, we propose a graph-based re-ranking method to improve learned features while still keeping Euclidean distance as the similarity metric.
Communication Efficient SGD via Gradient Sampling With Bayes Prior
Different from all of them, we regard large and small gradients selection as the exploitation and exploration of gradient information, respectively.
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
It addresses the shortcomings of the previous works by repeatedly growing a subset of layers to dense and then pruning them back to sparse after some training.
KVT: k-NN Attention for Boosting Vision Transformers
A key component in vision transformers is the fully-connected self-attention which is more powerful than CNNs in modelling long range dependencies.
Unsupervised Visual Representation Learning by Online Constrained K-Means
Clustering is to assign each instance a pseudo label that will be used to learn representations in discrimination.
Ranked #36 on
Self-Supervised Image Classification
on ImageNet
Why Does Multi-Epoch Training Help?
Stochastic gradient descent (SGD) has become the most attractive optimization method in training large-scale deep neural networks due to its simplicity, low computational cost in each updating step, and good performance.
A Novel Convergence Analysis for Algorithms of the Adam Family and Beyond
Our analysis exhibits that an increasing or large enough "momentum" parameter for the first-order moment used in practice is sufficient to ensure Adam and its many variants converge under a mild boundness condition on the adaptive scaling factor of the step size.
Learning Position and Target Consistency for Memory-based Video Object Segmentation
To address this limitation, we propose to Learn position and target Consistency framework for Memory-based video object segmentation, termed as LCM.
One-shot visual object segmentation
Semantic Segmentation
+1
A Theoretical Analysis of Learning with Noisily Labeled Data
Noisy labels are very common in deep supervised learning.
Self-supervised Video Representation Learning by Context and Motion Decoupling
A key challenge in self-supervised video representation learning is how to effectively capture motion information besides context bias.
Self-supervised Motion Learning from Static Images
We furthermore introduce a static mask in pseudo motions to create local motion patterns, which forces the model to additionally locate notable motion areas for the correct classification. We demonstrate that MoSI can discover regions with large motion even without fine-tuning on the downstream datasets.
Visual Search at Alibaba
We hope visual search at Alibaba becomes more widely incorporated into today's commercial applications.
Large-Scale Training System for 100-Million Classification at Alibaba
In the last decades, extreme classification has become an essential topic for deep learning.
Large-Scale Visual Search with Binary Distributed Graph at Alibaba
For a deployed visual search system with several billions of online images in total, building a billion-scale offline graph in hours is essential, which is almost unachievable by most existing methods.
Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation Learning
However, scaling up the classification task from thousands of semantic labels to millions of instance labels brings specific challenges including 1) the large-scale softmax computation; 2) the slow convergence due to the infrequent visiting of instance samples; and 3) the massive number of negative classes that can be noisy.
Large Scale Long-tailed Product Recognition System at Alibaba
In this paper, we present a novel side information based large scale visual recognition co-training~(SICoT) system to deal with the long tail problem by leveraging the image related side information.
Virtual ID Discovery from E-commerce Media at Alibaba: Exploiting Richness of User Click Behavior for Visual Search Relevance
Benefiting from exploration of user click data, our networks are more effective to encode richer supervision and better distinguish real-shot images in terms of category and feature.
Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet.
Ranked #2 on
Neural Architecture Search
on ImageNet
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks
We show that convergence to a global minimum is guaranteed for networks with widths quadratic in the sample size and linear in their depth at a time logarithmic in both.
Schemes of Propagation Models and Source Estimators for Rumor Source Detection in Online Social Networks: A Short Survey of a Decade of Research
Recent years have seen various rumor diffusion models being assumed in detection of rumor source research of the online social network.
Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition
To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures.
Attentional-Biased Stochastic Gradient Descent
In this paper, we present a simple yet effective method (ABSGD) for addressing the data imbalance issue in deep learning.
Price Suggestion for Online Second-hand Items with Texts and Images
The goal of price prediction is to help sellers set effective and reasonable prices for their second-hand items with the images and text descriptions uploaded to the online platforms.
Vision-based Price Suggestion for Online Second-hand Items
Then, we design a vision-based price suggestion module which takes the extracted visual features along with some statistical item features from the shopping platform as the inputs to determine whether an uploaded item image is qualified for price suggestion by a binary classification model, and provide price suggestions for items with qualified images by a regression model.
Learning Accurate Entropy Model with Global Reference for Image Compression
In this work, we propose a novel Global Reference Model for image compression to effectively leverage both the local and the global context information, leading to an enhanced compression rate.
WeMix: How to Better Utilize Data Augmentation
To this end, we develop two novel algorithms, termed "AugDrop" and "MixLoss", to correct the data bias in the data augmentation.
Neural Architecture Design for GPU-Efficient Networks
To address this issue, we propose a general principle for designing GPU-efficient networks based on extensive empirical studies.
Towards Understanding Label Smoothing
Label smoothing regularization (LSR) has a great success in training deep neural networks by stochastic algorithms such as stochastic gradient descent and its variants.
An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives
In this paper, we propose a practical online method for solving a class of distributionally robust optimization (DRO) with non-convex objectives, which has important applications in machine learning for improving the robustness of neural networks.
Weakly Supervised Representation Learning with Coarse Labels
To mitigate this challenge, we propose an algorithm to learn the fine-grained patterns for the target task, when only its coarse-class labels are available.
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling
The set of triplet constraints has to be sampled within the mini-batch.
Ranked #18 on
Metric Learning
on CUB-200-2011
(using extra training data)
DR Loss: Improving Object Detection by Distributional Ranking
This architecture is efficient but can suffer from the imbalance issue with respect to two aspects: the inter-class imbalance between the number of candidates from foreground and background classes and the intra-class imbalance in the hardness of background candidates, where only a few candidates are hard to be identified.
XNAS: Neural Architecture Search with Expert Advice
This paper introduces a novel optimization method for differential neural architecture search, based on the theory of prediction with expert advice.
Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement
We validate the superiority of the proposed method in our real-time high precision positioning system against several popular state-of-the-art robust regression methods.
On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization
We show that for stochastic non-convex optimization under the P-L condition, the classical data-parallel SGD with exponentially increasing batch sizes can achieve the fastest known $O(1/(NT))$ convergence with linear speedup using only $\log(T)$ communication rounds.
On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization
Recent developments on large-scale distributed machine learning applications, e. g., deep neural networks, benefit enormously from the advances in distributed non-convex optimization techniques, e. g., distributed Stochastic Gradient Descent (SGD).
Shrinking the Upper Confidence Bound: A Dynamic Product Selection Problem for Urban Warehouses
In this paper, we study algorithms for dynamically identifying a large number of products (i. e., SKUs) with top customer purchase probabilities on the fly, from an ocean of potential products to offer on retailers' ultra-fast delivery platforms.
Which Factorization Machine Modeling is Better: A Theoretical Answer with Optimal Guarantee
This bound is sub-optimal comparing to the information theoretical lower bound $\mathcal{O}(kd)$.
Stagewise Training Accelerates Convergence of Testing Error Over SGD
For convex loss functions and two classes of "nice-behaviored" non-convex objectives that are close to a convex function, we establish faster convergence of stagewise training than the vanilla SGD under the PL condition on both training error and testing error.
Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence
In this paper, we propose new stochastic optimization algorithms and study their first-order convergence theories for solving a broad family of DC functions.
Large-scale Distance Metric Learning with Uncertainty
Furthermore, we can show that the metric is learned from latent examples only, but it can preserve the large margin property even for the original data.
Learning with Non-Convex Truncated Losses by SGD
Learning with a {\it convex loss} function has been a dominating paradigm for many years.
Robust Optimization over Multiple Domains
Hence, we propose to learn the model and the adversarial distribution simultaneously with the stochastic algorithm for efficiency.
BIG-bench Machine Learning
Fine-Grained Visual Categorization
Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions
Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set.
Multinomial Logit Bandit with Linear Utility Functions
In each round, the player selects a $K$-cardinality subset from $N$ candidate items, and receives a reward which is governed by a {\it multinomial logit} (MNL) choice model considering both item utility and substitution property among items.
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization
Accelerated gradient (AG) methods are breakthroughs in convex optimization, improving the convergence rate of the gradient descent method for optimization with smooth functions.
First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time
Two classes of methods have been proposed for escaping from saddle points with one using the second-order information carried by the Hessian and the other adding the noise into the first-order information.
Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM
Although deep learning models are highly effective for various learning tasks, their high computational costs prohibit the deployment to scenarios where either memory or computational resources are limited.
Missing Modalities Imputation via Cascaded Residual Autoencoder
To leverage the valuable information in the corrupted data, we propose to impute the missing data by leveraging the relatedness among different modalities.
Empirical Risk Minimization for Stochastic Convex Optimization: $O(1/n)$- and $O(1/n^2)$-type of Risk Bounds
First, we establish an $\widetilde{O}(d/n + \sqrt{F_*/n})$ risk bound when the random function is nonnegative, convex and smooth, and the expected function is Lipschitz continuous, where $d$ is the dimensionality of the problem, $n$ is the number of samples, and $F_*$ is the minimal risk.
Dynamic Regret of Strongly Adaptive Methods
To cope with changing environments, recent developments in online learning have introduced the concepts of adaptive regret and dynamic regret independently.
Improved Dynamic Regret for Non-degenerate Functions
When multiple gradients are accessible to the learner, we first demonstrate that the dynamic regret of strongly convex functions can be upper bounded by the minimum of the path-length and the squared path-length.
Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient
Secondly, we present a lower bound with noisy gradient feedback and then show that we can achieve optimal dynamic regrets under a stochastic gradient feedback and two-point bandit feedback.
Similarity Learning via Adaptive Regression and Its Application to Image Retrieval
The similarity between pairs of images can be measured by the distances between their high dimensional representations, and the problem of learning the appropriate similarity is often addressed by distance metric learning.
Sparse Learning for Large-scale and High-dimensional Data: A Randomized Convex-concave Optimization Approach
In this paper, we develop a randomized algorithm and theory for learning a sparse model from large-scale and high-dimensional data, which is usually formulated as an empirical risk minimization problem with a sparsity-inducing regularizer.
Stochastic Proximal Gradient Descent for Nuclear Norm Regularization
In this paper, we utilize stochastic optimization to reduce the space complexity of convex composite optimization with a nuclear norm regularizer, where the variable is a matrix of size $m \times n$.
Online Stochastic Linear Optimization under One-bit Feedback
In this paper, we study a special bandit setting of online stochastic linear optimization, where only one-bit of information is revealed to the learner at each round.
Towards Making High Dimensional Distance Metric Learning Practical
In this work, we present a dual random projection frame for DML with high dimensional data that explicitly addresses the limitation of dimensionality reduction for DML.
Fast Sparse Least-Squares Regression with Non-Asymptotic Guarantees
In this paper, we study a fast approximation method for {\it large-scale high-dimensional} sparse least-squares regression problem by exploiting the Johnson-Lindenstrauss (JL) transforms, which embed a set of high-dimensional vectors into a low-dimensional space.
An Explicit Sampling Dependent Spectral Error Bound for Column Subset Selection
In this paper, we consider the problem of column subset selection.
Analysis of Nuclear Norm Regularization for Full-rank Matrix Completion
To the best of our knowledge, this is first time such a relative bound is proved for the regularized formulation of matrix completion.
Theory of Dual-sparse Regularized Randomized Reduction
In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e. g., random projection, random hashing), for large-scale high-dimensional classification.
Extracting Certainty from Uncertainty: Transductive Pairwise Classification from Pairwise Similarities
In this work, we study the problem of transductive pairwise classification from pairwise similarities~\footnote{The pairwise similarities are usually derived from some side information instead of the underlying class labels.}.
CUR Algorithm for Partially Observed Matrices
In particular, the proposed algorithm computes the low rank approximation of the target matrix based on (i) the randomly sampled rows and columns, and (ii) a subset of observed entries that are randomly sampled from the matrix.
Top Rank Optimization in Linear Time
Recent efforts of bipartite ranking are focused on optimizing ranking accuracy at the top of the ranked list.
On Data Preconditioning for Regularized Loss Minimization
In this work, we study data preconditioning, a well-known and long-existing technique, for boosting the convergence of first-order methods for regularized loss minimization.
CUR Algorithm with Incomplete Matrix Observation
Our goal is to develop a low rank approximation algorithm, similar to CUR, based on (i) randomly sampled rows and columns from A, and (ii) randomly sampled entries from A.
Scalable Kernel Clustering: Approximate Kernel k-means
Kernel-based clustering algorithms have the ability to capture the non-linear structure in real world data.
Binary Excess Risk for Smooth Convex Surrogates
In statistical learning theory, convex surrogates of the 0-1 loss are highly preferred because of the computational and theoretical virtues that convexity brings in.
Fine-Grained Visual Categorization via Multi-stage Metric Learning
To this end, we proposed a multi-stage metric learning framework that divides the large-scale high dimensional learning problem to a series of simple subproblems, achieving $\mathcal{O}(d)$ computational complexity.
Excess Risk Bounds for Exponentially Concave Losses
The overarching goal of this paper is to derive excess risk bounds for learning from exp-concave loss functions in passive and sequential learning settings.
Analysis of Distributed Stochastic Dual Coordinate Ascent
Extraordinary performances have been observed and reported for the well-motivated updates, as referred to the practical updates, compared to the naive updates.
Mixed Optimization for Smooth Functions
It is well known that the optimal convergence rate for stochastic optimization of smooth functions is $[O(1/\sqrt{T})]$, which is same as stochastic optimization of Lipschitz continuous convex functions.
Speedup Matrix Completion with Side Information: Application to Multi-Label Learning
In standard matrix completion theory, it is required to have at least $O(n\ln^2 n)$ observed entries to perfectly recover a low-rank matrix $M$ of size $n\times n$, leading to a large number of observations when $n$ is large.
Stochastic Convex Optimization with Multiple Objectives
It leverages on the theory of Lagrangian method in constrained optimization and attains the optimal convergence rate of $[O(1/ \sqrt{T})]$ in high probability for general Lipschitz continuous objectives.
Linear Convergence with Condition Number Independent Access of Full Gradients
For smooth and strongly convex optimization, the optimal iteration complexity of the gradient-based algorithm is $O(\sqrt{\kappa}\log 1/\epsilon)$, where $\kappa$ is the conditional number.
Stochastic Optimization of Smooth Loss
In this paper, we first prove a high probability bound rather than an expectation bound for stochastic optimization with smooth loss.
Beating the Minimax Rate of Active Learning with Prior Knowledge
Under the assumption that the norm of the optimal classifier that minimizes the convex risk is available, our analysis shows that the introduction of the convex surrogate loss yields an exponential reduction in the label complexity even when the parameter $\kappa$ of the Tsybakov noise is larger than $1$.
MixedGrad: An O(1/T) Convergence Rate Algorithm for Stochastic Smooth Optimization
It is well known that the optimal convergence rate for stochastic optimization of smooth functions is $O(1/\sqrt{T})$, which is same as stochastic optimization of Lipschitz continuous convex functions.
Compressed Hashing
Recent studies have shown that hashing methods are effective for high dimensional nearest neighbor search.
One-Pass AUC Optimization
AUC is an important performance measure and many algorithms have been devoted to AUC optimization, mostly by minimizing a surrogate convex loss on a training data set.
Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD)
Although stochastic gradient descent (SGD) has been successfully applied to improve the efficiency of DML, it can still be computationally expensive because in order to ensure that the solution is a PSD matrix, it has to, at every iteration, project the updated distance metric onto the PSD cone, an expensive operation.
O(logT) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions
Traditional algorithms for stochastic optimization require projecting the solution at each iteration into a given domain to ensure its feasibility.
Passive Learning with Target Risk
In this paper we consider learning in passive setting but with a slight modification.
Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning
One difficulty in learning the pairwise similarity measure is that there is a significant amount of noise and inter-worker variations in the manual annotations obtained via crowdsourcing.
Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison
Both random Fourier features and the Nyström method have been successfully applied to efficient kernel learning.
Stochastic Gradient Descent with Only One Projection
Although many variants of stochastic gradient descent have been proposed for large-scale convex optimization, most of them require projecting the solution at {\it each} iteration to ensure that the obtained solution stays within the feasible domain.
Online Stochastic Optimization with Multiple Objectives
We first propose a projection based algorithm which attains an $O(T^{-1/3})$ convergence rate.
Recovering the Optimal Solution by Dual Random Projection
Random projection has been widely used in data classification.
A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound
In this work, we develop a simple algorithm for semi-supervised regression.
Robust Metric Learning by Smooth Optimization
Most existing distance metric learning methods assume perfect side information that is usually given in pairwise or triplet constraints.
An Efficient Primal-Dual Prox Method for Non-Smooth Optimization
We study the non-smooth optimization problems in machine learning, where both the loss function and the regularizer are non-smooth functions.
Multi-label Multiple Kernel Learning by Stochastic Approximation: Application to Visual Object Recognition
Recent studies have shown that multiple kernel learning is very effective for object recognition, leading to the popularity of kernel learning in computer vision problems.
Active Learning by Querying Informative and Representative Examples
Most active learning approaches select either informative or representative unlabeled instances to query their labels.
Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering
Learning distance functions with side information plays a key role in many machine learning and data mining applications.
Adaptive Regularization for Transductive Support Vector Machine
In this framework, SVM and TSVM can be regarded as a learning machine without regularization and one with full regularization from the unlabeled data, respectively.
DUOL: A Double Updating Approach for Online Learning
This is clearly insufficient since when a new misclassified example is added to the pool of support vectors, we generally expect it to affect the weights for the existing support vectors.
Regularized Distance Metric Learning:Theory and Algorithm
In this paper, we examine the generalization error of regularized distance metric learning.
Learning to Rank by Optimizing NDCG Measure
Learning to rank is a relatively new field of study, aiming to learn a ranking function from a set of training data with relevancy labels.
Semi-supervised Learning with Weakly-Related Unlabeled Data : Towards Better Text Categorization
For empirical evaluation, we present a direct comparison with a number of state-of-the-art methods for inductive semi-supervised learning and text categorization; and we show that SSLW results in a significant improvement in categorization accuracy, equipped with a small training set and an unlabeled resource that is weakly related to the test beds."
An Extended Level Method for Efficient Multiple Kernel Learning
We consider the problem of multiple kernel learning (MKL), which can be formulated as a convex-concave problem.
Multi-label Multiple Kernel Learning
We present a multi-label multiple kernel learning (MKL) formulation, in which the data are embedded into a low-dimensional space directed by the instance-label correlations encoded into a hypergraph.
Efficient Convex Relaxation for Transductive Support Vector Machine
We consider the problem of Support Vector Machine transduction, which involves a combinatorial problem with exponential computational complexity in the number of unlabeled examples.
