Search Results for author:
Found 365 papers, 110 papers with code
A General Boosting Method and its Application to Learning Ranking Functions for Web Search
We present a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems.
The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information
We present Epoch-Greedy, an algorithm for multi-armed bandits with observable side information.
A Spectral Algorithm for Learning Hidden Markov Models
Hidden Markov Models (HMMs) are one of the most fundamental and widely used statistical tools for modeling discrete time series.
Sparse Online Learning via Truncated Gradient
We propose a general method called truncated gradient to induce sparsity in the weights of online-learning algorithms with convex loss.
Multi-stage Convex Relaxation for Learning with Sparse Regularization
We study learning formulations with non-convex regularizaton that are natural for sparse linear models.
Adaptive Forward-Backward Greedy Algorithm for Sparse Learning with Linear Models
Consider linear prediction models where the target function is a sparse linear combination of a set of basis functions.
Nonlinear Learning using Local Coordinate Coding
This paper introduces a new method for semi-supervised learning on high dimensional nonlinear manifolds, which includes a phase of unsupervised basis learning and a phase of supervised function learning.
Agnostic Active Learning Without Constraints
We present and analyze an agnostic active learning algorithm that works without keeping a version space.
Deep Coding Network
This paper proposes a principled extension of the traditional single-layer flat sparse coding scheme, where a two-layer coding scheme is derived based on theoretical analysis of nonlinear functional approximation that extends recent results for local coordinate coding.
Random design analysis of ridge regression
The analysis also reveals the effect of errors in the estimated covariance structure, as well as the effect of modeling errors, neither of which effects are present in the fixed design setting.
Learning Nonlinear Functions Using Regularized Greedy Forest
We consider the problem of learning a forest of nonlinear decision rules with general loss functions.
Spectral Methods for Learning Multivariate Latent Tree Structure
The setting is one where we only have samples from certain observed variables in the tree, and our goal is to estimate the tree structure (i. e., the graph of how the underlying hidden variables are connected to each other and to the observed variables).
Learning to Search Efficiently in High Dimensions
Traditional approaches relied on algorithmic constructions that are often data independent (such as Locality Sensitive Hashing) or weakly dependent (such as kd-trees, k-means trees).
Greedy Model Averaging
The purpose of this paper is to present a new greedy model averaging procedure that improves EWMA.
Selective Labeling via Error Bound Minimization
In particular, we derive a deterministic generalization error bound for LapRLS trained on subsampled data, and propose to select a subset of data points to label by minimizing this upper bound.
Learning Sparse Low-Threshold Linear Classifiers
We consider the problem of learning a non-negative linear classifier with a $1$-norm of at most $k$, and a fixed threshold, under the hinge-loss.
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent
Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning.
Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
In particular, our analysis improves upon existing results by providing a more refined sample complexity bound as well as an exact support recovery result for the final estimator.
Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure.
High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning
The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks.
Compressed Counting Meets Compressed Sensing
In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.
Aggregation of Affine Estimators
While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability.
Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics
We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e. g., Mercer kernels.
Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization
Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Stochastic gradient descent is popular for large scale optimization but has slow convergence asymptotically due to the inherent variance.
Adaptive Stochastic Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers (ADMM) has been studied for years.
Communication Efficient Distributed Optimization using an Approximate Newton-type Method
We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems.
Sparse Recovery with Very Sparse Compressed Counting
In this paper, we adopt very sparse Compressed Counting for nonnegative signal recovery.
Stochastic Optimization with Importance Sampling
Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA).
A Proximal Stochastic Gradient Method with Progressive Variance Reduction
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping.
Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling
Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks.
Randomized Dual Coordinate Ascent with Arbitrary Sampling
The distributed variant of Quartz is the first distributed SDCA-like method with an analysis for non-separable data.
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data.
Ranked #29 on
Sentiment Analysis
on IMDb
Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory
This is the first result on the computational and statistical guarantees of the pathwise coordinate optimization framework in high dimensions.
Adjusting Leverage Scores by Row Weighting: A Practical Approach to Coherent Matrix Completion
Low-rank matrix completion is an important problem with extensive real-world applications.
Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition
The Nystr\"om method is a special instance of our fast model and is approximation to the prototype model.
Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
This paper presents a new semi-supervised framework with convolutional neural networks (CNNs) for text categorization.
Ranked #1000000000 on
Text Classification
on IMDb
Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
To recover $\beta^*$, we propose an $\ell_1$-regularized least-squares estimator.
Quartz: Randomized Dual Coordinate Ascent with Arbitrary Sampling
We study the problem of minimizing the average of a large number of smooth convex functions penalized with a strongly convex regularizer.
Local Smoothness in Variance Reduced Optimization
Abstract We propose a family of non-uniform sampling strategies to provably speed up a class of stochastic optimization algorithms with linear convergence including Stochastic Variance Reduced Gradient (SVRG) and Stochastic Dual Coordinate Ascent (SDCA).
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data.
Ranked #1 on
Text Classification
on RCV1
Near-Optimal Stochastic Approximation for Online Principal Component Estimation
We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm.
A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines.
Local Uncertainty Sampling for Large-Scale Multi-Class Logistic Regression
A major challenge for building statistical models in the big data era is that the available data volume far exceeds the computational capability.
Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
Sparse generalized eigenvalue problem (GEP) plays a pivotal role in a large family of high-dimensional statistical models, including sparse Fisher's discriminant analysis, canonical correlation analysis, and sufficient dimension reduction.
Efficient Distributed Learning with Sparsity
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines.
Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level
This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016).
Exact Recovery of Hard Thresholding Pursuit
In this paper, we bridge this gap by showing, for the first time, that exact recovery of the global sparse minimizer is possible for HTP-style methods under restricted strong condition number bounding conditions.
Learning Additive Exponential Family Graphical Models via \ell_{2,1}-norm Regularized M-Estimation
We investigate a subclass of exponential family graphical models of which the sufficient statistics are defined by arbitrary additive forms.
Multi-scale Convolutional Neural Networks for Crowd Counting
Crowd counting on static images is a challenging problem due to scale variations.
Spatial-Temporal Recurrent Neural Network for Emotion Recognition
Then a bi-directional temporal RNN layer is further used to learn discriminative temporal dependencies from the sequences concatenating spatial features of each time slice produced from the spatial RNN layer.
Robust Frequent Directions with Application in Online Learning
We also apply RFD to online learning and propose an effective hyperparameter-free online Newton algorithm.
Deep manifold-to-manifold transforming network for action recognition
Symmetric positive definite (SPD) matrices (e. g., covariances, graph Laplacians, etc.)
End-to-end Active Object Tracking via Reinforcement Learning
We study active object tracking, where a tracker takes as input the visual observation (i. e., frame sequence) and produces the camera control signal (e. g., move forward, turn left, etc.).
Graphical Nonconvex Optimization for Optimal Estimation in Gaussian Graphical Models
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations
We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed.
Deep Pyramid Convolutional Neural Networks for Text Categorization
This paper proposes a low-complexity word-level deep convolutional neural network (CNN) architecture for text categorization that can efficiently represent long-range associations in text.
Ranked #2 on
Sentiment Analysis
on Amazon Review Full
Projection-free Distributed Online Learning in Networks
The conditional gradient algorithm has regained a surge of research interest in recent years due to its high efficiency in handling large-scale machine learning problems.
Deep Subspace Clustering Networks
We present a novel deep neural network architecture for unsupervised subspace clustering.
Ranked #3 on
Image Clustering
on Extended Yale-B
Gradient Sparsification for Communication-Efficient Distributed Optimization
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures.
Candidates vs. Noises Estimation for Large Multi-Class Classification Problem
This paper proposes a method for multi-class classification problems, where the number of classes K is large.
Learning to Remember Translation History with a Continuous Cache
Existing neural machine translation (NMT) models generally translate sentences in isolation, missing the opportunity to take advantage of document-level information.
Efficient Optimization for Linear Dynamical Systems with Applications to Clustering and Sparse Coding
Linear Dynamical Systems (LDSs) are fundamental tools for modeling spatio-temporal data in various disciplines.
On Quadratic Convergence of DC Proximal Newton Algorithm in Nonconvex Sparse Learning
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
PARAMETRIZED DEEP Q-NETWORKS LEARNING: PLAYING ONLINE BATTLE ARENA WITH DISCRETE-CONTINUOUS HYBRID ACTION SPACE
Most existing deep reinforcement learning (DRL) frameworks consider action spaces that are either discrete or continuous space.
Translating Pro-Drop Languages with Reconstruction Models
Next, the annotated source sentence is reconstructed from hidden representations in the NMT model.
Composite Functional Gradient Learning of Generative Adversarial Models
This paper first presents a theory for generative adversarial methods that does not rely on the traditional minimax formulation.
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Neural Stereoscopic Image Style Transfer
While each view of the stereoscopic pair is processed in an individual path, a novel feature aggregation strategy is proposed to effectively share information between the two paths.
Communication Compression for Decentralized Training
In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?}
Tensor graph convolutional neural network
For cross graph convolution, a parameterized Kronecker sum operation is proposed to generate a conjunctive adjacency matrix characterizing the relationship between every pair of nodes across two subgraphs.
Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
Such supervision, while labor-intensive and not always possible, tends to hinder the generalization ability of the learned models.
Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset
First, we demonstrate that video attractiveness and different engagements present different relationships.
Walk-Steered Convolution for Graph Classification
As an analogy to a standard convolution kernel on image, Gaussian models implicitly coordinate those unordered vertices/nodes and edges in a local receptive field after projecting to the gradient space of Gaussian parameters.
QuaSE: Accurate Text Style Transfer under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method
Specifically, we find that a large class of primal and primal-dual operator splitting algorithms are all special cases of VMOR-HPE.
Safe Element Screening for Submodular Function Minimization
Relying on this study, we subsequently propose a novel safe screening method to quickly identify the elements guaranteed to be included (we refer to them as active) or excluded (inactive) in the final optimal solution of SFM during the optimization process.
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization
Large-scale distributed optimization is of great importance in various applications.
Graphical Nonconvex Optimization via an Adaptive Convex Relaxation
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose two new algorithms, namely SPIDER-SFO and SPIDER-SFO\textsuperscript{+}, that solve non-convex stochastic optimization problems using stochastic gradients only.
When Work Matters: Transforming Classical Network Structures to Graph CNN
In this paper, we aim to give a comprehensive analysis of when work matters by transforming different classical network structures to graph CNN, particularly in the basic graph recognition problem.
Ranked #3 on
Graph Classification
on IMDb-B
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
Recent studies on unsupervised image-to-image translation have made a remarkable progress by training a pair of generative adversarial networks with a cycle-consistent loss.
Recurrent Fusion Network for Image Captioning
In this paper, in order to exploit the complementary information from multiple encoders, we propose a novel Recurrent Fusion Network (RFNet) for tackling image captioning.
Video Re-localization
We first exploit and reorganize the videos in ActivityNet to form a new dataset for video re-localization research, which consists of about 10, 000 videos of diverse visual appearances associated with localized boundary information.
End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning
We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training.
Diffusion Approximations for Online Principal Component Estimation and Global Convergence
In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis.
Adaptive Sampling Towards Fast Graph Representation Learning
Graph Convolutional Networks (GCNs) have become a crucial tool on learning representations of graph vertices.
Ranked #2 on
Node Classification
on Pubmed Full-supervised
A convex formulation for high-dimensional sparse sliced inverse regression
Sliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates.
TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game
Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting.
Fully Implicit Online Learning
Regularized online learning is widely used in machine learning applications.
DOMAIN ADAPTATION VIA DISTRIBUTION AND REPRESENTATION MATCHING: A CASE STUDY ON TRAINING DATA SELECTION VIA REINFORCEMENT LEARNING
Following the TDS methodology, in this paper, we propose a general data selection framework with representation learning and distribution matching simultaneously for domain adaptation on neural models.
QuaSE: Sequence Editing under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space
Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely.
Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry
Our approach is able to handle the rolling-shutter effects and imperfect sensor synchronization in a unified way.
Orthogonal Deep Features Decomposition for Age-Invariant Face Recognition
Extensive experiments conducted on the three public domain face aging datasets (MORPH Album 2, CACD-VS and FG-NET) have shown the effectiveness of the proposed approach and the value of the constructed CAF dataset on AIFR.
Ranked #3 on
Age-Invariant Face Recognition
on MORPH Album2
Modeling Localness for Self-Attention Networks
Self-attention networks have proven to be of profound value for its strength of capturing global dependencies.
Ranked #29 on
Machine Translation
on WMT2014 English-German
Exploiting Deep Representations for Neural Machine Translation
Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers, which allows systems to model complex functions and capture complicated linguistic structures.
Multi-Head Attention with Disagreement Regularization
Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions.
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
We prove that the proposed method globally converges to a stationary point.
Scalable Deep $k$-Subspace Clustering
In this paper, we introduce a method that simultaneously learns an embedding space along subspaces within it to minimize a notion of reconstruction error, thus addressing the problem of subspace clustering in an end-to-end learning paradigm.
Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Super-Identity Convolutional Neural Network for Face Hallucination
Face hallucination is a generative task to super-resolve the facial image with low resolution while human perception of face heavily relies on identity information.
Neural Machine Translation with Adequacy-Oriented Learning
Although Neural Machine Translation (NMT) models have advanced state-of-the-art performance in machine translation, they face problems like the inadequate translation.
Ranked #35 on
Machine Translation
on WMT2014 English-German
Cross-database non-frontal facial expression recognition based on transductive deep transfer learning
Cross-database non-frontal expression recognition is a very meaningful but rather difficult subject in the fields of computer vision and affect computing.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Exponentially Weighted Imitation Learning for Batched Historical Data
We consider deep policy learning with only batched historical trajectories.
Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Stochastic Expectation Maximization with Variance Reduction
However, sEM has a slower asymptotic convergence rate than batch EM, and requires a decreasing sequence of step sizes, which is difficult to tune.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator
Specially, we prove that the SPIDER-SFO algorithm achieves a gradient computation cost of $\mathcal{O}\left( \min( n^{1/2} \epsilon^{-2}, \epsilon^{-3} ) \right)$ to find an $\epsilon$-approximate first-order stationary point.
Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents
This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Cross-Database Micro-Expression Recognition: A Benchmark
Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis.
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Zeroth-order optimization is an important research topic in machine learning.
Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model.
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points
In this paper, we give a sharp analysis for Stochastic Gradient Descent (SGD) and prove that SGD is able to efficiently escape from saddle points and find an $(\epsilon, O(\epsilon^{0. 5}))$-approximate second-order stationary point in $\tilde{O}(\epsilon^{-3. 5})$ stochastic gradient computations for generic nonconvex optimization problems, when the objective function satisfies gradient-Lipschitz, Hessian-Lipschitz, and dispersive noise assumptions.
Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
With the promising progress of deep neural networks, layer aggregation has been used to fuse information across layers in various fields, such as computer vision and machine translation.
Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
In this paper, we evaluate the robustness of state-of-the-art face recognition models in the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire hard-label predictions by sending queries to the target model.
Neural Collaborative Subspace Clustering
We introduce the Neural Collaborative Subspace Clustering, a neural model that discovers clusters of data points drawn from a union of low-dimensional subspaces.
DHER: Hindsight Experience Replay for Dynamic Goals
Dealing with sparse rewards is one of the most important challenges in reinforcement learning (RL), especially when a goal is dynamic (e. g., to grasp a moving object).
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques.
NATTACK: A STRONG AND UNIVERSAL GAUSSIAN BLACK-BOX ADVERSARIAL ATTACK
In other words, there is a population of adversarial examples, instead of only one, for any input to a DNN.
MAP Inference via L2-Sphere Linear Program Reformulation
Thus, LS-LP is equivalent to the original MAP inference problem.
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
For example, under the popular parameter server model for distributed learning, the worker nodes need to send the compressed local gradients to the parameter server, which performs the aggregation.
Estimating and Inferring the Maximum Degree of Stimulus-Locked Time-Varying Brain Connectivity Networks
To address this issue, neuroscientists have been measuring brain activity under natural viewing experiments in which the subjects are given continuous stimuli, such as watching a movie or listening to a story.
Reinforced Training Data Selection for Domain Adaptation
Supervised models suffer from the problem of domain shifting where distribution mismatch in the data across domains greatly affect model performance.
Optimal Feature Transport for Cross-View Image Geo-Localization
This paper proposes a novel Cross-View Feature Transport (CVFT) technique to explicitly establish cross-view domain transfer that facilitates feature alignment between ground and aerial images.
$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression
Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized learning to further reduce the communication cost."
Self-supervised Recurrent Neural Network for 4D Abdominal and In-utero MR Imaging
The results show that the proposed pipeline can accurately estimate the respiratory state and reconstruct 4D SR volumes with better or similar performance to the 3D SVR pipeline with less than 20\% sparsely selected slices.
Multi-objective Neural Architecture Search via Predictive Network Performance Optimization
Inspired by the nature of the graph structure of a neural network, we propose BOGCN-NAS, a NAS algorithm using Bayesian Optimization with Graph Convolutional Network (GCN) predictor.
Hierarchical Neural Architecture Search via Operator Clustering
Recently, the efficiency of automatic neural architecture design has been significantly improved by gradient-based search methods such as DARTS.
A Stochastic Extra-Step Quasi-Newton Method for Nonsmooth Nonconvex Optimization
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems.
Mirror Natural Evolution Strategies
We show that the estimated covariance matrix of MiNES converges to the inverse of Hessian matrix of the objective function with a sublinear convergence rate.
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations
Recently, over-parameterized neural networks have been extensively analyzed in the literature.
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data.
Ranked #1 on
Chinese Part-of-Speech Tagging
on CTB5 Dev
Chinese Named Entity Recognition
Chinese Word Segmentation
+5
Spatial Sparse subspace clustering for Compressive Spectral imaging
This paper aims at developing a clustering approach with spectral images directly from CASSI compressive measurements.
Sparse Coding on Cascaded Residuals
The residual at a layer is computed by the difference between the aggregated reconstructions of the previous layers and the downsampled original image at that layer.
Black-Box Adversarial Attack with Transferable Model-based Embedding
We present a new method for black-box adversarial attack.
Convex Formulation of Overparameterized Deep Neural Networks
This new analysis is consistent with empirical observations that deep neural networks are capable of learning efficient feature representations.
A Fast Sampling Gradient Tree Boosting Framework
As an adaptive, interpretable, robust, and accurate meta-algorithm for arbitrary differentiable loss functions, gradient tree boosting is one of the most popular machine learning techniques, though the computational expensiveness severely limits its usage.
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously.
Stable Learning via Sample Reweighting
We consider the problem of learning linear prediction models with model misspecification bias.
Dual-Attention Graph Convolutional Network
Graph convolutional networks (GCNs) have shown the powerful ability in text structure representation and effectively facilitate the task of text classification.
Divergence-Augmented Policy Optimization
In deep reinforcement learning, policy optimization methods need to deal with issues such as function approximation and the reuse of off-policy data.
Fast Generalized Matrix Regression with Applications in Machine Learning
Fast matrix algorithms have become the fundamental tools of machine learning in big data era.
LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition
In this paper, we first examine traditional active learning strategies in a specific case of BiLstm-CRF that has widely used in named entity recognition on several typical datasets.
Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problems
We consider nonconvex-concave minimax optimization problems of the form $\min_{\bf x}\max_{\bf y\in{\mathcal Y}} f({\bf x},{\bf y})$, where $f$ is strongly-concave in $\bf y$ but possibly nonconvex in $\bf x$ and ${\mathcal Y}$ is a convex and compact set.
Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization
Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e. g., sparsity) efficiently.
Graph Inference Learning for Semi-supervised Classification
In this work, we address semi-supervised classification of graph data, where the categories of those unlabeled nodes are inferred from labeled nodes as well as graph structures.
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks
In this paper, we provide a generalized neural tangent kernel analysis and show that noisy gradient descent with weight decay can still exhibit a "kernel-like" behavior.
Bidirectional Generative Modeling Using Adversarial Gradient Estimation
This paper considers the general $f$-divergence formulation of bidirectional generative modeling, which includes VAE and BiGAN as special cases.
Attention-guided Chained Context Aggregation for Semantic Segmentation
The way features propagate in Fully Convolutional Networks is of momentous importance to capture multi-scale contexts for obtaining precise segmentation masks.
Ranked #23 on
Semantic Segmentation
on SUN-RGBD
(using extra training data)
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
To remedy this, this paper proposes \mldas, a mixed-level reformulation for NAS that can be optimized efficiently and reliably.
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process.
Ranked #4 on
RGB-D Salient Object Detection
on LFSD
Keyphrase Generation with Cross-Document Attention
Keyphrase generation aims to produce a set of phrases summarizing the essentials of a given document.
NetML: A Challenge for Network Traffic Analytics
Classifying network traffic is the basis for important network applications.
Multi-consensus Decentralized Accelerated Gradient Descent
This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory.
Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Data
In this paper, we propose a data privacy-preserving and communication efficient distributed GAN learning framework named Distributed Asynchronized Discriminator GAN (AsynDGAN).
End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera
In this paper, we propose a monocular camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network.
Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python
We describe a new library named picasso, which implements a unified framework of pathwise coordinate optimization for a variety of sparse learning problems (e. g., sparse linear regression, sparse logistic regression, sparse Poisson regression and scaled sparse linear regression) combined with efficient active set selection strategies.
Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization
This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space.
Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-way Attentions of Auto-analyzed Knowledge
Chinese word segmentation (CWS) and part-of-speech (POS) tagging are important fundamental tasks for Chinese language processing, where joint learning of them is an effective one-step solution for both tasks.
Improving Chinese Word Segmentation with Wordhood Memory Networks
Contextual features always play an important role in Chinese word segmentation (CWS).
Ranked #1 on
Chinese Word Segmentation
on CITYU
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
This new representation overcomes the degenerate situation where all the hidden units essentially have only one meaningful hidden unit in each middle layer, and further leads to a simpler representation of DNNs, for which the training objective can be reformulated as a convex optimization problem via suitable re-parameterization.
CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
In spite of its remarkable progress, many algorithms are restricted to particular search spaces.
Ranked #13 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Towards Purely Unsupervised Disentanglement of Appearance and Shape for Person Images Generation
To be clear, in this paper, we refer unsupervised learning as learning without task-specific human annotations, pairs or any form of weak supervision.)
Graph Wasserstein Correlation Analysis for Movie Retrieval
Spectral graph filtering is introduced to encode graph signals, which are then embedded as probability distributions in a Wasserstein space, called graph Wasserstein metric learning.
Instance-Aware Graph Convolutional Network for Multi-Label Classification
For label diffusion of instance-awareness in graph convolution, rather than using the statistical label correlation alone, an image-dependent label correlation matrix (LCM), fusing both the statistical LCM and an individual one of each image instance, is constructed for graph inference on labels to inject adaptive information of label-awareness into the learned features of the model.
Interest-Behaviour Multiplicative Network for Resource-limited Recommendation
In this paper, we aim to mine the cue of user preferences in resource-limited recommendation tasks, for which purpose we specifically build a large used car transaction dataset possessing resource-limitation characteristics.
CorrAttack: Black-box Adversarial Attack with Structured Search
We show that searching over the structured space can be approximated by a time-varying contextual bandits problem, where the attacker takes feature of the associated arm to make modifications of the input, and receives an immediate reward as the reduction of the loss function.
Weakly Supervised Disentangled Generative Causal Representation Learning
This paper proposes a Disentangled gEnerative cAusal Representation (DEAR) learning method under appropriate supervised information.
DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks
Specifically, we search for a meta graph, which can capture more complex semantic relations than a meta path, to determine how graph neural networks (GNNs) propagate messages along different types of edges.
Improving Constituency Parsing with Span Attention
Constituency parsing is a fundamental and important task for natural language understanding, where a good representation of contextual information can help this task.
Ranked #1 on
Constituency Parsing
on ATB
Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
To this end, we summarize a tiny formula for downsizing neural architectures through a series of smaller models derived from the EfficientNet-B0 with the FLOPs constraint.
Ranked #692 on
Image Classification
on ImageNet
VEGA: Towards an End-to-End Configurable AutoML Pipeline
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models.
Decentralized Accelerated Proximal Gradient Descent
In this paper, we propose a new method which establishes the optimal computational complexity and a near optimal communication complexity.
Model Rubik’s Cube: Twisting Resolution, Depth and Width for TinyNets
To this end, we summarize a tiny formula for downsizing neural architectures through a series of smaller models derived from the EfficientNet-B0 with the FLOPs constraint.
How to Characterize The Landscape of Overparameterized Convolutional Neural Networks
With the help of a new technique called {\it neural network grafting}, we demonstrate that even during the entire training process, feature distributions of differently initialized networks remain similar at each layer.
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts
In particular, we propose a novel joint-training framework to train plain CNN by leveraging the gradients of the ResNet counterpart.
Evidence of topological nodal lines and surface states in the centrosymmetric superconductor SnTaS2
Utilizing angle-resolved photoemission spectroscopy and first-principles calculations, here, we demonstrate the existence of topological nodal-line states and drumheadlike surface states in centrosymmetric superconductor SnTaS2, which is a type-II superconductor with a critical transition temperature of about 3 K. The valence bands from Ta 5d orbitals and the conduction bands from Sn 5p orbitals cross each other, forming two nodal lines in the vicinity of the Fermi energy without the inclusion of spin-orbit coupling (SOC), protected by the spatial-inversion symmetry and time-reversal symmetry.
Superconductivity
Multi-modal AsynDGAN: Learn From Distributed Medical Image Data without Sharing Private Information
As deep learning technologies advance, increasingly more data is necessary to generate general and robust models for various tasks.
Mathematical Models of Overparameterized Neural Networks
Deep learning has received considerable empirical successes in recent years.
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
This paper considers the decentralized composite optimization problem.
Graph Deformer Network
In this paper, we propose a simple yet effective graph deformer network (GDN) to fulfill anisotropic convolution filtering on graphs, analogous to the standard convolution operation on images.
Invariant Batch Normalization for Multi-source Domain Generalization
We consider the domain generalization problem, where the test domain differs from the training domain.
TransNAS-Bench-101: Improving Transferrability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on one single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
Effective Training of Sparse Neural Networks under Global Sparsity Constraint
An appealing feature of ProbMask is that the amounts of weight redundancy can be learned automatically via our constraint and thus we avoid the problem of tuning pruning rates individually for different layers in a network.
Wasserstein Coupled Graph Learning for Cross-Modal Retrieval
Then, a Wasserstein coupled dictionary, containing multiple pairs of counterpart graph keys with each key corresponding to one modality, is constructed for further feature learning.
On Secure Degrees of Freedom of the MIMO Interference Channel with Local Output Feedback
This paper studies the problem of sum-secure degrees of freedom (SDoF) of the (M, M, N, N) multiple-input multiple-output (MIMO) interference channel with local output feedback, so as to build an information-theoretic foundation and provide practical transmission schemes for 6G-enabled vehicles-to-vehicles (V2V).
Information Theory Information Theory
Nondiscriminatory Treatment: a straightforward framework for multi-human parsing
In inference time, we design a brand-new grouping post-processing method that relates each part instance with one single human instance and groups them together to obtain the final human-level parsing result.
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
This leads to a decentralized PCA algorithm called \texttt{DeEPCA}, which has a convergence rate similar to that of the centralized PCA, while achieving the best communication complexity among existing decentralized PCA algorithms.
Siamese Labels Auxiliary Learning
In general, the main work of this paper include: (1) propose SiLa Learning, which improves the performance of common models without increasing test parameters; (2) compares SiLa with DML and proves that SiLa can improve the generalization of the model; (3) SiLa is applied to Dynamic Neural Networks, and proved that SiLa can be used for various types of network structures.
Towards Unbiased COVID-19 Lesion Localisation and Segmentation via Weakly Supervised Learning
Despite tremendous efforts, it is very challenging to generate a robust model to assist in the accurate quantification assessment of COVID-19 on chest CT images.
Spatial-Temporal Tensor Graph Convolutional Network for Traffic Prediction
We further extend graph convolution into tensor space and propose a tensor graph convolution network to extract more discriminating features from spatial-temporal graph data.
Ranked #1 on
Traffic Prediction
on SZ-Taxi
Involution: Inverting the Inherence of Convolution for Visual Recognition
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision.
Ranked #703 on
Image Classification
on ImageNet
Few-Shot Human Motion Transfer by Personalized Geometry and Texture Modeling
We present a new method for few-shot human motion transfer that achieves realistic human image generation with only a small number of appearance inputs.
Uncertainty-aware Joint Salient Object and Camouflaged Object Detection
Visual salient object detection (SOD) aims at finding the salient object(s) that attract human attention, while camouflaged object detection (COD) on the contrary intends to discover the camouflaged object(s) that hidden in the surrounding.
Modeling Object Dissimilarity for Deep Saliency Prediction
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects.
Reinforced Attention for Few-Shot Learning and Beyond
Few-shot learning aims to correctly recognize query samples from unseen classes given a limited number of support samples, often by relying on global embeddings of images.
Exploring Geometric Consistency for Monocular 3D Object Detection
In addition, we demonstrate that the augmentation methods are well suited for semi-supervised training and cross-dataset generalization.
Effective Sparsification of Neural Networks with Global Sparsity Constraint
Weight pruning is an effective technique to reduce the model size and inference time for deep neural networks in real-world deployments.
ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders
Pre-trained text encoders have drawn sustaining attention in natural language processing (NLP) and shown their capability in obtaining promising results in different tasks.
