Search Results for author:
Found 366 papers, 113 papers with code
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data.
Ranked #30 on
Sentiment Analysis
on IMDb
Exponentially Weighted Imitation Learning for Batched Historical Data
We consider deep policy learning with only batched historical trajectories.
Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
To this end, we summarize a tiny formula for downsizing neural architectures through a series of smaller models derived from the EfficientNet-B0 with the FLOPs constraint.
Ranked #695 on
Image Classification
on ImageNet
Adaptive Sampling Towards Fast Graph Representation Learning
Graph Convolutional Networks (GCNs) have become a crucial tool on learning representations of graph vertices.
Ranked #2 on
Node Classification
on Pubmed Full-supervised
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
Utilizing a reward model and a sufficient number of samples, our approach selects the high-quality samples, discarding those that exhibit undesired behavior, and subsequently enhancing the model by fine-tuning on these filtered samples.
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
As the number of available models and specialized tasks keeps growing, the job of general finetuning becomes highly nontrivial.
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
Attempting to complement this deficiency, we investigate layerwise properties of LoRA on fine-tuning tasks and observe an uncommon skewness of weight norms across different layers.
Involution: Inverting the Inherence of Convolution for Visual Recognition
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision.
Ranked #706 on
Image Classification
on ImageNet
Model Rubik’s Cube: Twisting Resolution, Depth and Width for TinyNets
To this end, we summarize a tiny formula for downsizing neural architectures through a series of smaller models derived from the EfficientNet-B0 with the FLOPs constraint.
Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model.
Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space
Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely.
PARAMETRIZED DEEP Q-NETWORKS LEARNING: PLAYING ONLINE BATTLE ARENA WITH DISCRETE-CONTINUOUS HYBRID ACTION SPACE
Most existing deep reinforcement learning (DRL) frameworks consider action spaces that are either discrete or continuous space.
VEGA: Towards an End-to-End Configurable AutoML Pipeline
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models.
DetGPT: Detect What You Need via Reasoning
Overall, our proposed paradigm and DetGPT demonstrate the potential for more sophisticated and intuitive interactions between humans and machines.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data.
Ranked #1 on
Chinese Part-of-Speech Tagging
on CTB5 Dev
Chinese Named Entity Recognition
Chinese Word Segmentation
+5
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously.
ADAPT: Action-aware Driving Caption Transformer
To bridge the gap, we propose an end-to-end transformer-based architecture, ADAPT (Action-aware Driving cAPtion Transformer), which provides user-friendly natural language narrations and reasoning for each decision making step of autonomous vehicular control and action.
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
TransNAS-Bench-101: Improving Transferrability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on one single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
In this paper, we present ExtremeBERT, a toolkit for accelerating and customizing BERT pretraining.
Deep Subspace Clustering Networks
We present a novel deep neural network architecture for unsupervised subspace clustering.
Ranked #3 on
Image Clustering
on Extended Yale-B
RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering
Finding accurate correspondences among different views is the Achilles' heel of unsupervised Multi-View Stereo (MVS).
Deep Pyramid Convolutional Neural Networks for Text Categorization
This paper proposes a low-complexity word-level deep convolutional neural network (CNN) architecture for text categorization that can efficiently represent long-range associations in text.
Ranked #2 on
Sentiment Analysis
on Amazon Review Full
Active Prompting with Chain-of-Thought for Large Language Models
For this purpose, we propose a solution to the key problem of determining which questions are the most important and helpful ones to annotate from a pool of task-specific queries.
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process.
Ranked #4 on
RGB-D Salient Object Detection
on LFSD
Improving Chinese Word Segmentation with Wordhood Memory Networks
Contextual features always play an important role in Chinese word segmentation (CWS).
Ranked #1 on
Chinese Word Segmentation
on CITYU
Plum: Prompt Learning using Metaheuristic
Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models.
VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction
The success of the Neural Radiance Fields (NeRF) in novel view synthesis has inspired researchers to propose neural implicit scene reconstruction.
Multi-scale Convolutional Neural Networks for Crowd Counting
Crowd counting on static images is a challenging problem due to scale variations.
MAP Inference via L2-Sphere Linear Program Reformulation
Thus, LS-LP is equivalent to the original MAP inference problem.
Learning Nonlinear Functions Using Regularized Greedy Forest
We consider the problem of learning a forest of nonlinear decision rules with general loss functions.
Video Re-localization
We first exploit and reorganize the videos in ActivityNet to form a new dataset for video re-localization research, which consists of about 10, 000 videos of diverse visual appearances associated with localized boundary information.
Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python
We describe a new library named picasso, which implements a unified framework of pathwise coordinate optimization for a variety of sparse learning problems (e. g., sparse linear regression, sparse logistic regression, sparse Poisson regression and scaled sparse linear regression) combined with efficient active set selection strategies.
Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-way Attentions of Auto-analyzed Knowledge
Chinese word segmentation (CWS) and part-of-speech (POS) tagging are important fundamental tasks for Chinese language processing, where joint learning of them is an effective one-step solution for both tasks.
Few-Shot Human Motion Transfer by Personalized Geometry and Texture Modeling
We present a new method for few-shot human motion transfer that achieves realistic human image generation with only a small number of appearance inputs.
TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game
Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting.
DHER: Hindsight Experience Replay for Dynamic Goals
Dealing with sparse rewards is one of the most important challenges in reinforcement learning (RL), especially when a goal is dynamic (e. g., to grasp a moving object).
Self-Reference Deep Adaptive Curve Estimation for Low-Light Image Enhancement
In this paper, we propose a 2-stage low-light image enhancement method called Self-Reference Deep Adaptive Curve Estimation (Self-DACE).
CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer
Reconstructing personalized animatable head avatars has significant implications in the fields of AR/VR.
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in large language models (LLMs).
Black-Box Adversarial Attack with Transferable Model-based Embedding
We present a new method for black-box adversarial attack.
R-Tuning: Teaching Large Language Models to Refuse Unknown Questions
This approach is formalized by first identifying the knowledge gap between parametric knowledge and the instruction tuning data.
Weakly Supervised Disentangled Generative Causal Representation Learning
This paper proposes a Disentangled gEnerative cAusal Representation (DEAR) learning method under appropriate supervised information.
Black-box Prompt Learning for Pre-trained Language Models
Particularly, instead of fine-tuning the model in the cloud, we adapt PLMs by prompt learning, which efficiently optimizes only a few parameters of the discrete prompts.
End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera
In this paper, we propose a monocular camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network.
Uncertainty-aware Joint Salient Object and Camouflaged Object Detection
Visual salient object detection (SOD) aims at finding the salient object(s) that attract human attention, while camouflaged object detection (COD) on the contrary intends to discover the camouflaged object(s) that hidden in the surrounding.
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques.
Translating Pro-Drop Languages with Reconstruction Models
Next, the annotated source sentence is reconstructed from hidden representations in the NMT model.
Learning to Remember Translation History with a Continuous Cache
Existing neural machine translation (NMT) models generally translate sentences in isolation, missing the opportunity to take advantage of document-level information.
LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition
In this paper, we first examine traditional active learning strategies in a specific case of BiLstm-CRF that has widely used in named entity recognition on several typical datasets.
Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models Memories
Pre-trained language models (PLMs) demonstrate excellent abilities to understand texts in the generic domain while struggling in a specific domain.
Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Data
In this paper, we propose a data privacy-preserving and communication efficient distributed GAN learning framework named Distributed Asynchronized Discriminator GAN (AsynDGAN).
Spatiotemporal Self-supervised Learning for Point Clouds in the Wild
In this paper, we introduce an SSL strategy that leverages positive pairs in both the spatial and temporal domain.
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
To remedy this, this paper proposes \mldas, a mixed-level reformulation for NAS that can be optimized efficiently and reliably.
Divergence-Augmented Policy Optimization
In deep reinforcement learning, policy optimization methods need to deal with issues such as function approximation and the reuse of off-policy data.
Model Agnostic Sample Reweighting for Out-of-Distribution Learning
The overfitting issue is addressed by considering a bilevel formulation to search for the sample reweighting, in which the generalization complexity depends on the search space of sample weights instead of the model size.
Attention-guided Chained Context Aggregation for Semantic Segmentation
The way features propagate in Fully Convolutional Networks is of momentous importance to capture multi-scale contexts for obtaining precise segmentation masks.
Ranked #23 on
Semantic Segmentation
on SUN-RGBD
(using extra training data)
Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Offline reinforcement learning (RL) presents a promising approach for learning reinforced policies from offline datasets without the need for costly or unsafe interactions with the environment.
Time Series Generation with Masked Autoencoder
This paper shows that masked autoencoder with extrapolator (ExtraMAE) is a scalable self-supervised model for time series generation.
Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases
In particular, updating an intelligent diagnosis system with training data of new diseases would cause catastrophic forgetting of old disease knowledge.
ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders
Pre-trained text encoders have drawn sustaining attention in natural language processing (NLP) and shown their capability in obtaining promising results in different tasks.
Reinforced Training Data Selection for Domain Adaptation
Supervised models suffer from the problem of domain shifting where distribution mismatch in the data across domains greatly affect model performance.
DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks
Specifically, we search for a meta graph, which can capture more complex semantic relations than a meta path, to determine how graph neural networks (GNNs) propagate messages along different types of edges.
Effective Sparsification of Neural Networks with Global Sparsity Constraint
Weight pruning is an effective technique to reduce the model size and inference time for deep neural networks in real-world deployments.
Optimal Feature Transport for Cross-View Image Geo-Localization
This paper proposes a novel Cross-View Feature Transport (CVFT) technique to explicitly establish cross-view domain transfer that facilitates feature alignment between ground and aerial images.
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
Additionally, DPA models user preferences as directions (i. e., unit vectors) in the reward space to achieve user-dependent preference control.
Keyphrase Generation with Cross-Document Attention
Keyphrase generation aims to produce a set of phrases summarizing the essentials of a given document.
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance
The deployment of multimodal large language models (MLLMs) has brought forth a unique vulnerability: susceptibility to malicious attacks through visual inputs.
NetML: A Challenge for Network Traffic Analytics
Classifying network traffic is the basis for important network applications.
Multi-Task Hierarchical Learning Based Network Traffic Analytics
Classifying network traffic is the basis for important network applications.
TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on a single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
Mathematical Models of Overparameterized Neural Networks
Deep learning has received considerable empirical successes in recent years.
Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation
In this paper, we aim to adapt a generic pretrained model with a relatively small amount of domain-specific data.
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
However, most CoT studies rely on carefully designed human-annotated rational chains to prompt LLMs, posing challenges for real-world applications where labeled data is available without rational chains.
Hierarchical Neural Architecture Search via Operator Clustering
Recently, the efficiency of automatic neural architecture design has been significantly improved by gradient-based search methods such as DARTS.
Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning
Recent progress in this setting assumes that the base knowledge and novel query samples are distributed in the same domains, which are usually infeasible for realistic applications.
EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain
Multi-modal large language models (MLLMs) have demonstrated remarkable success in vision and visual-language tasks within the natural image domain.
Optimizing Latent Space Directions For GAN-based Local Image Editing
Generative Adversarial Network (GAN) based localized image editing can suffer from ambiguity between semantic attributes.
Improving Constituency Parsing with Span Attention
Constituency parsing is a fundamental and important task for natural language understanding, where a good representation of contextual information can help this task.
Ranked #1 on
Constituency Parsing
on ATB
Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain
Currently, under supervised learning, a model pretrained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated the knowledge transfer learning.
A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization
Afterward, we prove that under ideal conditions, independence-driven importance weighting algorithms could identify the variables in this set.
MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation
Commonsense reasoning tasks such as commonsense knowledge graph completion and commonsense question answering require powerful representation learning.
TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity.
TempSAL - Uncovering Temporal Information for Deep Saliency Prediction
Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information such as scene context, semantic relationships, gaze direction, and object dissimilarity.
Ranked #3 on
Saliency Prediction
on SALECI
HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning
However, it is limited to the case where 1) a good feature is known in advance and 2) this feature is fixed during the training: if otherwise, RLSVI suffers an unbearable computational burden to obtain the posterior samples of the parameter in the $Q$-value function.
Normalizing Flow with Variational Latent Representation
The resulting method is significantly more powerful than the standard normalization flow approach for generating data distributions with multiple modes.
What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
In this paper, we study out-of-distribution (OOD) generalization of offline GCRL both theoretically and empirically to identify factors that are important.
Two-person Graph Convolutional Network for Skeleton-based Human Interaction Recognition
To overcome the above shortcoming, we introduce a novel unified two-person graph to represent inter-body and intra-body correlations between joints.
Speeding up Transformer Decoding via an Attention Refinement Network
Extensive experiments on ten WMT machine translation tasks show that the proposed model yields an average of 1. 35x faster (with almost no decrease in BLEU) over the state-of-the-art inference implementation.
CLIP Meets Video Captioning: Concept-Aware Representation Learning Does Matter
DCD is an auxiliary task that requires a caption model to learn the correspondence between video content and concepts and the co-occurrence relations between concepts.
Ranked #16 on
Video Captioning
on MSR-VTT
UniKG: A Benchmark and Universal Embedding for Large-Scale Knowledge Graphs
To perform effective learning on the large-scale UniKG, two key measures are taken, including (i) the semantic alignment strategy for multi-attribute entities, which projects the feature description of multi-attribute nodes into a common embedding space to facilitate node aggregation in a large receptive field; (ii) proposing a novel plug-and-play anisotropy propagation module (APM) to learn effective multi-hop anisotropy propagation kernels, which extends methods of large-scale homogeneous graphs to heterogeneous graphs.
Efficient Neural Network Training via Forward and Backward Propagation Sparsification
For the latter step, instead of using the chain rule based gradient estimators as in existing methods, we propose a variance reduced policy gradient estimator, which only requires two forward passes without backward propagation, thus achieving completely sparse training.
Hashtag-Guided Low-Resource Tweet Classification
Social media classification tasks (e. g., tweet sentiment analysis, tweet stance detection) are challenging because social media posts are typically short, informal, and ambiguous.
An Incremental Unified Framework for Small Defect Inspection
Artificial Intelligence (AI)-driven defect inspection is pivotal in industrial manufacturing.
The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs
In this paper, we identify a typical class of inputs that baffles MLLMs, which consist of images that are highly relevant but inconsistent with answers, causing MLLMs to suffer from hallucination.
DVMNet: Computing Relative Pose for Unseen Objects Beyond Hypotheses
Determining the relative pose of an object between two images is pivotal to the success of generalizable object pose estimation.
Stochastic Expectation Maximization with Variance Reduction
However, sEM has a slower asymptotic convergence rate than batch EM, and requires a decreasing sequence of step sizes, which is difficult to tune.
Bidirectional Generative Modeling Using Adversarial Gradient Estimation
This paper considers the general $f$-divergence formulation of bidirectional generative modeling, which includes VAE and BiGAN as special cases.
Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting
However, the imbalanced training dataset leads to poor performance on rare senses and zero-shot senses.
Communication Efficient Distributed Optimization using an Approximate Newton-type Method
We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems.
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts
In particular, we propose a novel joint-training framework to train plain CNN by leveraging the gradients of the ResNet counterpart.
Modeling Object Dissimilarity for Deep Saliency Prediction
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects.
Corruption-Robust Offline Reinforcement Learning with General Function Approximation
Notably, under the assumption of single policy coverage and the knowledge of $\zeta$, our proposed algorithm achieves a suboptimality bound that is worsened by an additive factor of $\mathcal{O}(\zeta (C(\widehat{\mathcal{F}},\mu)n)^{-1})$ due to the corruption.
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
In this paper, we propose Eigencurve, the first family of learning rate schedules that can achieve minimax optimal convergence rates (up to a constant) for SGD on quadratic objectives when the eigenvalue distribution of the underlying Hessian matrix is skewed.
Composite Functional Gradient Learning of Generative Adversarial Models
This paper first presents a theory for generative adversarial methods that does not rely on the traditional minimax formulation.
Safe Element Screening for Submodular Function Minimization
Relying on this study, we subsequently propose a novel safe screening method to quickly identify the elements guaranteed to be included (we refer to them as active) or excluded (inactive) in the final optimal solution of SFM during the optimization process.
End-to-end Active Object Tracking via Reinforcement Learning
We study active object tracking, where a tracker takes as input the visual observation (i. e., frame sequence) and produces the camera control signal (e. g., move forward, turn left, etc.).
Communication Compression for Decentralized Training
In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?}
QuaSE: Accurate Text Style Transfer under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
Walk-Steered Convolution for Graph Classification
As an analogy to a standard convolution kernel on image, Gaussian models implicitly coordinate those unordered vertices/nodes and edges in a local receptive field after projecting to the gradient space of Gaussian parameters.
Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset
First, we demonstrate that video attractiveness and different engagements present different relationships.
Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
Such supervision, while labor-intensive and not always possible, tends to hinder the generalization ability of the learned models.
Tensor graph convolutional neural network
For cross graph convolution, a parameterized Kronecker sum operation is proposed to generate a conjunctive adjacency matrix characterizing the relationship between every pair of nodes across two subgraphs.
Neural Stereoscopic Image Style Transfer
While each view of the stereoscopic pair is processed in an individual path, a novel feature aggregation strategy is proposed to effectively share information between the two paths.
On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
Robust Frequent Directions with Application in Online Learning
We also apply RFD to online learning and propose an effective hyperparameter-free online Newton algorithm.
Candidates vs. Noises Estimation for Large Multi-Class Classification Problem
This paper proposes a method for multi-class classification problems, where the number of classes K is large.
Gradient Sparsification for Communication-Efficient Distributed Optimization
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures.
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations
We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed.
Near-Optimal Stochastic Approximation for Online Principal Component Estimation
We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm.
Deep manifold-to-manifold transforming network for action recognition
Symmetric positive definite (SPD) matrices (e. g., covariances, graph Laplacians, etc.)
A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines.
Graphical Nonconvex Optimization for Optimal Estimation in Gaussian Graphical Models
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
Spatial-Temporal Recurrent Neural Network for Emotion Recognition
Then a bi-directional temporal RNN layer is further used to learn discriminative temporal dependencies from the sequences concatenating spatial features of each time slice produced from the spatial RNN layer.
Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory
This is the first result on the computational and statistical guarantees of the pathwise coordinate optimization framework in high dimensions.
Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition
The Nystr\"om method is a special instance of our fast model and is approximation to the prototype model.
Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level
This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016).
Local Uncertainty Sampling for Large-Scale Multi-Class Logistic Regression
A major challenge for building statistical models in the big data era is that the available data volume far exceeds the computational capability.
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data.
Ranked #1 on
Text Classification
on RCV1
Efficient Distributed Learning with Sparsity
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines.
Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
Sparse generalized eigenvalue problem (GEP) plays a pivotal role in a large family of high-dimensional statistical models, including sparse Fisher's discriminant analysis, canonical correlation analysis, and sufficient dimension reduction.
Learning Sparse Low-Threshold Linear Classifiers
We consider the problem of learning a non-negative linear classifier with a $1$-norm of at most $k$, and a fixed threshold, under the hinge-loss.
Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
To recover $\beta^*$, we propose an $\ell_1$-regularized least-squares estimator.
Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
This paper presents a new semi-supervised framework with convolutional neural networks (CNNs) for text categorization.
Ranked #1000000000 on
Text Classification
on IMDb
Adjusting Leverage Scores by Row Weighting: A Practical Approach to Coherent Matrix Completion
Low-rank matrix completion is an important problem with extensive real-world applications.
Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
In particular, our analysis improves upon existing results by providing a more refined sample complexity bound as well as an exact support recovery result for the final estimator.
Stochastic Optimization with Importance Sampling
Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA).
Randomized Dual Coordinate Ascent with Arbitrary Sampling
The distributed variant of Quartz is the first distributed SDCA-like method with an analysis for non-separable data.
Adaptive Stochastic Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers (ADMM) has been studied for years.
Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling
Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks.
Random design analysis of ridge regression
The analysis also reveals the effect of errors in the estimated covariance structure, as well as the effect of modeling errors, neither of which effects are present in the fixed design setting.
A Proximal Stochastic Gradient Method with Progressive Variance Reduction
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping.
Sparse Recovery with Very Sparse Compressed Counting
In this paper, we adopt very sparse Compressed Counting for nonnegative signal recovery.
Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization
Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.
Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics
We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e. g., Mercer kernels.
Aggregation of Affine Estimators
While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability.
Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure.
Compressed Counting Meets Compressed Sensing
In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.
High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning
The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks.
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent
Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning.
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization
Large-scale distributed optimization is of great importance in various applications.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose two new algorithms, namely SPIDER-SFO and SPIDER-SFO\textsuperscript{+}, that solve non-convex stochastic optimization problems using stochastic gradients only.
When Work Matters: Transforming Classical Network Structures to Graph CNN
In this paper, we aim to give a comprehensive analysis of when work matters by transforming different classical network structures to graph CNN, particularly in the basic graph recognition problem.
Ranked #3 on
Graph Classification
on IMDb-B
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
Recent studies on unsupervised image-to-image translation have made a remarkable progress by training a pair of generative adversarial networks with a cycle-consistent loss.
Recurrent Fusion Network for Image Captioning
In this paper, in order to exploit the complementary information from multiple encoders, we propose a novel Recurrent Fusion Network (RFNet) for tackling image captioning.
End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning
We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training.
Diffusion Approximations for Online Principal Component Estimation and Global Convergence
In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis.
A convex formulation for high-dimensional sparse sliced inverse regression
Sliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates.
Fully Implicit Online Learning
Regularized online learning is widely used in machine learning applications.
Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry
Our approach is able to handle the rolling-shutter effects and imperfect sensor synchronization in a unified way.
Orthogonal Deep Features Decomposition for Age-Invariant Face Recognition
Extensive experiments conducted on the three public domain face aging datasets (MORPH Album 2, CACD-VS and FG-NET) have shown the effectiveness of the proposed approach and the value of the constructed CAF dataset on AIFR.
Ranked #3 on
Age-Invariant Face Recognition
on MORPH Album2
Multi-Head Attention with Disagreement Regularization
Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions.
Modeling Localness for Self-Attention Networks
Self-attention networks have proven to be of profound value for its strength of capturing global dependencies.
Ranked #29 on
Machine Translation
on WMT2014 English-German
Exploiting Deep Representations for Neural Machine Translation
Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers, which allows systems to model complex functions and capture complicated linguistic structures.
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
We prove that the proposed method globally converges to a stationary point.
Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Scalable Deep $k$-Subspace Clustering
In this paper, we introduce a method that simultaneously learns an embedding space along subspaces within it to minimize a notion of reconstruction error, thus addressing the problem of subspace clustering in an end-to-end learning paradigm.
Super-Identity Convolutional Neural Network for Face Hallucination
Face hallucination is a generative task to super-resolve the facial image with low resolution while human perception of face heavily relies on identity information.
Neural Machine Translation with Adequacy-Oriented Learning
Although Neural Machine Translation (NMT) models have advanced state-of-the-art performance in machine translation, they face problems like the inadequate translation.
Ranked #35 on
Machine Translation
on WMT2014 English-German
Cross-database non-frontal facial expression recognition based on transductive deep transfer learning
Cross-database non-frontal expression recognition is a very meaningful but rather difficult subject in the fields of computer vision and affect computing.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents
This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Cross-Database Micro-Expression Recognition: A Benchmark
Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis.
QuaSE: Sequence Editing under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator
Specially, we prove that the SPIDER-SFO algorithm achieves a gradient computation cost of $\mathcal{O}\left( \min( n^{1/2} \epsilon^{-2}, \epsilon^{-3} ) \right)$ to find an $\epsilon$-approximate first-order stationary point.
Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
On Quadratic Convergence of DC Proximal Newton Algorithm in Nonconvex Sparse Learning
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
Efficient Optimization for Linear Dynamical Systems with Applications to Clustering and Sparse Coding
Linear Dynamical Systems (LDSs) are fundamental tools for modeling spatio-temporal data in various disciplines.
Exact Recovery of Hard Thresholding Pursuit
In this paper, we bridge this gap by showing, for the first time, that exact recovery of the global sparse minimizer is possible for HTP-style methods under restricted strong condition number bounding conditions.
Learning Additive Exponential Family Graphical Models via \ell_{2,1}-norm Regularized M-Estimation
We investigate a subclass of exponential family graphical models of which the sufficient statistics are defined by arbitrary additive forms.
Quartz: Randomized Dual Coordinate Ascent with Arbitrary Sampling
We study the problem of minimizing the average of a large number of smooth convex functions penalized with a strongly convex regularizer.
Local Smoothness in Variance Reduced Optimization
Abstract We propose a family of non-uniform sampling strategies to provably speed up a class of stochastic optimization algorithms with linear convergence including Stochastic Variance Reduced Gradient (SVRG) and Stochastic Dual Coordinate Ascent (SDCA).
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Stochastic gradient descent is popular for large scale optimization but has slow convergence asymptotically due to the inherent variance.
Selective Labeling via Error Bound Minimization
In particular, we derive a deterministic generalization error bound for LapRLS trained on subsampled data, and propose to select a subset of data points to label by minimizing this upper bound.
Greedy Model Averaging
The purpose of this paper is to present a new greedy model averaging procedure that improves EWMA.
Learning to Search Efficiently in High Dimensions
Traditional approaches relied on algorithmic constructions that are often data independent (such as Locality Sensitive Hashing) or weakly dependent (such as kd-trees, k-means trees).
Spectral Methods for Learning Multivariate Latent Tree Structure
The setting is one where we only have samples from certain observed variables in the tree, and our goal is to estimate the tree structure (i. e., the graph of how the underlying hidden variables are connected to each other and to the observed variables).
Agnostic Active Learning Without Constraints
We present and analyze an agnostic active learning algorithm that works without keeping a version space.
Deep Coding Network
This paper proposes a principled extension of the traditional single-layer flat sparse coding scheme, where a two-layer coding scheme is derived based on theoretical analysis of nonlinear functional approximation that extends recent results for local coordinate coding.
Nonlinear Learning using Local Coordinate Coding
This paper introduces a new method for semi-supervised learning on high dimensional nonlinear manifolds, which includes a phase of unsupervised basis learning and a phase of supervised function learning.
Sparse Online Learning via Truncated Gradient
We propose a general method called truncated gradient to induce sparsity in the weights of online-learning algorithms with convex loss.
Adaptive Forward-Backward Greedy Algorithm for Sparse Learning with Linear Models
Consider linear prediction models where the target function is a sparse linear combination of a set of basis functions.
Multi-stage Convex Relaxation for Learning with Sparse Regularization
We study learning formulations with non-convex regularizaton that are natural for sparse linear models.
The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information
We present Epoch-Greedy, an algorithm for multi-armed bandits with observable side information.
A General Boosting Method and its Application to Learning Ranking Functions for Web Search
We present a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems.
Projection-free Distributed Online Learning in Networks
The conditional gradient algorithm has regained a surge of research interest in recent years due to its high efficiency in handling large-scale machine learning problems.
Graphical Nonconvex Optimization via an Adaptive Convex Relaxation
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
NATTACK: A STRONG AND UNIVERSAL GAUSSIAN BLACK-BOX ADVERSARIAL ATTACK
In other words, there is a population of adversarial examples, instead of only one, for any input to a DNN.
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Zeroth-order optimization is an important research topic in machine learning.
