Search Results for author:
Found 366 papers, 114 papers with code
Composite Functional Gradient Learning of Generative Adversarial Models
This paper first presents a theory for generative adversarial methods that does not rely on the traditional minimax formulation.
Safe Element Screening for Submodular Function Minimization
Relying on this study, we subsequently propose a novel safe screening method to quickly identify the elements guaranteed to be included (we refer to them as active) or excluded (inactive) in the final optimal solution of SFM during the optimization process.
End-to-end Active Object Tracking via Reinforcement Learning
We study active object tracking, where a tracker takes as input the visual observation (i. e., frame sequence) and produces the camera control signal (e. g., move forward, turn left, etc.).
Communication Compression for Decentralized Training
In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?}
QuaSE: Accurate Text Style Transfer under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
Walk-Steered Convolution for Graph Classification
As an analogy to a standard convolution kernel on image, Gaussian models implicitly coordinate those unordered vertices/nodes and edges in a local receptive field after projecting to the gradient space of Gaussian parameters.
Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset
First, we demonstrate that video attractiveness and different engagements present different relationships.
Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
Such supervision, while labor-intensive and not always possible, tends to hinder the generalization ability of the learned models.
Tensor graph convolutional neural network
For cross graph convolution, a parameterized Kronecker sum operation is proposed to generate a conjunctive adjacency matrix characterizing the relationship between every pair of nodes across two subgraphs.
Neural Stereoscopic Image Style Transfer
While each view of the stereoscopic pair is processed in an individual path, a novel feature aggregation strategy is proposed to effectively share information between the two paths.
On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
Robust Frequent Directions with Application in Online Learning
We also apply RFD to online learning and propose an effective hyperparameter-free online Newton algorithm.
Candidates vs. Noises Estimation for Large Multi-Class Classification Problem
This paper proposes a method for multi-class classification problems, where the number of classes K is large.
Gradient Sparsification for Communication-Efficient Distributed Optimization
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures.
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations
We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed.
Near-Optimal Stochastic Approximation for Online Principal Component Estimation
We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm.
Deep manifold-to-manifold transforming network for action recognition
Symmetric positive definite (SPD) matrices (e. g., covariances, graph Laplacians, etc.)
A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines.
Graphical Nonconvex Optimization for Optimal Estimation in Gaussian Graphical Models
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
Spatial-Temporal Recurrent Neural Network for Emotion Recognition
Then a bi-directional temporal RNN layer is further used to learn discriminative temporal dependencies from the sequences concatenating spatial features of each time slice produced from the spatial RNN layer.
Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory
This is the first result on the computational and statistical guarantees of the pathwise coordinate optimization framework in high dimensions.
Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition
The Nystr\"om method is a special instance of our fast model and is approximation to the prototype model.
Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level
This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016).
Local Uncertainty Sampling for Large-Scale Multi-Class Logistic Regression
A major challenge for building statistical models in the big data era is that the available data volume far exceeds the computational capability.
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data.
Ranked #1 on
Text Classification
on RCV1
Efficient Distributed Learning with Sparsity
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines.
Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
Sparse generalized eigenvalue problem (GEP) plays a pivotal role in a large family of high-dimensional statistical models, including sparse Fisher's discriminant analysis, canonical correlation analysis, and sufficient dimension reduction.
Learning Sparse Low-Threshold Linear Classifiers
We consider the problem of learning a non-negative linear classifier with a $1$-norm of at most $k$, and a fixed threshold, under the hinge-loss.
Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
To recover $\beta^*$, we propose an $\ell_1$-regularized least-squares estimator.
Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
This paper presents a new semi-supervised framework with convolutional neural networks (CNNs) for text categorization.
Ranked #1000000000 on
Text Classification
on IMDb
Adjusting Leverage Scores by Row Weighting: A Practical Approach to Coherent Matrix Completion
Low-rank matrix completion is an important problem with extensive real-world applications.
Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
In particular, our analysis improves upon existing results by providing a more refined sample complexity bound as well as an exact support recovery result for the final estimator.
Stochastic Optimization with Importance Sampling
Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA).
Randomized Dual Coordinate Ascent with Arbitrary Sampling
The distributed variant of Quartz is the first distributed SDCA-like method with an analysis for non-separable data.
Adaptive Stochastic Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers (ADMM) has been studied for years.
Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling
Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks.
Random design analysis of ridge regression
The analysis also reveals the effect of errors in the estimated covariance structure, as well as the effect of modeling errors, neither of which effects are present in the fixed design setting.
A Proximal Stochastic Gradient Method with Progressive Variance Reduction
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping.
Sparse Recovery with Very Sparse Compressed Counting
In this paper, we adopt very sparse Compressed Counting for nonnegative signal recovery.
Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization
Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.
Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics
We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e. g., Mercer kernels.
Aggregation of Affine Estimators
While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability.
Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure.
Compressed Counting Meets Compressed Sensing
In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.
High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning
The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks.
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent
Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning.
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization
Large-scale distributed optimization is of great importance in various applications.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose two new algorithms, namely SPIDER-SFO and SPIDER-SFO\textsuperscript{+}, that solve non-convex stochastic optimization problems using stochastic gradients only.
When Work Matters: Transforming Classical Network Structures to Graph CNN
In this paper, we aim to give a comprehensive analysis of when work matters by transforming different classical network structures to graph CNN, particularly in the basic graph recognition problem.
Ranked #3 on
Graph Classification
on IMDb-B
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
Recent studies on unsupervised image-to-image translation have made a remarkable progress by training a pair of generative adversarial networks with a cycle-consistent loss.
Recurrent Fusion Network for Image Captioning
In this paper, in order to exploit the complementary information from multiple encoders, we propose a novel Recurrent Fusion Network (RFNet) for tackling image captioning.
End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning
We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training.
Diffusion Approximations for Online Principal Component Estimation and Global Convergence
In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis.
A convex formulation for high-dimensional sparse sliced inverse regression
Sliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates.
Fully Implicit Online Learning
Regularized online learning is widely used in machine learning applications.
Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry
Our approach is able to handle the rolling-shutter effects and imperfect sensor synchronization in a unified way.
Orthogonal Deep Features Decomposition for Age-Invariant Face Recognition
Extensive experiments conducted on the three public domain face aging datasets (MORPH Album 2, CACD-VS and FG-NET) have shown the effectiveness of the proposed approach and the value of the constructed CAF dataset on AIFR.
Ranked #3 on
Age-Invariant Face Recognition
on MORPH Album2
Multi-Head Attention with Disagreement Regularization
Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions.
Modeling Localness for Self-Attention Networks
Self-attention networks have proven to be of profound value for its strength of capturing global dependencies.
Ranked #29 on
Machine Translation
on WMT2014 English-German
Exploiting Deep Representations for Neural Machine Translation
Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers, which allows systems to model complex functions and capture complicated linguistic structures.
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
We prove that the proposed method globally converges to a stationary point.
Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Scalable Deep $k$-Subspace Clustering
In this paper, we introduce a method that simultaneously learns an embedding space along subspaces within it to minimize a notion of reconstruction error, thus addressing the problem of subspace clustering in an end-to-end learning paradigm.
Super-Identity Convolutional Neural Network for Face Hallucination
Face hallucination is a generative task to super-resolve the facial image with low resolution while human perception of face heavily relies on identity information.
Neural Machine Translation with Adequacy-Oriented Learning
Although Neural Machine Translation (NMT) models have advanced state-of-the-art performance in machine translation, they face problems like the inadequate translation.
Ranked #35 on
Machine Translation
on WMT2014 English-German
Cross-database non-frontal facial expression recognition based on transductive deep transfer learning
Cross-database non-frontal expression recognition is a very meaningful but rather difficult subject in the fields of computer vision and affect computing.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents
This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Cross-Database Micro-Expression Recognition: A Benchmark
Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis.
QuaSE: Sequence Editing under Quantifiable Guidance
For example, an input sequence could be a word sequence, such as review sentence and advertisement text.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator
Specially, we prove that the SPIDER-SFO algorithm achieves a gradient computation cost of $\mathcal{O}\left( \min( n^{1/2} \epsilon^{-2}, \epsilon^{-3} ) \right)$ to find an $\epsilon$-approximate first-order stationary point.
Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
On Quadratic Convergence of DC Proximal Newton Algorithm in Nonconvex Sparse Learning
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.
Efficient Optimization for Linear Dynamical Systems with Applications to Clustering and Sparse Coding
Linear Dynamical Systems (LDSs) are fundamental tools for modeling spatio-temporal data in various disciplines.
Exact Recovery of Hard Thresholding Pursuit
In this paper, we bridge this gap by showing, for the first time, that exact recovery of the global sparse minimizer is possible for HTP-style methods under restricted strong condition number bounding conditions.
Learning Additive Exponential Family Graphical Models via \ell_{2,1}-norm Regularized M-Estimation
We investigate a subclass of exponential family graphical models of which the sufficient statistics are defined by arbitrary additive forms.
Quartz: Randomized Dual Coordinate Ascent with Arbitrary Sampling
We study the problem of minimizing the average of a large number of smooth convex functions penalized with a strongly convex regularizer.
Local Smoothness in Variance Reduced Optimization
Abstract We propose a family of non-uniform sampling strategies to provably speed up a class of stochastic optimization algorithms with linear convergence including Stochastic Variance Reduced Gradient (SVRG) and Stochastic Dual Coordinate Ascent (SDCA).
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Stochastic gradient descent is popular for large scale optimization but has slow convergence asymptotically due to the inherent variance.
Selective Labeling via Error Bound Minimization
In particular, we derive a deterministic generalization error bound for LapRLS trained on subsampled data, and propose to select a subset of data points to label by minimizing this upper bound.
Greedy Model Averaging
The purpose of this paper is to present a new greedy model averaging procedure that improves EWMA.
Learning to Search Efficiently in High Dimensions
Traditional approaches relied on algorithmic constructions that are often data independent (such as Locality Sensitive Hashing) or weakly dependent (such as kd-trees, k-means trees).
Spectral Methods for Learning Multivariate Latent Tree Structure
The setting is one where we only have samples from certain observed variables in the tree, and our goal is to estimate the tree structure (i. e., the graph of how the underlying hidden variables are connected to each other and to the observed variables).
Agnostic Active Learning Without Constraints
We present and analyze an agnostic active learning algorithm that works without keeping a version space.
Deep Coding Network
This paper proposes a principled extension of the traditional single-layer flat sparse coding scheme, where a two-layer coding scheme is derived based on theoretical analysis of nonlinear functional approximation that extends recent results for local coordinate coding.
Nonlinear Learning using Local Coordinate Coding
This paper introduces a new method for semi-supervised learning on high dimensional nonlinear manifolds, which includes a phase of unsupervised basis learning and a phase of supervised function learning.
Sparse Online Learning via Truncated Gradient
We propose a general method called truncated gradient to induce sparsity in the weights of online-learning algorithms with convex loss.
Adaptive Forward-Backward Greedy Algorithm for Sparse Learning with Linear Models
Consider linear prediction models where the target function is a sparse linear combination of a set of basis functions.
Multi-stage Convex Relaxation for Learning with Sparse Regularization
We study learning formulations with non-convex regularizaton that are natural for sparse linear models.
The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information
We present Epoch-Greedy, an algorithm for multi-armed bandits with observable side information.
A General Boosting Method and its Application to Learning Ranking Functions for Web Search
We present a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems.
Projection-free Distributed Online Learning in Networks
The conditional gradient algorithm has regained a surge of research interest in recent years due to its high efficiency in handling large-scale machine learning problems.
Graphical Nonconvex Optimization via an Adaptive Convex Relaxation
Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence.
NATTACK: A STRONG AND UNIVERSAL GAUSSIAN BLACK-BOX ADVERSARIAL ATTACK
In other words, there is a population of adversarial examples, instead of only one, for any input to a DNN.
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Zeroth-order optimization is an important research topic in machine learning.
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points
In this paper, we give a sharp analysis for Stochastic Gradient Descent (SGD) and prove that SGD is able to efficiently escape from saddle points and find an $(\epsilon, O(\epsilon^{0. 5}))$-approximate second-order stationary point in $\tilde{O}(\epsilon^{-3. 5})$ stochastic gradient computations for generic nonconvex optimization problems, when the objective function satisfies gradient-Lipschitz, Hessian-Lipschitz, and dispersive noise assumptions.
Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
With the promising progress of deep neural networks, layer aggregation has been used to fuse information across layers in various fields, such as computer vision and machine translation.
Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
In this paper, we evaluate the robustness of state-of-the-art face recognition models in the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire hard-label predictions by sending queries to the target model.
Neural Collaborative Subspace Clustering
We introduce the Neural Collaborative Subspace Clustering, a neural model that discovers clusters of data points drawn from a union of low-dimensional subspaces.
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
For example, under the popular parameter server model for distributed learning, the worker nodes need to send the compressed local gradients to the parameter server, which performs the aggregation.
Estimating and Inferring the Maximum Degree of Stimulus-Locked Time-Varying Brain Connectivity Networks
To address this issue, neuroscientists have been measuring brain activity under natural viewing experiments in which the subjects are given continuous stimuli, such as watching a movie or listening to a story.
An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method
Specifically, we find that a large class of primal and primal-dual operator splitting algorithms are all special cases of VMOR-HPE.
$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression
Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized learning to further reduce the communication cost."
Self-supervised Recurrent Neural Network for 4D Abdominal and In-utero MR Imaging
The results show that the proposed pipeline can accurately estimate the respiratory state and reconstruct 4D SR volumes with better or similar performance to the 3D SVR pipeline with less than 20\% sparsely selected slices.
A Stochastic Extra-Step Quasi-Newton Method for Nonsmooth Nonconvex Optimization
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems.
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations
Recently, over-parameterized neural networks have been extensively analyzed in the literature.
Mirror Natural Evolution Strategies
We show that the estimated covariance matrix of MiNES converges to the inverse of Hessian matrix of the objective function with a sublinear convergence rate.
Spatial Sparse subspace clustering for Compressive Spectral imaging
This paper aims at developing a clustering approach with spectral images directly from CASSI compressive measurements.
Sparse Coding on Cascaded Residuals
The residual at a layer is computed by the difference between the aggregated reconstructions of the previous layers and the downsampled original image at that layer.
Convex Formulation of Overparameterized Deep Neural Networks
This new analysis is consistent with empirical observations that deep neural networks are capable of learning efficient feature representations.
A Fast Sampling Gradient Tree Boosting Framework
As an adaptive, interpretable, robust, and accurate meta-algorithm for arbitrary differentiable loss functions, gradient tree boosting is one of the most popular machine learning techniques, though the computational expensiveness severely limits its usage.
Stable Learning via Sample Reweighting
We consider the problem of learning linear prediction models with model misspecification bias.
Dual-Attention Graph Convolutional Network
Graph convolutional networks (GCNs) have shown the powerful ability in text structure representation and effectively facilitate the task of text classification.
Fast Generalized Matrix Regression with Applications in Machine Learning
Fast matrix algorithms have become the fundamental tools of machine learning in big data era.
Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problems
We consider nonconvex-concave minimax optimization problems of the form $\min_{\bf x}\max_{\bf y\in{\mathcal Y}} f({\bf x},{\bf y})$, where $f$ is strongly-concave in $\bf y$ but possibly nonconvex in $\bf x$ and ${\mathcal Y}$ is a convex and compact set.
Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization
Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e. g., sparsity) efficiently.
Graph Inference Learning for Semi-supervised Classification
In this work, we address semi-supervised classification of graph data, where the categories of those unlabeled nodes are inferred from labeled nodes as well as graph structures.
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks
In this paper, we provide a generalized neural tangent kernel analysis and show that noisy gradient descent with weight decay can still exhibit a "kernel-like" behavior.
Multi-consensus Decentralized Accelerated Gradient Descent
This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory.
Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization
This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space.
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
This new representation overcomes the degenerate situation where all the hidden units essentially have only one meaningful hidden unit in each middle layer, and further leads to a simpler representation of DNNs, for which the training objective can be reformulated as a convex optimization problem via suitable re-parameterization.
CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
In spite of its remarkable progress, many algorithms are restricted to particular search spaces.
Ranked #13 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Towards Purely Unsupervised Disentanglement of Appearance and Shape for Person Images Generation
To be clear, in this paper, we refer unsupervised learning as learning without task-specific human annotations, pairs or any form of weak supervision.)
Graph Wasserstein Correlation Analysis for Movie Retrieval
Spectral graph filtering is introduced to encode graph signals, which are then embedded as probability distributions in a Wasserstein space, called graph Wasserstein metric learning.
Instance-Aware Graph Convolutional Network for Multi-Label Classification
For label diffusion of instance-awareness in graph convolution, rather than using the statistical label correlation alone, an image-dependent label correlation matrix (LCM), fusing both the statistical LCM and an individual one of each image instance, is constructed for graph inference on labels to inject adaptive information of label-awareness into the learned features of the model.
Interest-Behaviour Multiplicative Network for Resource-limited Recommendation
In this paper, we aim to mine the cue of user preferences in resource-limited recommendation tasks, for which purpose we specifically build a large used car transaction dataset possessing resource-limitation characteristics.
Effective Training of Sparse Neural Networks under Global Sparsity Constraint
An appealing feature of ProbMask is that the amounts of weight redundancy can be learned automatically via our constraint and thus we avoid the problem of tuning pruning rates individually for different layers in a network.
Graph Deformer Network
In this paper, we propose a simple yet effective graph deformer network (GDN) to fulfill anisotropic convolution filtering on graphs, analogous to the standard convolution operation on images.
Invariant Batch Normalization for Multi-source Domain Generalization
We consider the domain generalization problem, where the test domain differs from the training domain.
CorrAttack: Black-box Adversarial Attack with Structured Search
We show that searching over the structured space can be approximated by a time-varying contextual bandits problem, where the attacker takes feature of the associated arm to make modifications of the input, and receives an immediate reward as the reduction of the loss function.
How to Characterize The Landscape of Overparameterized Convolutional Neural Networks
With the help of a new technique called {\it neural network grafting}, we demonstrate that even during the entire training process, feature distributions of differently initialized networks remain similar at each layer.
Decentralized Accelerated Proximal Gradient Descent
In this paper, we propose a new method which establishes the optimal computational complexity and a near optimal communication complexity.
Multi-modal AsynDGAN: Learn From Distributed Medical Image Data without Sharing Private Information
As deep learning technologies advance, increasingly more data is necessary to generate general and robust models for various tasks.
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
This paper considers the decentralized composite optimization problem.
Evidence of topological nodal lines and surface states in the centrosymmetric superconductor SnTaS2
Utilizing angle-resolved photoemission spectroscopy and first-principles calculations, here, we demonstrate the existence of topological nodal-line states and drumheadlike surface states in centrosymmetric superconductor SnTaS2, which is a type-II superconductor with a critical transition temperature of about 3 K. The valence bands from Ta 5d orbitals and the conduction bands from Sn 5p orbitals cross each other, forming two nodal lines in the vicinity of the Fermi energy without the inclusion of spin-orbit coupling (SOC), protected by the spatial-inversion symmetry and time-reversal symmetry.
Superconductivity
Nondiscriminatory Treatment: a straightforward framework for multi-human parsing
In inference time, we design a brand-new grouping post-processing method that relates each part instance with one single human instance and groups them together to obtain the final human-level parsing result.
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
This leads to a decentralized PCA algorithm called \texttt{DeEPCA}, which has a convergence rate similar to that of the centralized PCA, while achieving the best communication complexity among existing decentralized PCA algorithms.
On Secure Degrees of Freedom of the MIMO Interference Channel with Local Output Feedback
This paper studies the problem of sum-secure degrees of freedom (SDoF) of the (M, M, N, N) multiple-input multiple-output (MIMO) interference channel with local output feedback, so as to build an information-theoretic foundation and provide practical transmission schemes for 6G-enabled vehicles-to-vehicles (V2V).
Information Theory Information Theory
A Spectral Algorithm for Learning Hidden Markov Models
Hidden Markov Models (HMMs) are one of the most fundamental and widely used statistical tools for modeling discrete time series.
Towards Unbiased COVID-19 Lesion Localisation and Segmentation via Weakly Supervised Learning
Despite tremendous efforts, it is very challenging to generate a robust model to assist in the accurate quantification assessment of COVID-19 on chest CT images.
Siamese Labels Auxiliary Learning
In general, the main work of this paper include: (1) propose SiLa Learning, which improves the performance of common models without increasing test parameters; (2) compares SiLa with DML and proves that SiLa can improve the generalization of the model; (3) SiLa is applied to Dynamic Neural Networks, and proved that SiLa can be used for various types of network structures.
Spatial-Temporal Tensor Graph Convolutional Network for Traffic Prediction
We further extend graph convolution into tensor space and propose a tensor graph convolution network to extract more discriminating features from spatial-temporal graph data.
Ranked #1 on
Traffic Prediction
on SZ-Taxi
Reinforced Attention for Few-Shot Learning and Beyond
Few-shot learning aims to correctly recognize query samples from unseen classes given a limited number of support samples, often by relying on global embeddings of images.
Exploring Geometric Consistency for Monocular 3D Object Detection
In addition, we demonstrate that the augmentation methods are well suited for semi-supervised training and cross-dataset generalization.
KECRS: Towards Knowledge-Enriched Conversational Recommendation System
The chit-chat-based conversational recommendation systems (CRS) provide item recommendations to users through natural language interactions.
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
For student morphism, weight inheritance strategy is adopted, allowing the student to flexibly update its architecture while fully utilize the predecessor's weights, which considerably accelerates the search; To facilitate dynamic distillation, an elastic teacher pool is trained via integrated progressive shrinking strategy, from which teacher detectors can be sampled without additional cost in subsequent searches.
Universal Adder Neural Networks
The widely-used convolutions in deep neural networks are exactly cross-correlation to measure the similarity between input feature and convolution filters, which involves massive multiplications between float values.
Near Optimal Stochastic Algorithms for Finite-Sum Unbalanced Convex-Concave Minimax Optimization
This paper considers stochastic first-order algorithms for convex-concave minimax problems of the form $\min_{\bf x}\max_{\bf y}f(\bf x, \bf y)$, where $f$ can be presented by the average of $n$ individual components which are $L$-average smooth.
Multi-Hop Transformer for Document-Level Machine Translation
Document-level neural machine translation (NMT) has proven to be of profound value for its effectiveness on capturing contextual information.
Accelerating Edge Intelligence via Integrated Sensing and Communication
Conventionally, the sensing and communication stages are executed sequentially, which results in excessive amount of dataset generation and uploading time.
G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation
In this paper, we investigate the knowledge distillation (KD) strategy for object detection and propose an effective framework applicable to both homogeneous and heterogeneous student-teacher pairs.
Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation
This paper proposes a sophisticated neural architecture to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models.
Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
The core concept of GNNs is to find a representation by recursively aggregating the representations of a central node and those of its neighbors.
EllipseNet: Anchor-Free Ellipse Detection for Automatic Cardiac Biometrics in Fetal Echocardiography
In this paper, we present an anchor-free ellipse detection network, namely EllipseNet, which detects the cardiac and thoracic regions in ellipse and automatically calculates the CTR and cardiac axis for fetal cardiac biometrics in 4-chamber view.
Feel-Good Thompson Sampling for Contextual Bandits and Reinforcement Learning
In this setting, we show that the standard Thompson Sampling is not aggressive enough in exploring new actions, leading to suboptimality in some pessimistic situations.
Wasserstein Coupled Graph Learning for Cross-Modal Retrieval
Then, a Wasserstein coupled dictionary, containing multiple pairs of counterpart graph keys with each key corresponding to one modality, is constructed for further feature learning.
When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint
There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into the analysis of Langevin algorithms in MCMC sampling.
Interest-based Item Representation Framework for Recommendation with Multi-Interests Capsule Network
To make the framework model-agnostic, user Multi Interests Capsule Network is designed as an auxiliary task to jointly learn item-based item representations and interest-based item representations.
Improving Adversarial Defense with Self-supervised Test-time Fine-tuning
Although adversarial training and its variants currently constitute the most effective way to achieve robustness against adversarial attacks, their poor generalization limits their performance on the test samples.
DOMAIN ADAPTATION VIA DISTRIBUTION AND REPRESENTATION MATCHING: A CASE STUDY ON TRAINING DATA SELECTION VIA REINFORCEMENT LEARNING
Following the TDS methodology, in this paper, we propose a general data selection framework with representation learning and distribution matching simultaneously for domain adaptation on neural models.
Multi-objective Neural Architecture Search via Predictive Network Performance Optimization
Inspired by the nature of the graph structure of a neural network, we propose BOGCN-NAS, a NAS algorithm using Bayesian Optimization with Graph Convolutional Network (GCN) predictor.
On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training
This lets us show that the decay in generalization performance of adversarial training is a result of the model's attempt to fit hard adversarial instances.
Frequency-Aware Contrastive Learning for Neural Machine Translation
Inspired by the observation that low-frequency words form a more compact embedding space, we tackle this challenge from a representation learning perspective.
IDEA: Interpretable Dynamic Ensemble Architecture for Time Series Prediction
We enhance the accuracy and generalization of univariate time series point prediction by an explainable ensemble on the fly.
OneDConv: Generalized Convolution For Transform-Invariant Representation
Convolutional Neural Networks (CNNs) have exhibited their great power in a variety of vision tasks.
A Novel Multi-Task Learning Method for Symbolic Music Emotion Recognition
In this paper, we present a simple multi-task framework for SMER, which incorporates the emotion recognition task with other emotion-related auxiliary tasks derived from the intrinsic structure of the music.
Fast Rates in Pool-Based Batch Active Learning
We consider a batch active learning scenario where the learner adaptively issues batches of points to a labeling oracle.
Minimax Regret Optimization for Robust Machine Learning under Distribution Shift
We instead propose an alternative method called Minimax Regret Optimization (MRO), and show that under suitable conditions this method achieves uniformly low regret across all test distributions.
Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets
When the dataset does not have uniform coverage over all policy pairs, finding an approximate NE involves challenges in three aspects: (i) distributional shift between the behavior policy and the optimal policy, (ii) function approximation to handle large state space, and (iii) minimax optimization for equilibrium solving.
Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Sampling
Provably sample-efficient Reinforcement Learning (RL) with rich observations and function approximation has witnessed tremendous recent progress, particularly when the underlying function approximators are linear.
Leverage Your Local and Global Representations: A New Self-Supervised Learning Strategy
In essence, this strategy ignores the fact that two crops may truly contain different image information, e. g., background and small objects, and thus tends to restrain the diversity of the learned representations.
Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective
In this paper, we propose to model deep NRSfM from a sequence-to-sequence translation perspective, where the input 2D frame sequence is taken as a whole to reconstruct the deforming 3D non-rigid shape sequence.
Toward Knowledge-Enriched Conversational Recommendation Systems
Conversational Recommendation Systems recommend items through language based interactions with users. In order to generate naturalistic conversations and effectively utilize knowledge graphs (KGs) containing background information, we propose a novel Bag-of-Entities loss, which encourages the generated utterances to mention concepts related to the item being recommended, such as the genre or director of a movie.
Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions
We show that for both known $C$ and unknown $C$ cases, our algorithm with proper choice of hyperparameter achieves a regret that nearly matches the lower bounds.
MulT: An End-to-End Multitask Learning Transformer
At the heart of our approach is a shared attention mechanism modeling the dependencies across the tasks.
Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
We also extend our techniques to the two-player zero-sum Markov games (MGs), and establish a new performance lower bound for MGs, which tightens the existing result, and verifies the nearly minimax optimality of the proposed algorithm.
Bayesian Invariant Risk Minimization
Generalization under distributional shift is an open challenge for machine learning.
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization
This paper considers the generalization performance of differentially private convex learning.
An Indoor Environment Sensing and Localization System via mmWave Phased Array
At the end, we show that the reconstructed room layout can be utilized to locate a mobile device according to its AoA spectrum, even with single access point.
Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint
Overparameterized neural networks enjoy great representation power on complex data, and more importantly yield sufficiently smooth output, which is crucial to their generalization and robustness.
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
Motivated by this observation, we propose a new quantity, average drift at optimum, to measure the effects of data heterogeneity, and explicitly use it to present a new theoretical analysis of FedAvg.
Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexity
We propose a general framework to design posterior sampling methods for model-based RL.
Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Most prior results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i. e., the per-sample gradients are uniformly bounded.
A GAN-Based Short-Term Link Traffic Prediction Approach for Urban Road Networks Under a Parallel Learning Framework
This paper proposes a short-term traffic speed prediction approach, called PL-WGAN, for urban road networks, which is considered an important part of a novel parallel learning framework for traffic control and operation.
OpenMedIA: Open-Source Medical Image Analysis Toolbox and Benchmark under Heterogeneous AI Computing Platforms
In this paper, we present OpenMedIA, an open-source toolbox library containing a rich set of deep learning methods for medical image analysis under heterogeneous Artificial Intelligence (AI) computing platforms.
A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning
Thompson Sampling is one of the most effective methods for contextual bandits and has been generalized to posterior sampling for certain MDP settings.
Exploiting Hybrid Semantics of Relation Paths for Multi-hop Question Answering Over Knowledge Graphs
Answering natural language questions on knowledge graphs (KGQA) remains a great challenge in terms of understanding complex questions via multi-hop reasoning.
Asymptotic Statistical Analysis of $f$-divergence GAN
We show that for parametric generative models that are correctly specified, all $f$-divergence GANs with the same discriminator classes are asymptotically equivalent under suitable regularity conditions.
Optimal Operation of a Tidal Lagoon as a Flexible Source of Electricity
It was demonstrated that by exploiting the flexibility offered by the tidal lagoon, it can achieve a higher revenue in the day-ahead market, although their total electricity generation is reduced.
A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
Existing studies on provably efficient algorithms for Markov games (MGs) almost exclusively build on the "optimism in the face of uncertainty" (OFU) principle.
Multilingual Word Sense Disambiguation with Unified Sense Representation
As a key natural language processing (NLP) task, word sense disambiguation (WSD) evaluates how well NLP models can understand the lexical semantics of words under specific contexts.
Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective
Multimodal machine translation (MMT) aims to improve translation quality by equipping the source sentence with its corresponding image.
Large-Scale Bandwidth and Power Optimization for Multi-Modal Edge Intelligence Autonomous Driving
However, wireless channels between the edge server and the autonomous vehicles are time-varying due to the high-mobility of vehicles.
GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
The proposed algorithm modifies the standard posterior sampling algorithm in two aspects: (i) we use an optimistic prior distribution that biases towards hypotheses with higher values and (ii) a loglikelihood function is set to be the empirical loss evaluated on the historical data, where the choice of loss function supports both model-free and model-based learning.
Prior-mean-assisted Bayesian optimization application on FRIB Front-End tunning
Bayesian optimization~(BO) is often used for accelerator tuning due to its high sample efficiency.
DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata
Current Dynamic Texture Synthesis (DyTS) models can synthesize realistic videos.
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
With the popularity of Stein variational gradient descent (SVGD), the focus of particle-based VI algorithms has been on the properties of functions in Reproducing Kernel Hilbert Space (RKHS) to approximate the gradient flow.
FAF: A novel multimodal emotion recognition approach integrating face, body and text
In this paper, we developed a large multimodal emotion dataset, named "HED" dataset, to facilitate the emotion recognition task, and accordingly propose a multimodal emotion recognition method.
On Robust Observer Design for System Motion on SE(3) Using Onboard Visual Sensors
The key design idea for this observer is to estimate the visible set and identify the mis-identified features from the measurements.
Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision Processes
In this paper, we consider the contextual bandit with general function approximation and propose a computationally efficient algorithm to achieve a regret of $\tilde{O}(\sqrt{T}+\zeta)$.
VO$Q$L: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximation
We study time-inhomogeneous episodic reinforcement learning (RL) under general function approximation and sparse rewards.
