Search Results for author:
Found 280 papers, 35 papers with code
One sketch for all: Theory and Application of Conditional Random Sampling
Conditional Random Sampling (CRS) was originally proposed for efficiently computing pairwise ($l_2$, $l_1$) distances, in static, large-scale, and sparse data sets such as text and Web data.
b-Bit Minwise Hashing for Estimating Three-Way Similarities
Computing two-way and multi-way set similarities is a fundamental problem.
b-Bit Minwise Hashing for Large-Scale Linear SVM
Interestingly, our proof for the positive definiteness of the b-bit minwise hashing kernel naturally suggests a simple strategy to integrate b-bit hashing with linear SVM.
Hashing Algorithms for Large-Scale Learning
Minwise hashing is a standard technique in the context of search for efficiently computing set similarities.
One Permutation Hashing
While minwise hashing is promising for large-scale learning in massive binary data, the preprocessing cost is prohibitive as it requires applying (e. g.,) $k=500$ permutations on the data.
Entropy Estimations Using Correlated Symmetric Stable Random Projections
Methods for efficiently estimating the Shannon entropy of data streams have important applications in learning, data mining, and network anomaly detections (e. g., the DDoS attacks).
Sign Stable Projections, Sign Cauchy Projections and Chi-Square Kernels
The method of stable random projections is popular for efficiently computing the Lp distances in high dimension (where 0<p<=2), using small space.
Coding for Random Projections
The method of random projections has become very popular for large-scale applications in statistical learning, information retrieval, bio-informatics and other applications.
Compressed Counting Meets Compressed Sensing
In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.
Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics
We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e. g., Mercer kernels.
Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization
Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.
Sign Cauchy Projections and Chi-Square Kernel
The method of Cauchy random projections is popular for computing the $l_1$ distance in high dimension.
Beyond Pairwise: Provably Fast Algorithms for Approximate k-Way Similarity Search
We go beyond the notion of pairwise similarity and look into search problems with $k$-way similarity functions.
Adaptive Stochastic Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers (ADMM) has been studied for years.
Sparse Recovery with Very Sparse Compressed Counting
In this paper, we adopt very sparse Compressed Counting for nonnegative signal recovery.
Multi-label ensemble based on variable pairwise constraint projection
For the boosting-like strategy, we employ both the variable pairwise constraints and the bootstrap steps to diversify the base classifiers.
Coding for Random Projections and Approximate Near Neighbor Search
This technical note compares two coding (quantization) schemes for random projections in the context of sub-linear time approximate near neighbor search.
Recovery of Coherent Data via Low-Rank Dictionary Pursuit
More precisely, we mathematically prove that if the dictionary itself is low-rank then LRR is immune to the coherence parameter which increases with the underlying cluster number.
Advancing Matrix Completion by Modeling Extra Structures beyond Low-Rankness
To better handle non-uniform data, in this paper we propose a method termed Low-Rank Factor Decomposition (LRFD), which imposes an additional restriction that the data points must be represented as linear combinations of the bases in a dictionary constructed or learnt in advance.
A New Space for Comparing Graphs
We show that the proposed matrix representation encodes the spectrum of the underlying adjacency matrix and it also contains information about the counts of small sub-structures present in the graph such as triangles and small paths.
Graph Kernels via Functional Embedding
We propose a representation of graph as a functional object derived from the power iteration of the underlying adjacency matrix.
Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)
Our proposal is based on an interesting mathematical phenomenon in which inner products, after independent asymmetric transformations, can be converted into the problem of approximate near neighbor search.
Online Optimization for Large-Scale Max-Norm Regularization
Max-norm regularizer has been extensively studied in the last decade as it promotes an effective low-rank estimation for the underlying data.
Improved Densification of One Permutation Hashing
The existing work on densification of one permutation hashing reduces the query processing cost of the $(K, L)$-parameterized Locality Sensitive Hashing (LSH) algorithm with minwise hashing, from $O(dKL)$ to merely $O(d + KL)$, where $d$ is the number of nonzeros of the data vector, $K$ is the number of hashes in each hash table, and $L$ is the number of hash tables.
In Defense of MinHash Over SimHash
To provide a common basis for comparison, we evaluate retrieval results in terms of $\mathcal{S}$ for both MinHash and SimHash.
Compressed Sensing with Very Sparse Gaussian Random Projections
We have developed two estimators: (i) the {\em tie estimator}, and (ii) the {\em absolute minimum estimator}.
Recovery of Sparse Signals Using Multiple Orthogonal Least Squares
We study the problem of recovering sparse signals from compressed linear measurements.
Improved Asymmetric Locality Sensitive Hashing (ALSH) for Maximum Inner Product Search (MIPS)
In the prior work, the authors use asymmetric transformations which convert the problem of approximate MIPS into the problem of approximate near neighbor search which can be efficiently solved using hashing.
Asymmetric Minwise Hashing
Minwise hashing (Minhash) is a widely popular indexing scheme in practice.
Online Optimization for Max-Norm Regularization
The key technique in our algorithm is to reformulate the max-norm into a matrix factorization form, consisting of a basis component and a coefficients one.
Object Proposal with Kernelized Partial Ranking
In order to determine a small set of proposals with a high recall, a common scheme is extracting multiple features followed by a ranking algorithm which however, incurs two major challenges: {\bf 1)} The ranking model often imposes pairwise constraints between each proposal, rendering the problem away from an efficient training/testing phase; {\bf 2)} Linear kernels are utilized due to the computational and memory bottleneck of training a kernelized model.
Min-Max Kernels
Via an extensive empirical study, we show that this 0-bit scheme does not lose essential information.
One Scan 1-Bit Compressed Sensing
Based on $\alpha$-stable random projections with small $\alpha$, we develop a simple algorithm for compressed sensing (sparse signal recovery) by utilizing only the signs (i. e., 1-bit) of the measurements.
Efficient Online Minimization for Low-Rank Subspace Clustering
Low-rank representation~(LRR) has been a significant method for segmenting data that are generated from a union of subspaces.
Regularization-free estimation in trace regression with symmetric positive semidefinite matrices
Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing.
Sign Stable Random Projections for Large-Scale Learning
When $\alpha =2$, it is known that the corresponding nonlinear kernel is the arc-cosine kernel.
b-bit Marginal Regression
We consider the problem of sparse signal recovery from $m$ linear measurements quantized to $b$ bits.
2-Bit Random Projections, NonLinear Estimators, and Approximate Near Neighbor Search
In this paper, we focus on a simple 2-bit coding scheme.
A Comparison Study of Nonlinear Kernels
In this paper, we compare 5 different nonlinear kernels: min-max, RBF, fRBF (folded RBF), acos, and acos-$\chi^2$, on a wide range of publicly available datasets.
Methods for Sparse and Low-Rank Recovery under Simplex Constraints
The de-facto standard approach of promoting sparsity by means of $\ell_1$-regularization becomes ineffective in the presence of simplex constraints, i. e.,~the target is known to have non-negative entries summing up to a given constant.
A Tight Bound of Hard Thresholding
This paper is concerned with the hard thresholding operator which sets all but the $k$ largest absolute elements of a vector to zero.
Linearized GMM Kernels and Normalized Random Fourier Features
The variance of RFF is actually large.
Engineering Deep Representations for Modeling Aesthetic Perception
To remedy these problems, we develop a deep architecture to learn aesthetically-relevant visual attributes from Flickr1, which are localized by multiple textual attributes in a weakly-supervised setting.
Nystrom Method for Approximating the GMM Kernel
In order to use the GMM kernel for large-scale datasets, the prior work resorted to the (generalized) consistent weighted sampling (GCWS) to convert the GMM kernel to linear kernel.
Theory of the GMM Kernel
We prove the theoretical limit of GMM and the consistency result, assuming that the data follow an elliptical distribution, which is a very general family of distributions and includes the multivariate $t$-distribution as a special case.
Constrained Low-Rank Learning Using Least Squares-Based Regularization
To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization.
Relational Multi-Manifold Co-Clustering
In this study, we focus on improving the co-clustering performance via manifold ensemble learning, which is able to maximally approximate the intrinsic manifolds of both the sample and feature spaces.
Quantized Random Projections and Non-Linear Estimation of Cosine Similarity
Random projections constitute a simple, yet effective technique for dimensionality reduction with applications in learning and search problems.
Exact Recovery of Hard Thresholding Pursuit
In this paper, we bridge this gap by showing, for the first time, that exact recovery of the global sparse minimizer is possible for HTP-style methods under restricted strong condition number bounding conditions.
Learning Additive Exponential Family Graphical Models via \ell_{2,1}-norm Regularized M-Estimation
We investigate a subclass of exponential family graphical models of which the sufficient statistics are defined by arbitrary additive forms.
Generalized Intersection Kernel
Following the very recent line of work on the ``generalized min-max'' (GMM) kernel, this study proposes the ``generalized intersection'' (GInt) kernel and the related ``normalized generalized min-max'' (NGMM) kernel.
Tunable GMM Kernels
The linearized GMM kernel was extensively compared in with linearized radial basis function (RBF) kernel.
L2GSCI: Local to Global Seam Cutting and Integrating for Accurate Face Contour Extraction
Current face alignment algorithms can robustly find a set of landmarks along face contour.
On the Iteration Complexity of Support Recovery via Hard Thresholding Pursuit
Recovering the support of a sparse signal from its compressed samples has been one of the most important problems in high dimensional statistics.
Image matting with normalized weight and semi-supervised learning
Image matting is an important vision problem.
Simple strategies for recovering inner products from coarsely quantized random projections
Random projections have been increasingly adopted for a diverse set of tasks in machine learning involving dimensionality reduction.
Partial Hard Thresholding: Towards A Principled Analysis of Support Recovery
In machine learning and compressed sensing, it is of central importance to understand when a tractable algorithm recovers the support of a sparse signal from its compressed measurements.
Sign-Full Random Projections
At high similarity ($\rho\rightarrow1$), the asymptotic variance of recommended estimator is only $\frac{4}{3\pi} \approx 0. 4$ of the estimator for sign-sign projections.
Several Tunable GMM Kernels
In this study, we propose a series of "tunable GMM kernels" which are simple and perform largely comparably to tree methods on the same datasets.
Provable Variable Selection for Streaming Features
As a remedy, online feature selection has attracted increasing attention in recent years.
Fast Binary Functional Search on Graph
As Approximate Nearest Neighbor Search (ANNS) techniques have specifications on metric distances, efficient searching by advanced measures is still an open question.
Logician and Orator: Learning from the Duality between Language and Knowledge in Open Domain
We propose the task of Open-Domain Information Narration (OIN) as the reverse task of Open Information Extraction (OIE), to implement the dual structure between language and knowledge in the open domain.
RGB-D SLAM in Dynamic Environments Using Point Correlations
This method utilizes the correlation between map points to separate points that are part of the static scene and points that are part of different moving objects into different groups.
An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks.
Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization
The evaluator defines a learnable information preserving metric between original video and summary video and "supervises" the selector to identify the most informative frames to form the summary video.
Ranked #7 on
Unsupervised Video Summarization
on TvSum
Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction
In this paper, we consider the problem of open information extraction (OIE) for extracting entity and relation level intermediate structures from sentences in open-domain.
DEEP GEOMETRICAL GRAPH CLASSIFICATION
In the second step, the GNN is applied to the point-cloud representation of the graph provided by the embedding method.
On Random Deep Weight-Tied Autoencoders: Exact Asymptotic Analysis, Phase Transitions, and Implications to Training
We study the behavior of weight-tied multilayer vanilla autoencoders under the assumption of random weights.
Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process
Leveraging domain knowledge is an effective strategy for enhancing the quality of inferred low-dimensional representations of documents by topic models.
End-to-end Deep Reinforcement Learning Based Coreference Resolution
Recent neural network models have significantly advanced the task of coreference resolution.
Ranked #13 on
Coreference Resolution
on CoNLL 2012
A Two-Stage Approach to Multivariate Linear Regression with Sparsely Mismatched Data
A tacit assumption in linear regression is that (response, predictor)-pairs correspond to identical observational units.
On Convergence of Distributed Approximate Newton Methods: Globalization, Sharper Bounds and Beyond
We first introduce a simple variant of DANE equipped with backtracking line search, for which global asymptotic convergence and sharper local non-asymptotic convergence rate guarantees can be proved for both quadratic and non-quadratic strongly convex functions.
Multi-Spectral Visual Odometry without Explicit Stereo Matching
Moreover, the proposed method can also provide a metric 3D reconstruction in semi-dense density with multi-spectral information, which is not available from existing multi-spectral methods.
The Benefits of Diversity: Permutation Recovery in Unlabeled Sensing from Multiple Measurement Vectors
For the case in which both the signal and permutation are unknown, the problem is reformulated as a bi-convex optimization problem with an auxiliary variable, which can be solved by the Alternating Direction Method of Multipliers (ADMM).
Zeroth Order Optimization by a Mixture of Evolution Strategies
Perhaps the only methods that enjoy convergence guarantees are the ones that sample the perturbed points uniformly from a unit sphere or from a multivariate Gaussian distribution with an isotropic covariance.
Graph Analysis and Graph Pooling in the Spatial Domain
The proposed approach leverages a spatial representation of the graph which makes the neural network aware of the differences between the nodes and also their locations in the graph.
On Efficient Retrieval of Top Similarity Vectors
Retrieval of relevant vectors produced by representation learning critically influences the efficiency in natural language processing (NLP) tasks.
Reinforced Product Metadata Selection for Helpfulness Assessment of Customer Reviews
Given a product, a selector (agent) learns from both the keys in the product metadata and one of its reviews to take an action that selects the correct value, and a successive predictor (network) makes the free-text review attend to this value to obtain better neural representations for helpfulness assessment.
A Fourier Analytical Approach to Estimation of Smooth Functions in Gaussian Shift Model
Let $\mathbf{x}_j = \mathbf{\theta} + \mathbf{\epsilon}_j$, $j=1,\dots, n$ be i. i. d.
Outlier Detection and Robust PCA Using a Convex Measure of Innovation
In other word, an outlier carries some innovation with respect to most of the other data points.
Generalization Error Analysis of Quantized Compressive Learning
In this paper, we consider the learning problem where the projected data is further compressed by scalar quantization, which is called quantized compressive learning.
Towards Practical Alternating Least-Squares for CCA
To promote the practical use of ALS for CCA, we propose truly alternating least-squares.
Möbius Transformation for Fast Inner Product Search on Graph
We present a fast search on graph algorithm for Maximum Inner Product Search (MIPS).
Re-randomized Densification for One Permutation Hashing and Bin-wise Consistent Weighted Sampling
Jaccard similarity is widely used as a distance measure in many machine learning and search applications.
Random Projections with Asymmetric Quantization
The method of random projection has been a popular tool for data compression, similarity search, and machine learning.
Outlier Detection and Data Clustering via Innovation Search
In this paper, we present a new discovery that the directions of innovation can be used to design a provable and strong robust (to outlier) PCA method.
Structure-Feature based Graph Self-adaptive Pooling
To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes.
Selective Convolutional Network: An Efficient Object Detector with Ignoring Background
It is well known that attention mechanisms can effectively improve the performance of many CNNs including object detectors.
Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems
For example, a sponsored online advertising system can contain more than $10^{11}$ sparse features, making the neural network a massive model with around 10 TB parameters.
Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse.
Solving the Robust Matrix Completion Problem via a System of Nonlinear Equations
We consider the problem of robust matrix completion, which aims to recover a low rank matrix $L_*$ and a sparse matrix $S_*$ from incomplete observations of their sum $M=L_*+S_*\in\mathbb{R}^{m\times n}$.
An Inverse-free Truncated Rayleigh-Ritz Method for Sparse Generalized Eigenvalue Problem
Particularly, a new truncation strategy is proposed, which is able to find the support set of the leading eigenvector effectively.
IVFS: Simple and Efficient Feature Selection for High Dimensional Topology Preservation
Feature selection is an important tool to deal with high dimensional data.
Randomized Kernel Multi-view Discriminant Analysis
In this paper, we propose the kernel version of multi-view discriminant analysis, called kernel multi-view discriminant analysis (KMvDA).
Distributed Primal-Dual Optimization for Online Multi-Task Learning
Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness.
Estimate the Implicit Likelihoods of GANs with Application to Anomaly Detection
The thriving of deep models and generative models provides approaches to model high dimensional distributions.
Cooperative Rate-Splitting for Secrecy Sum-Rate Enhancement in Multi-antenna Broadcast Channels
In this paper, we employ Cooperative Rate-Splitting (CRS) technique to enhance the Secrecy Sum Rate (SSR) for the Multiple Input Single Output (MISO) Broadcast Channel (BC), consisting of two legitimate users and one eavesdropper, with perfect Channel State Information (CSI) available at all nodes.
Learning Interpretable Relationships between Entities, Relations and Concepts via Bayesian Structure Learning on Open Domain Facts
Compared to the state-of-the-art methods, the learned network structures help improving the identification of concepts for entities based on the relations of entities on both datasets.
Cross-Lingual Unsupervised Sentiment Classification with Multi-View Transfer Learning
Recent neural network models have achieved impressive performance on sentiment classification in English as well as other languages.
MeDaS: An open-source platform as service to help break the walls between medicine and informatics
To the best of our knowledge, MeDaS is the first open-source platform proving a collaborative and interactive service for researchers from a medical background easily using DL related toolkits, and at the same time for scientists or engineers from information sciences to understand the medical knowledge side.
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
Communication complexity and privacy are the two key challenges in Federated Learning where the goal is to perform a distributed learning through a large volume of devices.
Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentum
We construct a statistical diagnostic test for convergence to the stationary phase using the inner product between successive gradients and demonstrate that the proposed diagnostic works well.
A Framework of Randomized Selection Based Certified Defenses Against Data Poisoning Attacks
Random selection based defenses can achieve certified robustness by averaging the classifiers' predictions on the sub-datasets sampled from the training set.
Exploring global diverse attention via pairwise temporal relation for video summarization
In this paper, we propose an efficient convolutional neural network architecture for video SUMmarization via Global Diverse Attention called SUM-GDA, which adapts attention mechanism in a global perspective to consider pairwise temporal relations of video frames.
Tensor Completion via Tensor Networks with a Tucker Wrapper
In this paper, we propose to solve LRTC via tensor networks with a Tucker wrapper.
Identification of Matrix Joint Block Diagonalization
This paper considers the identification problem for BJBDP, that is, under what conditions and by what means, we can identify the diagonalizer $A$ and the block diagonal structure of $\Sigma_i$, especially when there is noise in $C_i$'s.
HWA: Hyperparameters Weight Averaging in Bayesian Neural Networks
Bayesian neural networks attempt to combine the strong predictive performance of neural networks with formal quantification of uncertainty of the predicted output in the Bayesian framework.
Towards Better Generalization of Adaptive Gradient Methods
Adaptive gradient methods such as AdaGrad, RMSprop and Adam have been optimizers of choice for deep learning due to their fast training speed.
Thunder: a Fast Coordinate Selection Solver for Sparse Learning
L1 regularization has been broadly employed to pursue model sparsity.
Optimal Prediction of the Number of Unseen Species with Multiplicity
Based on a sample of size $n$, we consider estimating the number of symbols that appear at least $\mu$ times in an independent sample of size $a \cdot n$, where $a$ is a given parameter.
Ratio Trace Formulation of Wasserstein Discriminant Analysis
We reformulate the Wasserstein Discriminant Analysis (WDA) as a ratio trace problem and present an eigensolver-based algorithm to compute the discriminative subspace of WDA.
RANet: Region Attention Network for Semantic Segmentation
In contrast to the previous methods, RANet configures the information pathways between the pixels in different regions, enabling the region interaction to exchange the regional context for enhancing all of the pixels in the image.
Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler
In this paper, we propose to learn a variational auto-encoder (VAE) to initialize the finite-step MCMC, such as Langevin dynamics that is derived from the energy function, for efficient amortized sampling of the EBM.
Temporal Cue Guided Video Highlight Detection With Low-Rank Audio-Visual Fusion
Video highlight detection plays an increasingly important role in social media content filtering, however, it remains highly challenging to develop automated video highlight detection methods because of the lack of temporal annotations (i. e., where the highlight moments are in long videos) for supervised learning.
MISSO: Minimization by Incremental Stochastic Surrogate Optimization for Large Scale Nonconvex and Nonsmooth Problems
Many constrained, nonconvex and nonsmooth optimization problems can be tackled using the majorization-minimization (MM) method which alternates between constructing a surrogate function which upper bounds the objective function, and then minimizing this surrogate.
Cross-Probe BERT for Efficient and Effective Cross-Modal Search
Inspired by the great success of BERT in NLP tasks, many text-vision BERT models emerged recently.
Learning Energy-Based Generative Models via Coarse-to-Fine Expanding and Sampling
Energy-based models (EBMs) for generative modeling parametrize a single net and can be directly trained by maximum likelihood estimation.
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks
Under this optimization framework, the trigger generator function will learn to manipulate the input with imperceptible noise to preserve the model performance on the clean data and maximize the attack success rate on the poisoned data.
Convergent Adaptive Gradient Methods in Decentralized Optimization
Specifically, we propose a general algorithmic framework that can convert existing adaptive gradient methods to their decentralized counterparts.
Robust Watermarking for Deep Neural Networks via Bi-Level Optimization
Deep neural networks (DNNs) have become state-of-the-art in many application domains.
Simulation on the Transparency of Electrons and Ion Back Flow for a Time Projection Chamber based on Staggered Multiple THGEMs
The IBF and the transparent rate of electrons are two essential indicators of TPC, which affect the energy resolution and counting rate respectively.
Instrumentation and Detectors High Energy Physics - Experiment
Quantization Algorithms for Random Fourier Features
Closely related to RP, the method of random Fourier features (RFF) has also become popular, for approximating the Gaussian kernel.
High-quality Low-dose CT Reconstruction Using Convolutional Neural Networks with Spatial and Channel Squeeze and Excitation
Low-dose computed tomography (CT) allows the reduction of radiation risk in clinical applications at the expense of image quality, which deteriorates the diagnosis accuracy of radiologists.
Mitigating Forgetting in Online Continual Learning with Neuron Calibration
Though recent studies have achieved remarkable progress in improving the online continual learning performance empowered by the deep neural networks-based models, many of today's approaches still suffer a lot from catastrophic forgetting, a persistent challenge for continual learning.
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval
Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English.
S$^2$-MLP: Spatial-Shift MLP Architecture for Vision
We discover that the token-mixing MLP is a variant of the depthwise convolution with a global reception field and spatial-specific configuration.
Learning Deep Latent Variable Models by Short-Run MCMC Inference With Optimal Transport Correction
Learning latent variable models with deep top-down architectures typically requires inferring the latent variables for each training example based on the posterior distribution of these latent variables.
A Multi-Task Network for Joint Specular Highlight Detection and Removal
Specular highlight detection and removal are fundamental and challenging tasks.
Patchwise Generative ConvNet: Training Energy-Based Models From a Single Natural Image for Internal Learning
Exploiting internal statistics of a single natural image has long been recognized as a significant research paradigm where the goal is to learn the distribution of patches within the image without relying on external training data.
Closed-Form, Provable, and Robust PCA via Leverage Statistics and Innovation Search
In the application of Innovation Search for outlier detection, the directions of innovation were utilized to measure the innovation of the data points.
A Systematic Collection of Medical Image Datasets for Deep Learning
Thus, as comprehensive as possible, this paper provides a collection of medical image datasets with their associated challenges for deep learning research.
Rethinking Token-Mixing MLP for MLP-based Vision Backbone
By introducing the inductive bias from the image processing, convolution neural network (CNN) has achieved excellent performance in numerous computer vision tasks and has been established as \emph{de facto} backbone.
S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision
More recently, using smaller patches with a pyramid structure, Vision Permutator (ViP) and Global Filter Network (GFNet) achieve better performance than S$^2$-MLP.
Provable Data Clustering via Innovation Search
It is shown that in contrast to most of the existing methods which require the subspaces to be sufficiently incoherent with each other, Innovation Pursuit only requires the innovative components of the subspaces to be sufficiently incoherent with each other.
Non-Local Feature Aggregation on Graphs via Latent Fixed Data Structures
The proposed approach is used to design several novel global feature aggregation methods based on the choice of the LFDS.
On the Convergence of Decentralized Adaptive Gradient Methods
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks.
C-MinHash: Rigorously Reducing $K$ Permutations to Two
Unlike classical MinHash, these $K$ hashes are obviously correlated, but we are able to provide rigorous proofs that we still obtain an unbiased estimate of the Jaccard similarity and the theoretical variance is uniformly smaller than that of the classical MinHash with $K$ independent permutations.
Extreme Bandits using Robust Statistics
We consider a multi-armed bandit problem motivated by situations where only the extreme values, as opposed to expected values in the classical bandit setting, are of interest.
Toward Communication Efficient Adaptive Gradient Method
While adaptive gradient methods have been proven effective for training neural nets, the study of adaptive gradient methods in federated learning is scarce.
C-MinHash: Practically Reducing Two Permutations to Just One
That is, one single permutation is used for both the initial pre-processing step to break the structures in the data and the circulant hashing step to generate $K$ hashes.
Discriminative Similarity for Data Clustering
Similarity-based clustering methods separate data into clusters according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance.
C-MinHash: Improving Minwise Hashing with Circulant Permutation
Minwise hashing (MinHash) is an important and practical algorithm for generating random hashes to approximate the Jaccard (resemblance) similarity in massive binary (0/1) data.
Optimal Transport for Long-Tailed Recognition with Learnable Cost Matrix
It is attracting attention to the long-tailed recognition problem, a burning issue that has become very popular recently.
Ranked #42 on
Long-tail Learning
on CIFAR-100-LT (ρ=100)
A new look at fairness in stochastic multi-armed bandit problems
Under such a framework, we propose a hard-threshold UCB-like algorithm, which enjoys many merits including asymptotic fairness, nearly optimal regret, better tradeoff between reward and fairness.
Lifting Imbalanced Regression with Self-Supervised Learning
In this paper, we work on a seamless marriage of imbalanced regression and self-supervised learning.
Fairness-aware Federated Learning
Federated Learning is a machine learning technique where a network of clients collaborates with a server to learn a centralized model while keeping data localized.
Practical Adversarial Training with Differential Privacy for Deep Learning
In this work, we propose the practical adversarial training with differential privacy (DP-Adv), to combine the backbones from both communities and deliver robust and private models with high accuracy.
Constructing Orthogonal Convolutions in an Explicit Manner
A convolution layer with an orthogonal Jacobian matrix is 1-Lipschitz in the 2-norm, making the output robust to the perturbation in input.
Revisiting Locality-Sensitive Binary Codes from Random Fourier Features
We show the locality-sensitivity of SignRFF, and propose a new measure, called ranking efficiency, to theoretically compare different Locality-Sensitive Hashing (LSH) methods with practical implications.
k-Median Clustering via Metric Embedding: Towards Better Initialization with Privacy
Our method, named the HST initialization, can also be easily extended to the setting of differential privacy (DP) to generate private initial centers.
Unsupervised Contrastive Learning for Signal-Dependent Noise Synthesis
We present a simple yet robust noise synthesis framework based on unsupervised contrastive learning.
Boosting the Confidence of Near-Tight Generalization Bounds for Uniformly Stable Randomized Algorithms
We further substantialize these generic results to SGD to derive improved high probability generalization bounds for convex or non-convex optimization with natural time decaying learning rates, which have not been possible to prove with the existing uniform stability results.
$f$-Divergence Thermodynamic Variational Objective: a Deformed Geometry Perspective
In this paper, we propose a $f$-divergence Thermodynamic Variational Objective ($f$-TVO).
Causal Discovery via Cholesky Factorization
Discovering the causal relationship via recovering the directed acyclic graph (DAG) structure from the observed data is a challenging combinatorial problem.
Layer-wise and Dimension-wise Locally Adaptive Federated Learning
In the emerging paradigm of Federated Learning (FL), large amount of clients such as mobile devices are used to train possibly high-dimensional models on their respective data.
MVT: Multi-view Vision Transformer for 3D Object Recognition
Nevertheless, multi-view CNN models cannot model the communications between patches from different views, limiting its effectiveness in 3D object recognition.
Ranked #2 on
3D Object Recognition
on ModelNet40
C-OPH: Improving the Accuracy of One Permutation Hashing (OPH) with Circulant Permutations
Note that C-MinHash is different from the well-known work on "One Permutation Hashing (OPH)" published in NIPS'12.
Mitigating Forgetting in Online Continual Learning with Neuron Calibration
Though recent studies have achieved remarkable progress in improving the online continual learning performance empowered by the deep neural networks-based models, many of today's approaches still suffer a lot from catastrophic forgetting, a persistent challenge for continual learning.
A Note on Sparse Generalized Eigenvalue Problem
The sparse generalized eigenvalue problem (SGEP) aims to find the leading eigenvector with sparsity structure.
A Comprehensively Tight Analysis of Gradient Descent for PCA
We further give the first worst-case analysis that achieves a rate of convergence at $O(\frac{1}{\epsilon}\log\frac{1}{\epsilon})$.
Backdoor Attack with Imperceptible Input and Latent Modification
Many existing countermeasures found that backdoor tends to leave tangible footprints in the latent or feature space, which can be utilized to mitigate backdoor attacks. In this paper, we extend the concept of imperceptible backdoor from the input space to the latent representation, which significantly improves the effectiveness against the existing defense mechanisms, especially those relying on the distinguishability between clean inputs and backdoor inputs in latent space.
Rate-Optimal Subspace Estimation on Random Graphs
We show that the performance of estimating the connectivity matrix $M$ depends on the sparsity of the graph.
Causal Discovery with Flow-based Conditional Density Estimation
Then we formulate a novel evaluation metric to infer the scores for each potential causal direction based on the variance of the conditional density estimation.
Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction
In this paper, we take a step further by proposing a novel generative vision transformer with latent variables following an informative energy-based prior for salient object detection.
Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
The depth estimation branch is trained with RGB-D images and then used to estimate the pseudo depth maps for all unlabeled RGB images to form the paired data.
Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising
In this paper, we propose a hardware-aware training workflow that couples the hardware topology into the algorithm design.
GCWSNet: Generalized Consistent Weighted Sampling for Scalable and Accurate Training of Neural Networks
For example, one can apply GCWS on the outputs of the last layer to boost the accuracy of trained deep neural networks.
BOAT: Bilateral Local Attention Vision Transformer
To improve efficiency, recent Vision Transformers adopt local self-attention mechanisms, where self-attention is computed within local windows.
On the Power-Law Hessian Spectrums in Deep Learning
It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning.
Label-Smoothed Backdoor Attack
In order to achieve a high attack success rate using as few poisoned training samples as possible, most existing attack methods change the labels of the poisoned samples to the target class.
MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting
As a result, the final method takes the advantage of effective semantic & image-level filling for high-fidelity inpainting.
Stability and Risk Bounds of Iterative Hard Thresholding
In this paper, we analyze the generalization performance of the Iterative Hard Thresholding (IHT) algorithm widely used for sparse recovery problems.
A Class of Two-Timescale Stochastic EM Algorithms for Nonconvex Latent Variable Models
We motivate the choice of a double dynamic by invoking the variance reduction virtue of each stage of the method on both sources of noise: the index sampling for the incremental update and the MC approximation.
An Energy-Based Prior for Generative Saliency
We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution.
SpaceE: Knowledge Graph Embedding by Relational Linear Transformation in the Entity Space
Translation distance based knowledge graph embedding (KGE) methods, such as TransE and RotatE, model the relation in knowledge graphs as translation or rotation in the vector space.
Multi-view Geometry: Correspondences Refinement Based on Algebraic Properties
To our knowledge, there are no theoretical results for multi-view correspondences prior to this paper.
On Distributed Adaptive Optimization with Gradient Compression
We study COMP-AMS, a distributed optimization framework based on gradient averaging and adaptive AMSGrad algorithm.
A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model
Under the short-run non-mixing MCMC scenario, the estimation of the energy-based model is shown to follow the perturbation of maximum likelihood, and the short-run Langevin flow and the normalizing flow form a two-flow generator that we call CoopFlow.
Dataset Pruning: Reducing Training Data by Examining Generalization Influence
To answer these, we propose dataset pruning, an optimization-based sample selection method that can (1) examine the influence of removing a particular set of training samples on model's generalization ability with theoretical guarantee, and (2) construct the smallest subset of training data that yields strictly constrained generalization gap.
Fast ABC-Boost: A Unified Framework for Selecting the Base Class in Multi-Class Classification
Our framework has parameters $(s, g, w)$.
Revisiting the role of heterophily in graph representation learning: An edge classification perspective
Alternatively, it is possible to exploit the information about the presence of heterophilous neighbors for feature learning, so a hybrid message passing approach is devised to aggregate homophilious neighbors and diversify heterophilous neighbors based on edge classification.
One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matching
However, in the existing deep supervised hashing methods, coding balance and low-quantization error are difficult to achieve and involve several losses.
Boosting the Confidence of Generalization for $L_2$-Stable Randomized Learning Algorithms
We further substantialize these generic results to stochastic gradient descent (SGD) to derive improved high-probability generalization bounds for convex or non-convex optimization problems with natural time decaying learning rates, which have not been possible to prove with the existing hypothesis stability or uniform stability based results.
On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond
The FedProx algorithm is a simple yet powerful distributed proximal point optimization method widely used for federated learning (FL) over heterogeneous data.
From a few Accurate 2D Correspondences to 3D Point Clouds
This article proposes a new method to estimate the world points and projection matrices from their correspondences.
Proximity Graph Maintenance for Fast Online Nearest Neighbor Search
Given a vertex deletion request, we thoroughly investigate solutions to update the connections of the vertex.
Noisy $\ell^{0}$-Sparse Subspace Clustering on Dimensionality Reduced Data
Our results provide theoretical guarantee on the correctness of noisy $\ell^{0}$-SSC in terms of SDP on noisy data for the first time, which reveals the advantage of noisy $\ell^{0}$-SSC in terms of much less restrictive condition on subspace affinity.
CGAR: Critic Guided Action Redistribution in Reinforcement Leaning
As discussed in this paper, under the settings of the off-policy actor critic algorithms, we demonstrate that the critic can bring more expected discounted rewards than or at least equal to the actor.
Defending Backdoor Attacks on Vision Transformer via Patch Processing
We first examine the vulnerability of ViTs against various backdoor attacks and find that ViTs are also quite vulnerable to existing attacks.
$k$-Median Clustering via Metric Embedding: Towards Better Initialization with Differential Privacy
When designing clustering algorithms, the choice of initial centers is crucial for the quality of the learned clusters.
DeepAuth: A DNN Authentication Framework by Model-Unique and Fragile Signature Embedding
After embedding, each model will respond distinctively to these key samples, which creates a model-unique signature as a strong tool for authentication and user identity.
Best Subset Selection with Efficient Primal-Dual Algorithm
Best subset selection is considered the `gold standard' for many sparse learning problems.
Variational Flow Graphical Model
VFGs learn the representation of high dimensional data via a message-passing scheme by integrating flow-based functions through variational inference.
Package for Fast ABC-Boost
Although the gain formula in Li (2010) was derived for logistic regression loss, it is a generic formula for loss functions with second-derivatives.
pGMM Kernel Regression and Comparisons with Boosted Trees
In recent prior studies, the pGMM kernel has been extensively evaluated for classification tasks, for logistic regression, support vector machines, as well as deep neural networks.
Catoni-style Confidence Sequences under Infinite Variance
In this paper, we provide an extension of confidence sequences for settings where the variance of the data-generating distribution does not exist or is infinite.
NL2GDPR: Automatically Develop GDPR Compliant Android Application Features from Natural Language
At the core, NL2GDPR is a privacy-centric information extraction model, appended with a GDPR policy finder and a policy generator.
Tree-based Text-Vision BERT for Video Search in Baidu Video Advertising
How to pair the video ads with the user search is the core task of Baidu video advertising.
