Search Results for author:
Found 67 papers, 14 papers with code
Spectral Graph Matching and Regularized Quadratic Relaxations: Algorithm and Theory
Graph matching, also known as network alignment, aims at recovering the latent vertex correspondence between two unlabeled, edge-correlated weighted graphs.
Enabling Fast 2-bit LLM on GPUs: Memory Alignment, Sparse Outlier, and Asynchronous Dequantization
Weights are quantized by groups, while the ranges of weights are large in some groups, resulting in large quantization errors and nonnegligible accuracy loss (e. g. >3% for Llama2-7b with 2-bit quantization in GPTQ and Greenbit).
FlashDecoding++: Faster Large Language Model Inference on GPUs
A single and static dataflow may lead to a 50. 25% performance loss for GEMMs of different shapes in LLM inference.
Federated Learning in the Presence of Adversarial Client Unavailability
In this paper, we relax those assumptions and consider adversarial client unavailability.
Random graph matching at Otter's threshold via counting chandeliers
We propose an efficient algorithm for graph matching based on similarity scores constructed from counting a certain family of weighted trees rooted at each vertex.
Global Convergence of Federated Learning for Mixed Regression
This paper studies the problem of model training under Federated Learning when clients exhibit cluster structure.
SeedGNN: Graph Neural Networks for Supervised Seeded Graph Matching
However, most previous GNNs for this task use a semi-supervised approach, which requires a large number of seeds and cannot learn knowledge that is transferable to unseen graphs.
Random Graph Matching in Geometric Models: the Case of Complete Graphs
This paper studies the problem of matching two complete graphs with edge weights correlated through latent geometries, extending a recent line of research on random graph matching with independent edge weights to geometric models.
LiMuSE: Lightweight Multi-modal Speaker Extraction
Multi-modal cues, including spatial information, facial expression and voiceprint, are introduced to the speech separation and speaker extraction tasks to serve as complementary information to achieve better performance.
Testing network correlation efficiently via counting trees
We propose a new procedure for testing whether two networks are edge-correlated through some latent vertex correspondence.
A Non-parametric View of FedAvg and FedProx: Beyond Stationary Points
We discover that when the data heterogeneity is moderate, a client with limited local data can benefit from a common model with a large federation gain.
WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environments
In the speaker extraction problem, it is found that additional information from the target speaker contributes to the tracking and extraction of the target speaker, which includes voiceprint, lip movement, facial expression, and spatial information.
One-pass Stochastic Gradient Descent in Overparametrized Two-layer Neural Networks
There has been a recent surge of interest in understanding the convergence of gradient descent (GD) and stochastic gradient descent (SGD) in overparameterized neural networks.
MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation
The spatial self-attention module is designed to attend on the cross-channel correlation in the covariance matrices.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
The planted matching problem: Sharp threshold and infinite-order phase transition
Conversely, if $\sqrt{d} B(\mathcal{P},\mathcal{Q}) \ge 1+\epsilon$ for an arbitrarily small constant $\epsilon>0$, the reconstruction error for any estimator is shown to be bounded away from $0$ under both the sparse and dense model, resolving the conjecture in [Moharrami et al. 2019, Semerjian et al. 2020].
The Power of $D$-hops in Matching Power-Law Graphs
Under the Chung-Lu random graph model with $n$ vertices, max degree $\Theta(\sqrt{n})$, and the power-law exponent $2<\beta<3$, we show that as soon as $D> \frac{4-\beta}{3-\beta}$, by optimally choosing the first slice, with high probability our algorithm can correctly match a constant fraction of the true pairs without any error, provided with only $\Omega((\log n)^{4-\beta})$ initial seeds.
Learner-Private Convex Optimization
Convex optimization with feedback is a framework where a learner relies on iterative queries and feedback to arrive at the minimizer of a convex function.
Settling the Sharp Reconstruction Thresholds of Random Graph Matching
This paper studies the problem of recovering the hidden vertex correspondence between two edge-correlated random graphs.
Audio-visual Speech Separation with Adversarially Disentangled Visual Representation
In our model, we use the face detector to detect the number of speakers in the scene and use visual information to avoid the permutation problem.
Testing correlation of unlabeled random graphs
We study the problem of detecting the edge correlation between two random graphs with $n$ unlabeled nodes.
Speaker-Conditional Chain Model for Speech Separation and Extraction
With the predicted speaker information from whole observation, our model is helpful to solve the problem of conventional speech separation and speaker extraction for multi-round long recordings.
Audio and Speech Processing Sound
Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
This model additionally has a simple and efficient stop criterion for the end of the transduction, making it able to infer the variable number of output sequences.
Ranked #2 on
Speech Separation
on WSJ0-5mix
Graph Matching with Partially-Correct Seeds
We establish non-asymptotic performance guarantees of perfect matching for both $1$-hop and $2$-hop algorithms, showing that our new $2$-hop algorithm requires substantially fewer correct seeds than the $1$-hop algorithm when graphs are sparse.
DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog
Visual Dialog is a vision-language task that requires an AI agent to engage in a conversation with humans grounded in an image.
The Planted Matching Problem: Phase Transitions and Exact Results
We study the problem of recovering a planted matching in randomly weighted complete bipartite graphs $K_{n, n}$.
Consistent recovery threshold of hidden nearest neighbor graphs
Motivated by applications such as discovering strong ties in social networks and assembling genome subsequences in biology, we study the problem of recovering a hidden $2k$-nearest neighbor (NN) graph in an $n$-vertex complete graph, whose edge weights are independent and distributed according to $P_n$ for edges in the hidden $2k$-NN graph and $Q_n$ otherwise.
Optimal query complexity for private sequential learning against eavesdropping
We study the query complexity of a learner-private sequential learning problem, motivated by the privacy and security concerns due to eavesdropping that arise in practical applications such as pricing and Federated Learning.
Spectral Graph Matching and Regularized Quadratic Relaxations II: Erdős-Rényi Graphs and Universality
We analyze a new spectral graph matching algorithm, GRAph Matching by Pairwise eigen-Alignments (GRAMPA), for recovering the latent vertex correspondence between two unlabeled, edge-correlated weighted graphs.
Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model
Departing from prior spectral approaches that only compare top eigenvectors, or eigenvectors of the same order, GRAMPA first constructs a similarity matrix as a weighted sum of outer products between all pairs of eigenvectors of the two graphs, with weights given by a Cauchy kernel applied to the separation of the corresponding eigenvalues, then outputs a matching by a simple rounding procedure.
A Working Memory Model for Task-oriented Dialog Response Generation
Our WMM2Seq adopts a working memory to interact with two separated long-term memories, which are the episodic memory for memorizing dialog history and the semantic memory for storing KB tuples.
The World in My Mind: Visual Dialog with Adversarial Multi-modal Feature Encoding
Visual Dialog is a multi-modal task that requires a model to participate in a multi-turn human dialog grounded on an image, and generate correct, human-like responses.
POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion
In particular, there exist two requirements for fashion outfit recommendation: the Compatibility of the generated fashion outfits, and the Personalization in the recommendation process.
Efficient random graph matching via degree profiles
This work develops an $\tilde{O}(n d^2+n^2)$-time algorithm which perfectly recovers the true vertex correspondence with high probability, provided that the average degree is at least $d = \Omega(\log^2 n)$ and the two graphs differ by at most $\delta = O( \log^{-2}(n) )$ fraction of edges.
Concept Learning through Deep Reinforcement Learning with Memory-Augmented Neural Networks
In this paper, we present a memory-augmented neural network which is motivated by the process of human concept learning.
Convex Relaxation Methods for Community Detection
We introduce some important theoretical techniques and results for establishing the consistency of convex community detection under various statistical models.
Cascaded Mutual Modulation for Visual Reasoning
Our code is available at https://github. com/FlamingHorizon/CMM-VR.
Seeded Graph Matching via Large Neighborhood Statistics
We study a well known noisy model of the graph isomorphism problem.
Securing Distributed Gradient Descent in High Dimensional Statistical Learning
Nevertheless, the empirical risk (sample version) is allowed to be non-convex.
Hidden Hamiltonian Cycle Recovery via Linear Programming
Represented as bicolored multi-graphs, these extreme points are further decomposed into simpler "blossom-type" structures for the large deviation analysis and counting arguments.
Rates of Convergence of Spectral Methods for Graphon Estimation
Recent studies have identified the minimax error rate of estimating the graphon from a single realization of the random graph.
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
The total computational complexity of our algorithm is of $O((Nd/m) \log N)$ at each working machine and $O(md + kd \log^3 N)$ at the central server, and the total communication cost is of $O(m d \log N)$.
Learning from Comparisons and Choices
This also allows one to compute similarities among users and items to be used for categorization and search.
Self-Taught Convolutional Neural Networks for Short Text Clustering
Short text clustering is a challenging problem due to its sparseness of text representation.
Ranked #2 on
Short Text Clustering
on Stackoverflow
Combining Lexical and Semantic-based Features for Answer Sentence Selection
As the result of the evaluation shows, our solution provides a valuable and brief model which could be used in modelling question answering or sentence semantic relevance.
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
To integrate the features on both dimensions of the matrix, this paper explores applying 2D max pooling operation to obtain a fixed-length representation of the text.
Ranked #5 on
Text Classification
on TREC-6
Hierarchical Memory Networks for Answer Selection on Unknown Words
Recently, end-to-end memory networks have shown promising results on Question Answering task, which encode the past facts into an explicit memory and perform reasoning ability by making multiple computational steps on the memory.
Semidefinite Programs for Exact Recovery of a Hidden Community
We study a semidefinite programming (SDP) relaxation of the maximum likelihood estimation for exactly recovering a hidden community of cardinality $K$ from an $n \times n$ symmetric data matrix $A$, where for distinct indices $i, j$, $A_{ij} \sim P$ if $i, j$ are both in the community and $A_{ij} \sim Q$ otherwise, for two known probability distributions $P$ and $Q$.
Convexified Modularity Maximization for Degree-corrected Stochastic Block Models
We establish non-asymptotic theoretical guarantees for both approximate clustering and perfect clustering.
Submatrix localization via message passing
The principal submatrix localization problem deals with recovering a $K\times K$ principal submatrix of elevated mean $\mu$ in a large $n\times n$ symmetric matrix subject to additive standard Gaussian noise.
Recovering a Hidden Community Beyond the Kesten-Stigum Threshold in $O(|E| \log^*|V|)$ Time
We show that a belief propagation algorithm achieves weak recovery if $\lambda>1/e$, beyond the Kesten-Stigum threshold by a factor of $1/e.$ The belief propagation algorithm only needs to run for $\log^\ast n+O(1) $ iterations, with the total time complexity $O(|E| \log^*n)$, where $\log^*n$ is the iterated logarithm of $n.$ Conversely, if $\lambda \leq 1/e$, no local algorithm can asymptotically outperform trivial random guessing.
Information Limits for Recovering a Hidden Community
We study the problem of recovering a hidden community of cardinality $K$ from an $n \times n$ symmetric data matrix $A$, where for distinct indices $i, j$, $A_{ij} \sim P$ if $i, j$ both belong to the community and $A_{ij} \sim Q$ otherwise, for two known probability distributions $P$ and $Q$ depending on $n$.
Density Evolution in the Degree-correlated Stochastic Block Model
There is a recent surge of interest in identifying the sharp recovery thresholds for cluster recovery under the stochastic block model.
Local Algorithms for Block Models with Side Information
There has been a recent interest in understanding the power of local algorithms for optimization and inference problems on sparse graphs.
Convolutional Neural Networks for Text Hashing
Meanwhile word features and position features are together fed into a convolutional network to learn the implicit features which are further incorporated with the explicit features to fit the pretrained binary code.
Collaboratively Learning Preferences from Ordinal Data
In order to predict the preferences, we want to learn the underlying model from noisy observations of the low-rank matrix, collected as revealed preferences in various forms of ordinal data.
Short Text Hashing Improved by Integrating Multi-Granularity Topics and Tags
However, topics of certain granularity are not adequate to represent the intrinsic semantic information.
Achieving Exact Cluster Recovery Threshold via Semidefinite Programming: Extensions
Extending the proof techniques, in this paper it is shown that SDP relaxations also achieve the sharp recovery threshold in the following cases: (1) Binary stochastic block model with two clusters of sizes proportional to network size but not necessarily equal; (2) Stochastic block model with a fixed number of equal-sized clusters; (3) Binary censored block model with the background graph being Erd\H{o}s-R\'enyi.
Clustering and Inference From Pairwise Comparisons
We propose an efficient algorithm that accurately estimates the individual preferences for almost all users, if there are $r \max \{m, n\}\log m \log^2 n$ pairwise comparisons per type, which is near optimal in sample complexity when $r$ only grows logarithmically with $m$ or $n$.
Reconstruction in the Labeled Stochastic Block Model
The labeled stochastic block model is a random graph model representing networks with community structure and interactions of multiple types.
Achieving Exact Cluster Recovery Threshold via Semidefinite Programming
The binary symmetric stochastic block model deals with a random graph of $n$ vertices partitioned into two equal-sized clusters, such that each pair of vertices is connected independently with probability $p$ within clusters and $q$ across clusters.
Edge Label Inference in Generalized Stochastic Block Models: from Spectral Theory to Impossibility Results
The classical setting of community detection consists of networks exhibiting a clustered structure.
Computational Lower Bounds for Community Detection on Random Graphs
This paper studies the problem of detecting the presence of a small dense community planted in a large Erd\H{o}s-R\'enyi random graph $\mathcal{G}(N, q)$, where the edge probability within the community exceeds $q$ by a constant factor.
Minimax-optimal Inference from Partial Rankings
For a given assignment of items to users, we first derive an oracle lower bound of the estimation error that holds even for the more general Thurstone models.
Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatrices
Of particular interest is the setting where the number of clusters/submatrices is allowed to grow unbounded with the problem size.
Jointly Clustering Rows and Columns of Binary Matrices: Algorithms and Trade-offs
In standard clustering problems, data points are represented by vectors, and by stacking them together, one forms a data matrix with row or column cluster structure.
