Search Results for author:
Found 56 papers, 19 papers with code
Estimation of entropy-regularized optimal transport maps between non-compactly supported measures
In the case that the target measure is compactly supported or strongly log-concave, we show that for a recently proposed in-sample estimator, the expected squared $L^2$-error decays at least as fast as $O(n^{-1/3})$ where $n$ is the sample size.
On neural and dimensional collapse in supervised and unsupervised contrastive learning with hard negative sampling
For a widely-studied data model and general loss and sample-hardening functions we prove that the Supervised Contrastive Learning (SCL), Hard-SCL (HSCL), and Unsupervised Contrastive Learning (UCL) risks are minimized by representations that exhibit Neural Collapse (NC), i. e., the class means form an Equianglular Tight Frame (ETF) and data from the same class are mapped to the same representation.
Systematic comparison of semi-supervised and self-supervised learning for medical image classification
Yet past benchmarks do not focus on medical tasks and rarely compare self- and semi- methods together on an equal footing.
A principled approach to model validation in domain generalization
Domain generalization aims to learn a model with good generalization ability, that is, the learned model should not only perform well on several seen domains but also on unseen domains with different data distributions.
On Rank Energy Statistics via Optimal Transport: Continuity, Convergence, and Change Point Detection
This paper considers the use of recently proposed optimal transport-based multivariate test statistics, namely rank energy and its variant the soft rank energy derived from entropically regularized optimal transport, for the unsupervised nonparametric change point detection (CPD) problem.
Alternating minimization algorithm with initialization analysis for r-local and k-sparse unlabeled sensing
The unlabeled sensing problem is to recover an unknown signal from permuted linear measurements.
Trade-off between reconstruction loss and feature alignment for domain generalization
To deal with challenging settings in DG where both data and label of the unseen domain are not available at training time, the most common approach is to design the classifiers based on the domain-invariant representation features, i. e., the latent representations that are unchanged and transferable between domains.
Geometric Sparse Coding in Wasserstein Space
We show this approach leads to sparse representations in Wasserstein space and addresses the problem of non-uniqueness of barycentric representation.
Nonparametric and Regularized Dynamical Wasserstein Barycenters for Sequential Observations
We consider probabilistic models for sequential observations which exhibit gradual transitions among a finite number of states.
Supervised Contrastive Learning with Hard Negative Samples
In this paper, motivated by the effectiveness of hard-negative sampling strategies in H-UCL and the usefulness of label information in SCL, we propose a contrastive learning framework called hard-negative supervised contrastive learning (H-SCL).
Joint covariate-alignment and concept-alignment: a framework for domain generalization
Particularly, our framework proposes to jointly minimize both the covariate-shift as well as the concept-shift between the seen domains for a better performance on the unseen domain.
Easy Variational Inference for Categorical Models via an Independent Binary Approximation
This approximation makes inference straightforward and fast; using well-known auxiliary variables for probit or logistic regression, the product of binary models admits conjugate closed-form variational inference that is embarrassingly parallel across categories and invariant to category ordering.
Measure Estimation in the Barycentric Coding Model
Our first main result leverages the Riemannian geometry of Wasserstein-2 space to provide a procedure for recovering the barycentric coordinates as the solution to a quadratic optimization problem assuming access to the true reference measures.
Conditional entropy minimization principle for learning domain invariant representation features
Invariance-principle-based methods such as Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG).
Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning
We study the problem of designing hard negative sampling distributions for unsupervised contrastive representation learning.
Interpretable contrastive word mover's embedding
This paper shows that a popular approach to the supervised embedding of documents for classification, namely, contrastive Word Mover's Embedding, can be significantly enhanced by adding interpretability.
Multivariate rank via entropic optimal transport: sample efficiency and generative modeling
We leverage this result to demonstrate fast convergence of sample sRE and sRMMD to their population version making them useful for high-dimensional GoF testing.
r-local sensing: Improved algorithm and applications
The unlabeled sensing problem is to solve a noisy linear system of equations under unknown permutation of the measurements.
Dynamical Wasserstein Barycenters for Time-series Modeling
We propose a dynamical Wasserstein barycentric (DWB) model that estimates the system state over time as well as the data-generating distributions of pure states in an unsupervised manner.
Barycentric-alignment and reconstruction loss minimization for domain generalization
To bridge this gap between theory and practice, we introduce a new upper bound that is free of terms having such dual dependence, resulting in a fully optimizable risk upper bound for the unseen domain.
Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport
In this paper, we extend the recently proposed multivariate rank energy distance, based on the theory of optimal transport, for statistical testing of distributional similarity, to soft rank energy distance.
Multiview Sensing With Unknown Permutations: An Optimal Transport Approach
In several applications, including imaging of deformable objects while in motion, simultaneous localization and mapping, and unlabeled sensing, we encounter the problem of recovering a signal that is measured subject to unknown permutations.
Automatic coding of students' writing via Contrastive Representation Learning in the Wasserstein space
This work is a step towards building a statistical machine learning (ML) method for achieving an automated support for qualitative analyses of students' writing, here specifically in score laboratory reports in introductory biology for sophistication of argumentation and reasoning.
Robust Machine Learning via Privacy/Rate-Distortion Theory
Robust machine learning formulations have emerged to address the prevalent vulnerability of deep neural networks to adversarial examples.
Representation Learning via Adversarially-Contrastive Optimal Transport
To maximize extraction of such informative cues from the data, we set the problem within the context of contrastive representation learning and to that end propose a novel objective via optimal transport.
Domain Adaptation for Robust Workload Level Alignment Between Sessions and Subjects using fNIRS
Results: In a sample of six subjects, G-W resulted in an alignment accuracy of 68 $\pm$ 4 % (weighted mean $\pm$ standard error) for session-by-session alignment, FG-W resulted in an alignment accuracy of 55 $\pm$ 2 % for subject-by-subject alignment.
On Matched Filtering for Statistical Change Point Detection
Non-parametric and distribution-free two-sample tests have been the foundation of many change point detection algorithms.
R-local unlabeled sensing: A novel graph matching approach for multiview unlabeled sensing under local permutations
Unlabeled sensing is a linear inverse problem where the measurements are scrambled under an unknown permutation leading to loss of correspondence between the measurements and the rows of the sensing matrix.
Optimal Transport Based Change Point Detection and Time Series Segment Clustering
Two common problems in time series analysis are the decomposition of the data stream into disjoint segments that are each in some sense "homogeneous" - a problem known as Change Point Detection (CPD) - and the grouping of similar nonadjacent segments, a problem that we call Time Series Segment Clustering (TSSC).
On the modes of convergence of Stochastic Optimistic Mirror Descent (OMD) for saddle point problems
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence only occurs after some sufficiently large number of iterations.
Multi-View Graph Embedding Using Randomized Shortest Paths
This data is well represented by multi-view graphs, which consist of several distinct sets of edges over the same nodes.
Principal Component Analysis with Tensor Train Subspace
Tensor train is a hierarchical tensor network structure that helps alleviate the curse of dimensionality by parameterizing large-scale multidimensional data via a set of network of low-rank tensors.
LEARNING SEMANTIC WORD RESPRESENTATIONS VIA TENSOR FACTORIZATION
We present two new word embedding techniques based on tensor factorization and show that they outperform common methods on several semantic NLP tasks when given the same data.
Tensor Train Neighborhood Preserving Embedding
In this paper, we propose a Tensor Train Neighborhood Preserving Embedding (TTNPE) to embed multi-dimensional tensor data into low dimensional tensor subspace.
Faster Clustering via Non-Backtracking Random Walks
VEC employs a novel application of the state-of-the-art word2vec model to embed a graph in Euclidean space via random walks on the nodes of the graph.
Efficient Low Rank Tensor Ring Completion
Using the matrix product state (MPS) representation of the recently proposed tensor ring decompositions, in this paper we propose a tensor completion algorithm, which is an alternating minimization algorithm that alternates over the factors in the MPS representation.
Sample, computation vs storage tradeoffs for classification using tensor subspace models
Our main tool is the use of tensor subspaces, i. e. subspaces with a Kronecker structure, for embedding the data into lower dimensions.
Word Embeddings via Tensor Factorization
We show that embeddings based on tensor factorization can be used to discern the various meanings of polysemous words without being explicitly trained to do so, and motivate the intuition behind why this works in a way that doesn't with existing methods.
Unsupervised clustering under the Union of Polyhedral Cones (UOPC) model
Similar to the Union of Subspaces (UOS) model where each data from each subspace is generated from a (unknown) basis, in the UOPC model each data from each cone is assumed to be generated from a finite number of (unknown) \emph{extreme rays}. To cluster data under this model, we consider several algorithms - (a) Sparse Subspace Clustering by Non-negative constraints Lasso (NCL), (b) Least squares approximation (LSA), and (c) K-nearest neighbor (KNN) algorithm to arrive at affinity between data points.
Low-tubal-rank Tensor Completion using Alternating Minimization
The low-tubal-rank tensor model has been recently proposed for real-world multidimensional data.
Algorithms for item categorization based on ordinal ranking data
In this context we modify an existing algorithm - namely the label propagation algorithm to a variant that uses the distance between the nodes for weighting the label propagation - to identify the categories.
Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model
Using the matrix product state (MPS) representation of tensor train decompositions, in this paper we propose a tensor completion algorithm which alternates over the matrices (tensors) in the MPS representation.
On Deterministic Conditions for Subspace Clustering under Missing Data
In this paper we present deterministic conditions for success of sparse subspace clustering (SSC) under missing data, when data is assumed to come from a Union of Subspaces (UoS) model.
On deterministic conditions for subspace clustering under missing data
We provide extensive set of simulation results for clustering as well as completion of data under missing entries, under the UoS model.
Denoising and Completion of 3D Data via Multidimensional Dictionary Learning
In this paper a new dictionary learning algorithm for multidimensional data is proposed.
Multilinear Subspace Clustering
In this paper we present a new model and an algorithm for unsupervised clustering of 2-D data such as images.
Group-Invariant Subspace Clustering
In this paper we consider the problem of group invariant subspace clustering where the data is assumed to come from a union of group-invariant subspaces of a vector space, i. e. subspaces which are invariant with respect to action of a given group.
Information-theoretic Bounds on Matrix Completion under Union of Subspaces Model
In this short note we extend some of the recent results on matrix completion under the assumption that the columns of the matrix can be grouped (clustered) into subspaces (not necessarily disjoint or independent).
Adaptive Sampling of RF Fingerprints for Fine-grained Indoor Localization
In contrast to several existing work that rely on random sampling, this paper shows that adaptivity in sampling can lead to significant improvements in localization accuracy.
An algorithm for online tensor prediction
Then following a similar construction as in [3], we exploit this algorithm to propose an online algorithm for learning and prediction of tensors with provable regret guarantees.
Exact tensor completion using t-SVD
Using this factorization one can derive notion of tensor rank, referred to as the tensor tubal rank, which has optimality properties similar to that of matrix rank derived from SVD.
Clustering multi-way data: a novel algebraic approach
In this paper, we develop a method for unsupervised clustering of two-way (matrix) data by combining two recent innovations from different fields: the Sparse Subspace Clustering (SSC) algorithm [10], which groups points coming from a union of subspaces into their respective subspaces, and the t-product [18], which was introduced to provide a matrix-like multiplication for third order tensors.
Novel methods for multilinear data completion and de-noising based on tensor-SVD
Based on t-SVD, the notion of multilinear rank and a related tensor nuclear norm was proposed in [11] to characterize informational and structural complexity of multilinear data.
First Order Methods for Robust Non-negative Matrix Factorization for Large Scale Noisy Data
Nonnegative matrix factorization (NMF) has been shown to be identifiable under the separability assumption, under which all the columns(or rows) of the input data matrix belong to the convex cone generated by only a few of these columns(or rows) [1].
Robust Large Scale Non-negative Matrix Factorization using Proximal Point Algorithm
A robust algorithm for non-negative matrix factorization (NMF) is presented in this paper with the purpose of dealing with large-scale data, where the separability assumption is satisfied.
Novel Factorization Strategies for Higher Order Tensors: Implications for Compression and Recovery of Multi-linear Data
In this paper we propose novel methods for compression and recovery of multilinear data under limited sampling.
