Search Results for author:
Found 40 papers, 15 papers with code
Online Identification and Tracking of Subspaces from Highly Incomplete Information
GROUSE performs exceptionally well in practice both in tracking subspaces and as an online algorithm for matrix completion.
Online Robust Subspace Tracking from Partial Information
This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an efficient and robust online algorithm for tracking subspaces from highly incomplete information.
LEARNING TO SHARE: SIMULTANEOUS PARAMETER TYING AND SPARSIFICATION IN DEEP LEARNING
This has motivated a large body of work to reduce the complexity of the neural network by using sparsity-inducing regularizers.
Online matrix factorization for Markovian data and applications to Network Dictionary Learning
As the main application, by combining online non-negative matrix factorization and a recent MCMC algorithm for sampling motifs from networks, we propose a novel framework of Network Dictionary Learning, which extracts ``network dictionary patches' from a given network in an online manner that encodes main features of the network.
Grassmannian Optimization for Online Tensor Completion and Tracking with the t-SVD
We propose a new fast streaming algorithm for the tensor completion problem of imputing missing entries of a low-tubal-rank tensor using the tensor singular value decomposition (t-SVD) algebraic framework.
Deep Unsupervised Clustering Using Mixture of Autoencoders
Unsupervised clustering is one of the most fundamental challenges in machine learning.
The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks
Second, it allows us to better understand deep representation learning by elucidating the linear progressive separation and concentration of representations from shallow to deep layers.
Algebraic Variety Models for High-Rank Matrix Completion
We consider a generalization of low-rank matrix completion to the case where the data belongs to an algebraic variety, i. e. each data point is a solution to a system of polynomial equations.
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods
This paper introduces \textit{online bilevel optimization} in which a sequence of time-varying bilevel problems is revealed one after the other.
Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics
We empirically evaluate the effectiveness of our compression technique on matrix recovery problems.
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold
When training overparameterized deep networks for classification tasks, it has been widely observed that the learned features exhibit a so-called "neural collapse" phenomenon.
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
To the best of our knowledge, this is the first quantitative characterization of feature evolution in hierarchical representations of deep linear networks.
Preference Modeling with Context-Dependent Salient Features
Finally we demonstrate strong performance of maximum likelihood estimation of our model on both synthetic data and two real data sets: the UT Zappos50K data set and comparison data about the compactness of legislative districts in the US.
Convergence and Recovery Guarantees of the K-Subspaces Method for Subspace Clustering
The K-subspaces (KSS) method is a generalization of the K-means method for subspace clustering.
Streaming PCA and Subspace Tracking: The Missing Data Case
This survey article reviews a variety of classical and recent algorithms for solving this problem with low computational and memory complexities, particularly those applicable in the big data regime with missing data.
Real-Time Energy Disaggregation of a Distribution Feeder's Demand Using Online Learning
We use an online learning algorithm, Dynamic Fixed Share (DFS), that uses the real-time distribution feeder measurements as well as models generated from historical building- and device-level data.
Tensor Methods for Nonlinear Matrix Completion
This approach will succeed in many cases where traditional LRMC is guaranteed to fail because the data are low-rank in the tensorized representation but not in the original representation.
Subspace Clustering using Ensembles of $K$-Subspaces
We present a novel geometric approach to the subspace clustering problem that leverages ensembles of the K-subspaces (KSS) algorithm via the evidence accumulation clustering framework.
Leveraging Union of Subspace Structure to Improve Constrained Clustering
We demonstrate on several datasets that our algorithm drives the clustering error down considerably faster than the state-of-the-art active query algorithms on datasets with subspace structure and is competitive on other datasets.
Distance-Penalized Active Learning Using Quantile Search
Adaptive sampling theory has shown that, with proper assumptions on the signal class, algorithms exist to reconstruct a signal in $\mathbb{R}^{d}$ with an optimal number of samples.
On Learning High Dimensional Structured Single Index Models
Single Index Models (SIMs) are simple yet flexible semi-parametric models for machine learning, where the response variable is modeled as a monotonic function of a linear combination of features.
Towards a Theoretical Analysis of PCA for Heteroscedastic Data
Principal Component Analysis (PCA) is a method for estimating a subspace given noisy samples.
Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation From Undersampled Data
We study two sampling cases: where each data vector of the streaming matrix is fully sampled, or where it is undersampled by a sampling matrix $A_t\in \mathbb{R}^{m\times n}$ with $m\ll n$.
Online Algorithms for Factorization-Based Structure from Motion
We present a family of online algorithms for real-time factorization-based structure from motion, leveraging a relationship between incremental singular value decomposition and recently proposed methods for online matrix completion.
Global Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation
It has been observed in a variety of contexts that gradient descent methods have great success in solving low-rank matrix factorization problems, despite the relevant problem formulation being non-convex.
Matrix Completion Under Monotonic Single Index Models
Most recent results in matrix completion assume that the matrix under consideration is low-rank or that the columns are in a union of low-rank subspaces.
On GROUSE and Incremental SVD
GROUSE (Grassmannian Rank-One Update Subspace Estimation) is an incremental algorithm for identifying a subspace of Rn from a sequence of vectors in this subspace, where only a subset of components of each vector is revealed at each iteration.
Iterative Grassmannian Optimization for Robust Image Alignment
t-GRASTA iteratively performs incremental gradient descent constrained to the Grassmann manifold of subspaces in order to simultaneously estimate a decomposition of a collection of images into a low-rank subspace, a sparse part of occlusions and foreground objects, and a transformation such as rotation or translation of the image.
INFORMATION MAXIMIZATION AUTO-ENCODING
Unlike the Variational Autoencoder framework, IMAE starts from a stochastic encoder that seeks to map each input data to a hybrid discrete and continuous representation with the objective of maximizing the mutual information between the data and their representations.
Preference modelling with context-dependent salient features
Finally we demonstrate the strong performance of maximum likelihood estimation of our model on both synthetic data and two real data sets: the UT Zappos50K data set and comparison data about the compactness of legislative districts in the United States.
Supervised PCA: A Multiobjective Approach
Methods for supervised principal component analysis (SPCA) aim to incorporate label information into principal component analysis (PCA), so that the extracted features are more useful for a prediction task of interest.
Certainty Equivalent Quadratic Control for Markov Jump Systems
Real-world control applications often involve complex dynamics subject to abrupt changes or variations.
Identification and Adaptive Control of Markov Jump Systems: Sample Complexity and Regret Bounds
Combining our sample complexity results with recent perturbation results for certainty equivalent control, we prove that when the episode lengths are appropriately chosen, the proposed adaptive control scheme achieves $\mathcal{O}(\sqrt{T})$ regret, which can be improved to $\mathcal{O}(polylog(T))$ with partial knowledge of the system.
Fair Community Detection and Structure Learning in Heterogeneous Graphical Models
In particular, we assume there is some community or clustering structure in the true underlying graph, and we seek to learn a sparse undirected graph and its communities from the data such that demographic groups are fairly represented within the communities.
Mode Reduction for Markov Jump Systems
Switched systems are capable of modeling processes with underlying dynamics that may change abruptly over time.
HeMPPCAT: Mixtures of Probabilistic Principal Component Analysers for Data with Heteroscedastic Noise
Mixtures of probabilistic principal component analysis (MPPCA) is a well-known mixture model extension of principal component analysis (PCA).
Dynamic Subspace Estimation with Grassmannian Geodesics
We propose a novel algorithm for minimizing this objective and estimating the parameters of the model from data with Grassmannian-constrained optimization.
ALPCAH: Sample-wise Heteroscedastic PCA with Tail Singular Value Regularization
Other methods such as Weighted PCA (WPCA) assume the noise variances are known, which may be difficult to know in practice.
Streaming Probabilistic PCA for Missing Data with Heteroscedastic Noise
Streaming principal component analysis (PCA) is an integral tool in large-scale machine learning for rapidly estimating low-dimensional subspaces of very high dimensional and high arrival-rate data with missing entries and corrupting noise.
Convergence and complexity of block majorization-minimization for constrained block-Riemannian optimization
Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex optimization that sequentially minimizes a majorizing surrogate of the objective function in each block coordinate while the other block coordinates are held fixed.
