Search Results for author:
Found 66 papers, 18 papers with code
AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
The choice of batch sizes in stochastic gradient optimizers is critical for model training.
Inconsistency of cross-validation for structure learning in Gaussian graphical models
In this paper, we highlight the inherent limitations of cross-validation when employed to discern the structure of a Gaussian graphical model.
Addressing Budget Allocation and Revenue Allocation in Data Market Environments Using an Adaptive Sampling Algorithm
Data markets help with identifying valuable training data: model consumers pay to train a model, the market uses that budget to identify data and train the model (the budget allocation problem), and finally the market compensates data providers according to their data contribution (revenue allocation problem).
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching
Our method adaptively controls the accuracy of the randomized solver and the penalty parameters of the exact augmented Lagrangian, to ensure that the inexact Newton direction is a descent direction of the exact augmented Lagrangian.
Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems
We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints.
Latent Multimodal Functional Graphical Model Estimation
Joint multimodal functional data acquisition, where functional data from multiple modes are measured simultaneously from the same subject, has emerged as an exciting modern approach enabled by recent engineering breakthroughs in the neurological and biological sciences.
One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning
Although parallelism has been extensively used in reinforcement learning (RL), the quantitative effects of parallel exploration are not well understood theoretically.
Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline Reinforcement Learning
In the setting where the function approximation is employed to handle large state spaces, with only mild assumptions on the expressiveness of the function class, we are able to design a dynamic mechanism using offline reinforcement learning algorithms.
Personalized Federated Learning with Multiple Known Clusters
We consider the problem of personalized federated learning when there are known cluster structures within users.
L-SVRG and L-Katyusha with Adaptive Sampling
Stochastic gradient-based optimization methods, such as L-SVRG and its accelerated variant L-Katyusha (Kovalev et al., 2020), are widely used to train machine learning models. The theoretical and empirical performance of L-SVRG and L-Katyusha can be improved by sampling observations from a non-uniform distribution (Qian et al., 2021).
Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback
As a result, client sampling plays an important role in FL systems as it affects the convergence rate of optimization algorithms used to train machine learning models.
Dynamic Regret Minimization for Control of Non-stationary Linear Dynamical Systems
Under the assumption that a sequence of stabilizing, but potentially sub-optimal controllers is available for all $t$, we present an algorithm that achieves the optimal dynamic regret of $\tilde{\mathcal{O}}\left(V_T^{2/5}T^{3/5}\right)$.
Joint Gaussian Graphical Model Estimation: A Survey
Graphs from complex systems often share a partial underlying structure across domains while retaining individual features.
Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic Programming
We study nonlinear optimization problems with a stochastic objective and deterministic equality and inequality constraints, which emerge in numerous applications including finance, manufacturing, power systems and, recently, deep neural networks.
Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimization
Large scale convex-concave minimax problems arise in numerous applications, including game theory, robust training, and training of generative adversarial networks.
Robust Inference for High-Dimensional Linear Models via Residual Randomization
We propose a residual randomization procedure designed for robust Lasso-based inference in the high-dimensional setting.
High-dimensional Functional Graphical Model Structure Learning via Neighborhood Selection Approach
We propose a neighborhood selection approach to estimate the structure of Gaussian functional graphical models, where we first estimate the neighborhood of each node via a function-on-function regression and subsequently recover the entire graph structure by combining the estimated neighborhoods.
Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
Instrumental variables (IVs), in the context of RL, are the variables whose influence on the state variables are all mediated through the action.
Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
We investigate the optimization aspects of personalized Federated Learning (FL).
An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians
Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified.
Provably Training Overparameterized Neural Network Classifiers with Non-convex Constraints
Training a classifier under non-convex constraints has gotten increasing attention in the machine learning community thanks to its wide range of applications such as algorithmic fairness and class-imbalanced classification.
Provably Efficient Neural Estimation of Structural Equation Models: An Adversarial Approach
We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation.
A Nonconvex Framework for Structured Dynamic Covariance Recovery
We prove a linear convergence rate up to a nontrivial statistical error for the proposed descent scheme and establish sample complexity guarantees for the estimator.
Statistical Inference for Networks of High-Dimensional Point Processes
The key ingredient for this inference procedure is a new concentration inequality on the first- and second-order statistics for integrated stochastic processes, which summarize the entire history of the process.
Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach
We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation.
Gradient-Variation Bound for Online Convex Optimization with Constraints
We study online convex optimization with constraints consisting of multiple functional constraints and a relatively simple constraint set, such as a Euclidean ball.
FuDGE: A Method to Estimate a Functional Differential Graph in a High-Dimensional Setting
We first define a functional differential graph that captures the differences between two functional graphical models and formally characterize when the functional differential graph is well defined.
Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees
We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution.
Posterior Ratio Estimation of Latent Variables
Density Ratio Estimation has attracted attention from the machine learning community due to its ability to compare the underlying distributions of two datasets.
Natural Actor-Critic Converges Globally for Hierarchical Linear Quadratic Regulator
Multi-agent reinforcement learning has been successfully applied to a number of challenging problems.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Convergent Policy Optimization for Safe Reinforcement Learning
We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Direct Estimation of Differential Functional Graphical Models
We consider the problem of estimating the difference between two functional undirected graphical models with shared structures.
Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity
Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation.
Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees
In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of an ad-hoc way of transforming data to vectors due to the computational and under-sampling issues.
Partially Linear Additive Gaussian Graphical Models
We propose a partially linear additive Gaussian graphical model (PLA-GGM) for the estimation of associations between random variables distorted by observed confounders.
High-dimensional Index Volatility Models via Stein's Identity
We study the estimation of the parametric components of single and multiple index volatility models.
Provable Gaussian Embedding with One Observation
In this paper, we study the Gaussian embedding model and develop the first theoretical results for exponential family embedding models.
Joint Nonparametric Precision Matrix Estimation with Confounding
Empirical results are presented using simulated and real brain imaging data, which suggest that our approach improves precision matrix estimation, as compared to baselines, when confounding is present.
High-dimensional Varying Index Coefficient Models via Stein's Identity
We study the parameter estimation problem for a varying index coefficient model in high dimensions.
Learning Influence-Receptivity Network Structure with Guarantee
Specifically, we endow each node with two node-topic vectors: an influence vector that measures how influential/authoritative they are on each topic; and a receptivity vector that measures how receptive/susceptible they are to each topic.
Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach
We show linear convergence of the iterates obtained by GDT to a region within statistical error of an optimal solution.
Distributed Stochastic Multi-Task Learning with Graph Regularization
We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines.
The Expxorcist: Nonparametric Graphical Models Via Conditional Exponential Densities
Non-parametric multivariate density estimation faces strong statistical and computational bottlenecks, and the more practical approaches impose near-parametric assumptions on the form of the density functions.
Scalable Peaceman-Rachford Splitting Method with Proximal Terms
Along with developing of Peaceman-Rachford Splittling Method (PRSM), many batch algorithms based on it have been studied very deeply.
Estimation of a Low-rank Topic-Based Model for Information Cascades
We consider the problem of estimating the latent structure of a social network based on the observed information diffusion events, or cascades, where the observations for a given cascade consist of only the timestamps of infection for infected nodes but not the source of the infection.
Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores
The main technical result are the development of a Bahadur representation of the debiasing estimator that is uniform over a range of quantiles and uniform convergence of the quantile process to the Brownian bridge process, which are of independent interest.
Statistical Inference for Pairwise Graphical Models Using Score Matching
As a result, there is a large body of literature focused on consistent model selection.
Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data
Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data.
Efficient Distributed Learning with Sparsity
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines.
Distributed Multi-Task Learning with Shared Representation
We study the problem of distributed multi-task learning with shared representation, where each machine aims to learn a separate, but related, task in an unknown shared low-dimensional subspaces, i. e. when the predictor matrix has low rank.
Post-Regularization Inference for Time-Varying Nonparanormal Graphical Models
The testing procedures are based on a high dimensional, debiasing-free moment estimator, which uses a novel kernel smoothed Kendall's tau correlation matrix as an input statistic.
Learning structured densities via infinite dimensional exponential families
Learning the structure of a probabilistic graphical models is a well studied problem in the machine learning community due to its importance in many applications.
Distributed Multitask Learning
We present a communication-efficient estimator based on the debiased lasso and show that it is comparable with the optimal centralized method.
Kernel Meets Sieve: Post-Regularization Confidence Bands for Sparse Additive Model
We develop a novel procedure for constructing confidence bands for components of a sparse additive model.
ROCKET: Robust Confidence Intervals via Kendall's Tau for Transelliptical Graphical Models
Undirected graphical models are used extensively in the biological and social sciences to encode a pattern of conditional independences between variables, where the absence of an edge between two nodes $a$ and $b$ indicates that the corresponding two variables $X_a$ and $X_b$ are believed to be conditionally independent, after controlling for all other measured variables.
A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
Our de-biasing procedure does not require solving the $L_1$-penalized composite quantile regression.
Inference for Sparse Conditional Precision Matrices
observations of a random vector $(X, Z)$, where $X$ is a high-dimensional vector and $Z$ is a low-dimensional index variable, we study the problem of estimating the conditional inverse covariance matrix $\Omega(z) = (E[(X-E[X \mid Z])(X-E[X \mid Z])^T \mid Z=z])^{-1}$ under the assumption that the set of non-zero elements is small and does not depend on the index variable.
Optimal variable selection in multi-group sparse discriminant analysis
This article considers the problem of multi-group classification in the setting where the number of variables $p$ is larger than the number of observations $n$.
Estimating Undirected Graphs Under Weak Assumptions
In particular, we consider: cluster graphs, restricted partial correlation graphs and correlation graphs.
Optimal Feature Selection in High-Dimensional Discriminant Analysis
Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n, dimensionality p, and sparsity level s in the high-dimensional setting.
Graph Estimation From Multi-attribute Data
Many real world network problems often concern multivariate nodal attributes such as image, textual, and multi-view feature vectors on nodes, rather than simple univariate nodal attributes.
Recovering Block-structured Activations Using Compressive Measurements
We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements.
Minimax Localization of Structural Information in Large Noisy Matrices
We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries.
Sparsistent Learning of Varying-coefficient Models with Structural Changes
In this paper, we investigate sparsistent learning of a sub-family of this model --- piecewise constant VCVS models.
Time-Varying Dynamic Bayesian Networks
In this paper, we propose a time-varying dynamic Bayesian network (TV-DBN) for modeling the structurally varying directed dependency structures underlying non-stationary biological/neural time series.
Sparsistent Estimation of Time-Varying Discrete Markov Random Fields
Network models have been popular for modeling and representing complex relationships and dependencies between observed variables.
