Search Results for author: Mladen Kolar

Found 66 papers, 18 papers with code

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

no code implementations17 Feb 2024 Tim Tsz-Kit Lau, Han Liu, Mladen Kolar

The choice of batch sizes in stochastic gradient optimizers is critical for model training.

Image Classification

Inconsistency of cross-validation for structure learning in Gaussian graphical models

no code implementations28 Dec 2023 Zhao Lyu, Wai Ming Tai, Mladen Kolar, Bryon Aragam

In this paper, we highlight the inherent limitations of cross-validation when employed to discern the structure of a Gaussian graphical model.

Model Selection

Addressing Budget Allocation and Revenue Allocation in Data Market Environments Using an Adaptive Sampling Algorithm

1 code implementation5 Jun 2023 Boxin Zhao, Boxiang Lyu, Raul Castro Fernandez, Mladen Kolar

Data markets help with identifying valuable training data: model consumers pay to train a model, the market uses that budget to identify data and train the model (the budget allocation problem), and finally the market compensates data providers according to their data contribution (revenue allocation problem).

Fraud Detection

Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching

1 code implementation28 May 2023 Ilgee Hong, Sen Na, Michael W. Mahoney, Mladen Kolar

Our method adaptively controls the accuracy of the randomized solver and the penalty parameters of the exact augmented Lagrangian, to ensure that the inexact Newton direction is a descent direction of the exact augmented Lagrangian.

Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems

1 code implementation29 Nov 2022 Yuchen Fang, Sen Na, Michael W. Mahoney, Mladen Kolar

We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints.

Latent Multimodal Functional Graphical Model Estimation

no code implementations31 Oct 2022 Katherine Tsai, Boxin Zhao, Sanmi Koyejo, Mladen Kolar

Joint multimodal functional data acquisition, where functional data from multiple modes are measured simultaneously from the same subject, has emerged as an exciting modern approach enabled by recent engineering breakthroughs in the neurological and biological sciences.

One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning

no code implementations31 May 2022 Pedro Cisneros-Velarde, Boxiang Lyu, Sanmi Koyejo, Mladen Kolar

Although parallelism has been extensively used in reinforcement learning (RL), the quantitative effects of parallel exploration are not well understood theoretically.

Reinforcement Learning (RL)

Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline Reinforcement Learning

no code implementations5 May 2022 Boxiang Lyu, Zhaoran Wang, Mladen Kolar, Zhuoran Yang

In the setting where the function approximation is employed to handle large state spaces, with only mild assumptions on the expressiveness of the function class, we are able to design a dynamic mechanism using offline reinforcement learning algorithms.

Offline RL reinforcement-learning +1

Personalized Federated Learning with Multiple Known Clusters

1 code implementation28 Apr 2022 Boxiang Lyu, Filip Hanzely, Mladen Kolar

We consider the problem of personalized federated learning when there are known cluster structures within users.

Personalized Federated Learning

L-SVRG and L-Katyusha with Adaptive Sampling

no code implementations31 Jan 2022 Boxin Zhao, Boxiang Lyu, Mladen Kolar

Stochastic gradient-based optimization methods, such as L-SVRG and its accelerated variant L-Katyusha (Kovalev et al., 2020), are widely used to train machine learning models. The theoretical and empirical performance of L-SVRG and L-Katyusha can be improved by sampling observations from a non-uniform distribution (Qian et al., 2021).

Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback

1 code implementation28 Dec 2021 Boxin Zhao, Lingxiao Wang, Mladen Kolar, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Chaochao Chen

As a result, client sampling plays an important role in FL systems as it affects the convergence rate of optimization algorithms used to train machine learning models.

Federated Learning Stochastic Optimization

Dynamic Regret Minimization for Control of Non-stationary Linear Dynamical Systems

no code implementations6 Nov 2021 Yuwei Luo, Varun Gupta, Mladen Kolar

Under the assumption that a sequence of stabilizing, but potentially sub-optimal controllers is available for all $t$, we present an algorithm that achieves the optimal dynamic regret of $\tilde{\mathcal{O}}\left(V_T^{2/5}T^{3/5}\right)$.

Joint Gaussian Graphical Model Estimation: A Survey

1 code implementation19 Oct 2021 Katherine Tsai, Oluwasanmi Koyejo, Mladen Kolar

Graphs from complex systems often share a partial underlying structure across domains while retaining individual features.

Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic Programming

1 code implementation23 Sep 2021 Sen Na, Mihai Anitescu, Mladen Kolar

We study nonlinear optimization problems with a stochastic objective and deterministic equality and inequality constraints, which emerge in numerous applications including finance, manufacturing, power systems and, recently, deep neural networks.

Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimization

no code implementations18 Jun 2021 Luofeng Liao, Li Shen, Jia Duan, Mladen Kolar, DaCheng Tao

Large scale convex-concave minimax problems arise in numerous applications, including game theory, robust training, and training of generative adversarial networks.

Generative Adversarial Network Vocal Bursts Type Prediction

Robust Inference for High-Dimensional Linear Models via Residual Randomization

1 code implementation14 Jun 2021 Y. Samuel Wang, Si Kai Lee, Panos Toulis, Mladen Kolar

We propose a residual randomization procedure designed for robust Lasso-based inference in the high-dimensional setting.

valid Vocal Bursts Intensity Prediction

High-dimensional Functional Graphical Model Structure Learning via Neighborhood Selection Approach

1 code implementation6 May 2021 Boxin Zhao, Percy S. Zhai, Y. Samuel Wang, Mladen Kolar

We propose a neighborhood selection approach to estimate the structure of Gaussian functional graphical models, where we first estimate the neighborhood of each node via a function-on-function regression and subsequently recover the entire graph structure by combining the estimated neighborhoods.

Dimensionality Reduction EEG +1

Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning

no code implementations19 Feb 2021 Luofeng Liao, Zuyue Fu, Zhuoran Yang, Yixin Wang, Mladen Kolar, Zhaoran Wang

Instrumental variables (IVs), in the context of RL, are the variables whose influence on the state variables are all mediated through the action.

Offline RL reinforcement-learning +2

An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians

1 code implementation10 Feb 2021 Sen Na, Mihai Anitescu, Mladen Kolar

Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified.

Provably Training Overparameterized Neural Network Classifiers with Non-convex Constraints

no code implementations30 Dec 2020 You-Lin Chen, Zhaoran Wang, Mladen Kolar

Training a classifier under non-convex constraints has gotten increasing attention in the machine learning community thanks to its wide range of applications such as algorithmic fairness and class-imbalanced classification.

Fairness imbalanced classification

Provably Efficient Neural Estimation of Structural Equation Models: An Adversarial Approach

no code implementations NeurIPS 2020 Luofeng Liao, You-Lin Chen, Zhuoran Yang, Bo Dai, Mladen Kolar, Zhaoran Wang

We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation.

A Nonconvex Framework for Structured Dynamic Covariance Recovery

no code implementations11 Nov 2020 Katherine Tsai, Mladen Kolar, Oluwasanmi Koyejo

We prove a linear convergence rate up to a nontrivial statistical error for the proposed descent scheme and establish sample complexity guarantees for the estimator.

Statistical Inference for Networks of High-Dimensional Point Processes

1 code implementation15 Jul 2020 Xu Wang, Mladen Kolar, Ali Shojaie

The key ingredient for this inference procedure is a new concentration inequality on the first- and second-order statistics for integrated stochastic processes, which summarize the entire history of the process.

Point Processes Vocal Bursts Intensity Prediction

Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach

no code implementations2 Jul 2020 Luofeng Liao, You-Lin Chen, Zhuoran Yang, Bo Dai, Zhaoran Wang, Mladen Kolar

We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation.

Gradient-Variation Bound for Online Convex Optimization with Constraints

no code implementations22 Jun 2020 Shuang Qiu, Xiaohan Wei, Mladen Kolar

We study online convex optimization with constraints consisting of multiple functional constraints and a relatively simple constraint set, such as a Euclidean ball.

FuDGE: A Method to Estimate a Functional Differential Graph in a High-Dimensional Setting

no code implementations11 Mar 2020 Boxin Zhao, Y. Samuel Wang, Mladen Kolar

We first define a functional differential graph that captures the differences between two functional graphical models and formally characterize when the functional differential graph is well defined.


Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees

no code implementations ICML 2020 Sen Na, Yuwei Luo, Zhuoran Yang, Zhaoran Wang, Mladen Kolar

We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution.

Graph Representation Learning

Posterior Ratio Estimation of Latent Variables

no code implementations15 Feb 2020 Song Liu, Yulong Zhang, Mingxuan Yi, Mladen Kolar

Density Ratio Estimation has attracted attention from the machine learning community due to its ability to compare the underlying distributions of two datasets.

Density Ratio Estimation

Convergent Policy Optimization for Safe Reinforcement Learning

1 code implementation NeurIPS 2019 Ming Yu, Zhuoran Yang, Mladen Kolar, Zhaoran Wang

We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions.

Multi-agent Reinforcement Learning reinforcement-learning +2

Direct Estimation of Differential Functional Graphical Models

1 code implementation NeurIPS 2019 Boxin Zhao, Y. Samuel Wang, Mladen Kolar

We consider the problem of estimating the difference between two functional undirected graphical models with shared structures.


Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

1 code implementation12 Sep 2019 Sen Na, Mladen Kolar, Oluwasanmi Koyejo

Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation.

Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees

1 code implementation12 Jun 2019 You-Lin Chen, Mladen Kolar, Ruey S. Tsay

In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of an ad-hoc way of transforming data to vectors due to the computational and under-sampling issues.

Tensor Decomposition

Partially Linear Additive Gaussian Graphical Models

no code implementations8 Jun 2019 Sinong Geng, Minhao Yan, Mladen Kolar, Oluwasanmi Koyejo

We propose a partially linear additive Gaussian graphical model (PLA-GGM) for the estimation of associations between random variables distorted by observed confounders.

High-dimensional Index Volatility Models via Stein's Identity

no code implementations27 Nov 2018 Sen Na, Mladen Kolar

We study the estimation of the parametric components of single and multiple index volatility models.

Vocal Bursts Intensity Prediction

Provable Gaussian Embedding with One Observation

no code implementations NeurIPS 2018 Ming Yu, Zhuoran Yang, Tuo Zhao, Mladen Kolar, Zhaoran Wang

In this paper, we study the Gaussian embedding model and develop the first theoretical results for exponential family embedding models.

BIG-bench Machine Learning

High-dimensional Varying Index Coefficient Models via Stein's Identity

1 code implementation16 Oct 2018 Sen Na, Zhuoran Yang, Zhaoran Wang, Mladen Kolar

We study the parameter estimation problem for a varying index coefficient model in high dimensions.

Vocal Bursts Intensity Prediction

Joint Nonparametric Precision Matrix Estimation with Confounding

no code implementations16 Oct 2018 Sinong Geng, Mladen Kolar, Oluwasanmi Koyejo

Empirical results are presented using simulated and real brain imaging data, which suggest that our approach improves precision matrix estimation, as compared to baselines, when confounding is present.

Learning Influence-Receptivity Network Structure with Guarantee

no code implementations14 Jun 2018 Ming Yu, Varun Gupta, Mladen Kolar

Specifically, we endow each node with two node-topic vectors: an influence vector that measures how influential/authoritative they are on each topic; and a receptivity vector that measures how receptive/susceptible they are to each topic.

Community Detection

Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach

no code implementations20 Feb 2018 Ming Yu, Varun Gupta, Mladen Kolar

We show linear convergence of the iterates obtained by GDT to a region within statistical error of an optimal solution.

Multi-Task Learning

Distributed Stochastic Multi-Task Learning with Graph Regularization

no code implementations11 Feb 2018 Weiran Wang, Jialei Wang, Mladen Kolar, Nathan Srebro

We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines.

Multi-Task Learning

The Expxorcist: Nonparametric Graphical Models Via Conditional Exponential Densities

no code implementations NeurIPS 2017 Arun Suggala, Mladen Kolar, Pradeep K. Ravikumar

Non-parametric multivariate density estimation faces strong statistical and computational bottlenecks, and the more practical approaches impose near-parametric assumptions on the form of the density functions.

Density Estimation

Scalable Peaceman-Rachford Splitting Method with Proximal Terms

no code implementations14 Nov 2017 Sen Na, Mingyuan Ma, Mladen Kolar

Along with developing of Peaceman-Rachford Splittling Method (PRSM), many batch algorithms based on it have been studied very deeply.

Estimation of a Low-rank Topic-Based Model for Information Cascades

1 code implementation6 Sep 2017 Ming Yu, Varun Gupta, Mladen Kolar

We consider the problem of estimating the latent structure of a social network based on the observed information diffusion events, or cascades, where the observations for a given cascade consist of only the timestamps of infection for infected nodes but not the source of the infection.

Recommendation Systems

Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores

no code implementations20 Feb 2017 Jelena Bradic, Mladen Kolar

The main technical result are the development of a Bahadur representation of the debiasing estimator that is uniform over a range of quantiles and uniform convergence of the quantile process to the Brownian bridge process, which are of independent interest.

regression valid

Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data

no code implementations10 Oct 2016 Jialei Wang, Jason D. Lee, Mehrdad Mahdavi, Mladen Kolar, Nathan Srebro

Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data.

Efficient Distributed Learning with Sparsity

no code implementations ICML 2017 Jialei Wang, Mladen Kolar, Nathan Srebro, Tong Zhang

We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines.

General Classification regression +1

Distributed Multi-Task Learning with Shared Representation

no code implementations7 Mar 2016 Jialei Wang, Mladen Kolar, Nathan Srebro

We study the problem of distributed multi-task learning with shared representation, where each machine aims to learn a separate, but related, task in an unknown shared low-dimensional subspaces, i. e. when the predictor matrix has low rank.

Multi-Task Learning

Post-Regularization Inference for Time-Varying Nonparanormal Graphical Models

no code implementations28 Dec 2015 Junwei Lu, Mladen Kolar, Han Liu

The testing procedures are based on a high dimensional, debiasing-free moment estimator, which uses a novel kernel smoothed Kendall's tau correlation matrix as an input statistic.

Model Selection

Learning structured densities via infinite dimensional exponential families

no code implementations NeurIPS 2015 Siqi Sun, Mladen Kolar, Jinbo Xu

Learning the structure of a probabilistic graphical models is a well studied problem in the machine learning community due to its importance in many applications.

Distributed Multitask Learning

no code implementations2 Oct 2015 Jialei Wang, Mladen Kolar, Nathan Srebro

We present a communication-efficient estimator based on the debiased lasso and show that it is comparable with the optimal centralized method.

Multi-Task Learning

Kernel Meets Sieve: Post-Regularization Confidence Bands for Sparse Additive Model

no code implementations10 Mar 2015 Junwei Lu, Mladen Kolar, Han Liu

We develop a novel procedure for constructing confidence bands for components of a sparse additive model.

Additive models

ROCKET: Robust Confidence Intervals via Kendall's Tau for Transelliptical Graphical Models

no code implementations26 Feb 2015 Rina Foygel Barber, Mladen Kolar

Undirected graphical models are used extensively in the biological and social sciences to encode a pattern of conditional independences between variables, where the absence of an edge between two nodes $a$ and $b$ indicates that the corresponding two variables $X_a$ and $X_b$ are believed to be conditionally independent, after controlling for all other measured variables.

A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression

no code implementations30 Dec 2014 Tianqi Zhao, Mladen Kolar, Han Liu

Our de-biasing procedure does not require solving the $L_1$-penalized composite quantile regression.


Inference for Sparse Conditional Precision Matrices

no code implementations24 Dec 2014 Jialei Wang, Mladen Kolar

observations of a random vector $(X, Z)$, where $X$ is a high-dimensional vector and $Z$ is a low-dimensional index variable, we study the problem of estimating the conditional inverse covariance matrix $\Omega(z) = (E[(X-E[X \mid Z])(X-E[X \mid Z])^T \mid Z=z])^{-1}$ under the assumption that the set of non-zero elements is small and does not depend on the index variable.

Optimal variable selection in multi-group sparse discriminant analysis

no code implementations23 Nov 2014 Irina Gaynanova, Mladen Kolar

This article considers the problem of multi-group classification in the setting where the number of variables $p$ is larger than the number of observations $n$.

Variable Selection

Estimating Undirected Graphs Under Weak Assumptions

no code implementations26 Sep 2013 Larry Wasserman, Mladen Kolar, Alessandro Rinaldo

In particular, we consider: cluster graphs, restricted partial correlation graphs and correlation graphs.


Optimal Feature Selection in High-Dimensional Discriminant Analysis

no code implementations27 Jun 2013 Mladen Kolar, Han Liu

Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n, dimensionality p, and sparsity level s in the high-dimensional setting.

feature selection Variable Selection +1

Graph Estimation From Multi-attribute Data

no code implementations29 Oct 2012 Mladen Kolar, Han Liu, Eric P. Xing

Many real world network problems often concern multivariate nodal attributes such as image, textual, and multi-view feature vectors on nodes, rather than simple univariate nodal attributes.


Recovering Block-structured Activations Using Compressive Measurements

no code implementations15 Sep 2012 Sivaraman Balakrishnan, Mladen Kolar, Alessandro Rinaldo, Aarti Singh

We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements.

Minimax Localization of Structural Information in Large Noisy Matrices

no code implementations NeurIPS 2011 Mladen Kolar, Sivaraman Balakrishnan, Alessandro Rinaldo, Aarti Singh

We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries.

Clustering Two-sample testing

Time-Varying Dynamic Bayesian Networks

no code implementations NeurIPS 2009 Le Song, Mladen Kolar, Eric P. Xing

In this paper, we propose a time-varying dynamic Bayesian network (TV-DBN) for modeling the structurally varying directed dependency structures underlying non-stationary biological/neural time series.

Computational Efficiency Time Series +1

Sparsistent Estimation of Time-Varying Discrete Markov Random Fields

no code implementations14 Jul 2009 Mladen Kolar, Eric P. Xing

Network models have been popular for modeling and representing complex relationships and dependencies between observed variables.

regression Time Series +1

Cannot find the paper you are looking for? You can Submit a new open access paper.