Search Results for author: Arindam Banerjee

Found 52 papers, 5 papers with code

Stability Based Generalization Bounds for Exponential Family Langevin Dynamics

no code implementations9 Jan 2022 Arindam Banerjee, Tiancong Chen, Xinyan Li, Yingxue Zhou

We study generalization bounds for noisy stochastic mini-batch iterative algorithms based on the notion of stability.

Generalization Bounds

TorchGeo: deep learning with geospatial data

1 code implementation17 Nov 2021 Adam J. Stewart, Caleb Robinson, Isaac A. Corley, Anthony Ortiz, Juan M. Lavista Ferres, Arindam Banerjee

Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available.

Transfer Learning

EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits

no code implementations ICLR 2022 Yikun Ban, Yuchen Yan, Arindam Banerjee, Jingrui He

To overcome this challenge, a series of neural bandit algorithms have been proposed, where a neural network is used to learn the underlying reward function and TS or UCB are adapted for exploration.

Multi-Armed Bandits

Learning and Dynamical Models for Sub-seasonal Climate Forecasting: Comparison and Collaboration

1 code implementation29 Sep 2021 Sijie He, Xinyan Li, Laurie Trenary, Benjamin A Cash, Timothy DelSole, Arindam Banerjee

The SSF dataset constructed for the work, dynamical model predictions, and code for the ML models are released along with the paper for the benefit of the broader machine learning community.

Weather Forecasting

Noisy Truncated SGD: Optimization and Generalization

no code implementations26 Feb 2021 Yingxue Zhou, Xinyan Li, Arindam Banerjee

Our experiments on a variety of benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100) with various networks (VGG and ResNet) validate the theoretical properties of NT-SGD, i. e., NT-SGD matches the speed and accuracy of vanilla SGD while effectively working with sparse gradients, and can successfully escape poor local minima.

Experiments with Rich Regime Training for Deep Learning

no code implementations26 Feb 2021 Xinyan Li, Arindam Banerjee

Inspired by this, we investigate probabilistic LWS-SGD, which mostly updates the top layers and occasionally updates the full network.

Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds

no code implementations24 Jun 2020 Yingxue Zhou, Xiangyi Chen, Mingyi Hong, Zhiwei Steven Wu, Arindam Banerjee

We obtain this rate by providing the first analyses on a collection of private gradient-based methods, including adaptive algorithms DP RMSProp and DP Adam.

Generalization Bounds

Sub-Seasonal Climate Forecasting via Machine Learning: Challenges, Analysis, and Advances

no code implementations14 Jun 2020 Sijie He, Xinyan Li, Timothy DelSole, Pradeep Ravikumar, Arindam Banerjee

Sub-seasonal climate forecasting (SSF) focuses on predicting key climate variables such as temperature and precipitation in the 2-week to 2-month time scales.

Feature Importance

Gradient Boosted Normalizing Flows

1 code implementation NeurIPS 2020 Robert Giaquinto, Arindam Banerjee

We propose an alternative: Gradient Boosted Normalizing Flows (GBNF) model a density by successively adding new NF components with gradient boosting.

Density Estimation Variational Inference

De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors

no code implementations23 Feb 2020 Arindam Banerjee, Tiancong Chen, Yingxue Zhou

Existing approaches for deterministic non-smooth deep nets typically need to bound the Lipschitz constant of such deep nets but such bounds are quite large, may even increase with the training set size yielding vacuous generalization bounds.

Generalization Bounds

Random Quadratic Forms with Dependence: Applications to Restricted Isometry and Beyond

no code implementations NeurIPS 2019 Arindam Banerjee, Qilong Gu, Vidyashankar Sivakumar, Zhiwei Steven Wu

We also discuss stochastic process based forms of J-L, RIP, and sketching, to illustrate the generality of the results.

Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization

no code implementations24 Jul 2019 Xinyan Li, Qilong Gu, Yingxue Zhou, Tiancong Chen, Arindam Banerjee

(2) how can we characterize the stochastic optimization dynamics of SGD with fixed and adaptive step sizes and diagonal pre-conditioning based on the first and second moments of SGs?

Stochastic Optimization

Two-block vs. Multi-block ADMM: An empirical evaluation of convergence

no code implementations10 Jul 2019 Andre Goncalves, Xiaoli Liu, Arindam Banerjee

Alternating Direction Method of Multipliers (ADMM) has become a widely used optimization method for convex problems, particularly in the context of data mining in which large optimization problems are often encountered.

Multi-Task Learning

An Improved Analysis of Alternating Minimization for Structured Multi-Response Regression

no code implementations NeurIPS 2018 Sheng Chen, Arindam Banerjee

To find the coefficient vector, estimators with a joint approximation of the noise covariance are often preferred than the simple linear regression in view of their superior empirical performance, which can be generally solved by alternating-minimization type procedures.

DAPPER: Scaling Dynamic Author Persona Topic Model to Billion Word Corpora

1 code implementation3 Nov 2018 Robert Giaquinto, Arindam Banerjee

Extracting common narratives from multi-author dynamic text corpora requires complex models, such as the Dynamic Author Persona (DAP) topic model.

Variational Inference

Adversarial Recommendation: Attack of the Learned Fake Users

no code implementations21 Sep 2018 Konstantina Christakopoulou, Arindam Banerjee

We propose a framework for generating fake user profiles which, when incorporated in the training of a recommendation system, can achieve an adversarial intent, while remaining indistinguishable from real user profiles.

Recommendation Systems

Time Series Deinterleaving of DNS Traffic

no code implementations16 Jul 2018 Amir Asiaee, Hardik Goel, Shalini Ghosh, Vinod Yegneswaran, Arindam Banerjee

Stream deinterleaving is an important problem with various applications in the cybersecurity domain.

Time Series

High Dimensional Data Enrichment: Interpretable, Fast, and Data-Efficient

no code implementations11 Jun 2018 Amir Asiaee, Samet Oymak, Kevin R. Coombes, Arindam Banerjee

High dimensional structured data enriched model describes groups of observations by shared and per-group individual parameters, each with its own structure such as sparsity or group sparsity.

Sparse Linear Isotonic Models

no code implementations16 Oct 2017 Sheng Chen, Arindam Banerjee

In machine learning and data mining, linear models have been widely used to model the response as parametric linear functions of the predictors.

Additive models

High-Dimensional Dependency Structure Learning for Physical Processes

no code implementations12 Sep 2017 Jamal Golmohammadi, Imme Ebert-Uphoff, Sijie He, Yi Deng, Arindam Banerjee

We compare ACLIME-ADMM with baselines on both synthetic data simulated by partial differential equations (PDEs) that model advection-diffusion processes, and real data (50 years) of daily global geopotential heights to study information flow in the atmosphere.

R2N2: Residual Recurrent Neural Networks for Multivariate Time Series Forecasting

no code implementations10 Sep 2017 Hardik Goel, Igor Melnyk, Arindam Banerjee

In many multivariate time series modeling problems, there is usually a significant linear dependency component, for which VARs are suitable, and a nonlinear component, for which RNNs are suitable.

Multivariate Time Series Forecasting Time Series

Robust Structured Estimation with Single-Index Models

no code implementations ICML 2017 Sheng Chen, Arindam Banerjee

In this paper, we investigate general single-index models (SIMs) in high dimensions.

High-Dimensional Structured Quantile Regression

no code implementations ICML 2017 Vidyashankar Sivakumar, Arindam Banerjee

In this work we consider the problem of linear quantile regression in high dimensions where the number of predictor variables is much higher than the number of samples available for parameter estimation.

High Dimensional Structured Superposition Models

no code implementations NeurIPS 2016 Qilong Gu, Arindam Banerjee

High dimensional superposition models characterize observations using parameters which can be written as a sum of multiple component parameters, each with its own structure, e. g., sum of low rank and sparse matrices, sum of sparse and rotated sparse vectors, etc.

Spatial Projection of Multiple Climate Variables using Hierarchical Multitask Learning

no code implementations30 Jan 2017 André R. Gonçalves, Arindam Banerjee, Fernando J. Von Zuben

While IPCC has traditionally used a simple model output average, recent work has illustrated potential advantages of using a multitask learning (MTL) framework for projections of individual climate variables.

Recommendation under Capacity Constraints

no code implementations18 Jan 2017 Konstantina Christakopoulou, Jaya Kawale, Arindam Banerjee

In this paper, we investigate the common scenario where every candidate item for recommendation is characterized by a maximum capacity, i. e., number of seats in a Point-of-Interest (POI) or size of an item's inventory.

Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data

no code implementations27 Dec 2016 Anuj Karpatne, Gowtham Atluri, James Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, Vipin Kumar

Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of data science models in enabling scientific discovery.

Alternating Estimation for Structured High-Dimensional Multi-Response Models

no code implementations NeurIPS 2017 Sheng Chen, Arindam Banerjee

We consider learning high-dimensional multi-response linear models with structured parameters.

Structured Stochastic Linear Bandits

no code implementations17 Jun 2016 Nicholas Johnson, Vidyashankar Sivakumar, Arindam Banerjee

The goal in such a problem is to minimize the (pseudo) regret which is the difference between the total expected loss of the algorithm and the total expected loss of the best fixed vector in hindsight.

Generalized Direct Change Estimation in Ising Model Structure

no code implementations16 Jun 2016 Farideh Fazayeli, Arindam Banerjee

We consider the problem of estimating change in the dependency structure between two $p$-dimensional Ising models, based on respectively $n_1$ and $n_2$ samples drawn from the models.

Structured Matrix Recovery via the Generalized Dantzig Selector

no code implementations NeurIPS 2016 Sheng Chen, Arindam Banerjee

In recent years, structured matrix recovery problems have gained considerable attention for its real world applications, such as recommender systems and computer vision.

Recommendation Systems

The Matrix Generalized Inverse Gaussian Distribution: Properties and Applications

no code implementations12 Apr 2016 Farideh Fazayeli, Arindam Banerjee

Based on the property, we propose an importance sampling method for the $\mathcal{MGIG}$ where the mode of the proposal distribution matches that of the target.

Unified View of Matrix Completion under General Structural Constraints

no code implementations NeurIPS 2015 Suriya Gunasekar, Arindam Banerjee, Joydeep Ghosh

In this paper, we present a unified analysis of matrix completion under general low-dimensional structural constraints induced by {\em any} norm regularization.

Matrix Completion

Estimating Structured Vector Autoregressive Model

no code implementations21 Feb 2016 Igor Melnyk, Arindam Banerjee

While considerable advances have been made in estimating high-dimensional structured models from independent data using Lasso-type models, limited progress has been made for settings when the samples are dependent.

Semi-Markov Switching Vector Autoregressive Model-based Anomaly Detection in Aviation Systems

no code implementations21 Feb 2016 Igor Melnyk, Arindam Banerjee, Bryan Matthews, Nikunj Oza

In this context the goal is to detect anomalous flight segments, due to mechanical, environmental, or human factors in order to identifying operationally significant events and provide insights into the flight operations and highlight otherwise unavailable potential safety risks and precursors to accidents.

Anomaly Detection Time Series

Structured Estimation with Atomic Norms: General Bounds and Applications

no code implementations NeurIPS 2015 Sheng Chen, Arindam Banerjee

For structured estimation problems with atomic norms, recent advances in the literature express sample complexity and estimation error bounds in terms of certain geometric measures, in particular Gaussian width of the unit norm ball, Gaussian width of a spherical cap induced by a tangent cone, and a restricted norm compatibility constant.

Beyond Sub-Gaussian Measurements: High-Dimensional Structured Estimation with Sub-Exponential Designs

no code implementations NeurIPS 2015 Vidyashankar Sivakumar, Arindam Banerjee, Pradeep K. Ravikumar

In contrast, for the sub-exponential setting, we show that the sample complexity and the estimation error will depend on the exponential width of the corresponding sets, and the analysis holds for any norm.

Estimation with Norm Regularization

no code implementations NeurIPS 2014 Arindam Banerjee, Sheng Chen, Farideh Fazayeli, Vidyashankar Sivakumar

Analysis of non-asymptotic estimation error and structured statistical recovery based on norm regularized regression, such as Lasso, needs to consider four aspects: the norm, the loss function, the design matrix, and the noise model.

Parallel Direction Method of Multipliers

no code implementations NeurIPS 2014 Huahua Wang, Arindam Banerjee, Zhi-Quan Luo

In this paper, we propose a parallel randomized block coordinate method named Parallel Direction Method of Multipliers (PDMM) to solve the optimization problems with multi-block linear constraints.

Multi-task Sparse Structure Learning

no code implementations1 Sep 2014 Andre R. Goncalves, Puja Das, Soumyadeep Chatterjee, Vidyashankar Sivakumar, Fernando J. Von Zuben, Arindam Banerjee

We illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark datasets for regression and classification.

General Classification Multi-Task Learning

A Spectral Algorithm for Inference in Hidden Semi-Markov Models

no code implementations12 Jul 2014 Igor Melnyk, Arindam Banerjee

Hidden semi-Markov models (HSMMs) are latent variable models which allow latent state persistence and can be viewed as a generalization of the popular hidden Markov models (HMMs).

Randomized Block Coordinate Descent for Online and Stochastic Optimization

no code implementations1 Jul 2014 Huahua Wang, Arindam Banerjee

One is online or stochastic gradient descent (OGD/SGD), and the other is randomzied coordinate descent (RBCD).

Stochastic Optimization

Generalized Dantzig Selector: Application to the k-support norm

no code implementations NeurIPS 2014 Soumyadeep Chatterjee, Sheng Chen, Arindam Banerjee

For statistical analysis, we provide upper bounds for the Gaussian widths needed in the GDS analysis, yielding the first statistical recovery guarantee for estimation with the $k$-support norm.

Large Scale Distributed Sparse Precision Estimation

no code implementations NeurIPS 2013 Huahua Wang, Arindam Banerjee, Cho-Jui Hsieh, Pradeep K. Ravikumar, Inderjit S. Dhillon

We consider the problem of sparse precision matrix estimation in high dimensions using the CLIME estimator, which has several desirable theoretical properties.

Bethe-ADMM for Tree Decomposition based Parallel MAP Inference

no code implementations26 Sep 2013 Qiang Fu, Huahua Wang, Arindam Banerjee

We present a parallel MAP inference algorithm called Bethe-ADMM based on two ideas: tree-decomposition of the graph and the alternating direction method of multipliers (ADMM).

Online Alternating Direction Method (longer version)

no code implementations17 Jun 2013 Huahua Wang, Arindam Banerjee

Online optimization has emerged as powerful tool in large scale optimization.

online learning

Bregman Alternating Direction Method of Multipliers

no code implementations NeurIPS 2014 Huahua Wang, Arindam Banerjee

The mirror descent algorithm (MDA) generalizes gradient descent by using a Bregman divergence to replace squared Euclidean distance.

Cannot find the paper you are looking for? You can Submit a new open access paper.