Search Results for author: Vaneet Aggarwal

Found 64 papers, 4 papers with code

Multi-agent Covering Option Discovery based on Kronecker Product of Factor Graphs

no code implementations20 Jan 2022 Jiayu Chen, Jingdi Chen, Tian Lan, Vaneet Aggarwal

Covering option discovery has been developed to improve the exploration of reinforcement learning in single-agent scenarios with sparse reward signals, through connecting the most distant states in the embedding space provided by the Fiedler vector of the state transition graph.

Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients

1 code implementation4 Jan 2022 Hanhan Zhou, Tian Lan, Vaneet Aggarwal

To this end, we present LSF-SAC, a novel framework that features a variational inference-based information-sharing mechanism as extra state information to assist individual agents in the value function factorization.

Starcraft Starcraft II +1

Learning Circular Hidden Quantum Markov Models: A Tensor Network Approach

no code implementations29 Oct 2021 Mohammad Ali Javidian, Vaneet Aggarwal, Zubin Jacob

In this paper, we propose circular Hidden Quantum Markov Models (c-HQMMs), which can be applied for modeling temporal data in quantum datasets (with classical datasets as a special case).

Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programming

no code implementations22 Oct 2021 Alec Koppel, Amrit Singh Bedi, Bhargav Ganguly, Vaneet Aggarwal

We establish that the sample complexity to obtain near-globally optimal solutions matches tight dependencies on the cardinality of the state and action spaces, and exhibits classical scalings with respect to the network in accordance with multi-agent optimization.

Multi-agent Reinforcement Learning

Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approach

no code implementations13 Sep 2021 Qinbo Bai, Amrit Singh Bedi, Mridul Agarwal, Alec Koppel, Vaneet Aggarwal

To achieve that, we advocate the use of a randomized primal-dual approach to solving the CMDP problems and propose a conservative stochastic primal-dual algorithm (CSPDA) which is shown to exhibit $\tilde{\mathcal{O}}(1/\epsilon^2)$ sample complexity to achieve $\epsilon$-optimal cumulative reward with zero constraint violations.

Concave Utility Reinforcement Learning with Zero-Constraint Violations

no code implementations12 Sep 2021 Mridul Agarwal, Qinbo Bai, Vaneet Aggarwal

We consider the problem of tabular infinite horizon concave utility reinforcement learning (CURL) with convex constraints.

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

no code implementations9 Sep 2021 Xinwu Qian, Shuocheng Guo, Vaneet Aggarwal

This study proposes the deep relocating option policy (DROP) that supervises vehicle agents to escape from oversupply areas and effectively relocate to potentially underserved areas.

On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)

no code implementations9 Sep 2021 Washim Uddin Mondal, Mridul Agarwal, Vaneet Aggarwal, Satish V. Ukkusuri

We show that, in these cases, the $K$-class MARL problem can be approximated by MFC with errors given as $e_1=\mathcal{O}(\frac{\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}}{N_{\mathrm{pop}}}\sum_{k}\sqrt{N_k})$, $e_2=\mathcal{O}(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\sum_{k}\frac{1}{\sqrt{N_k}})$ and $e_3=\mathcal{O}\left(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\left[\frac{A}{N_{\mathrm{pop}}}\sum_{k\in[K]}\sqrt{N_k}+\frac{B}{\sqrt{N_{\mathrm{pop}}}}\right]\right)$, respectively, where $A, B$ are some constants and $|\mathcal{X}|,|\mathcal{U}|$ are the sizes of state and action spaces of each agent.

Multi-agent Reinforcement Learning

An FEA surrogate model with Boundary Oriented Graph Embedding approach

no code implementations30 Aug 2021 Xingyu Fu, Fengfeng Zhou, Dheeraj Peddireddy, Zhengyang Kang, Martin Byung-Guk Jun, Vaneet Aggarwal

In this work, we present a Boundary Oriented Graph Embedding (BOGE) approach for the Graph Neural Network (GNN) to serve as a general surrogate model for regressing physical fields and solving boundary value problems.

Cantilever Beam Decision Making +1

Markov Decision Processes with Long-Term Average Constraints

no code implementations12 Jun 2021 Mridul Agarwal, Qinbo Bai, Vaneet Aggarwal

We consider the problem of constrained Markov Decision Process (CMDP) where an agent interacts with a unichain Markov Decision Process.

Joint Optimization of Multi-Objective Reinforcement Learning with Policy Gradient Based Algorithm

no code implementations28 May 2021 Qinbo Bai, Mridul Agarwal, Vaneet Aggarwal

Many engineering problems have multiple objectives, and the overall aim is to optimize a non-linear function of these objectives.

Quantum Causal Inference in the Presence of Hidden Common Causes: an Entropic Approach

no code implementations24 Apr 2021 Mohammad Ali Javidian, Vaneet Aggarwal, Zubin Jacob

Quantum causality is an emerging field of study which has the potential to greatly advance our understanding of quantum systems.

Causal Inference

AdaPool: A Diurnal-Adaptive Fleet Management Framework using Model-Free Deep Reinforcement Learning and Change Point Detection

no code implementations1 Apr 2021 Marina Haliem, Vaneet Aggarwal, Bharat Bhargava

To mitigate this problem in highly dynamic environments, we (1) adopt an online Dirichlet change point detection (ODCP) algorithm to detect the changes in the distribution of experiences, (2) develop a Deep Q Network (DQN) agent that is capable of recognizing diurnal patterns and making informed dispatching decisions according to the changes in the underlying environment.

Change Point Detection

DeepFreight: A Model-free Deep-reinforcement-learning-based Algorithm for Multi-transfer Freight Delivery

no code implementations5 Mar 2021 Jiayu Chen, Abhishek K. Umrawal, Tian Lan, Vaneet Aggarwal

With the freight delivery demands and shipping costs increasing rapidly, intelligent control of fleets to enable efficient and cost-conscious solutions becomes an important problem.

Multi-agent Reinforcement Learning

Quantum Entropic Causal Inference

no code implementations23 Feb 2021 Mohammad Ali Javidian, Vaneet Aggarwal, Fanglin Bao, Zubin Jacob

This successful inference on a synthetic quantum dataset can have practical applications in identifying originators of malicious activity on future multi-node quantum networks as well as quantum error correction.

Causal Inference

Communication Efficient Parallel Reinforcement Learning

no code implementations22 Feb 2021 Mridul Agarwal, Bhargav Ganguly, Vaneet Aggarwal

We provide \NAM\ which runs at each agent and prove that the total cumulative regret of $M$ agents is upper bounded as $\Tilde{O}(DS\sqrt{MAT})$ for a Markov Decision Process with diameter $D$, number of states $S$, and number of actions $A$.

Multi-Agent Multi-Armed Bandits with Limited Communication

no code implementations10 Feb 2021 Mridul Agarwal, Vaneet Aggarwal, Kamyar Azizzadenesheli

With our algorithm, LCC-UCB, each agent enjoys a regret of $\tilde{O}\left(\sqrt{({K/N}+ N)T}\right)$, communicates for $O(\log T)$ steps and broadcasts $O(\log K)$ bits in each communication step.

Multi-Armed Bandits

A Supervised Learning Approach for Robust Health Monitoring using Face Videos

no code implementations30 Jan 2021 Mayank Gupta, Lingjun Chen, Denny Yu, Vaneet Aggarwal

Non-contact methods can have additional advantages since they are scalable with any environment where video can be captured, can be used for continuous measurements, and can be used on patients with varying levels of dexterity and independence, from people with physical impairments to infants (e. g., baby camera).

Model Free Reinforcement Learning Algorithm for Stationary Mean field Equilibrium for Multiple Types of Agents

no code implementations31 Dec 2020 Arnob Ghosh, Vaneet Aggarwal

We consider a multi-agent Markov strategic interaction over an infinite horizon where agents can be of multiple types.

A multi-agent evolutionary robotics framework to train spiking neural networks

no code implementations7 Dec 2020 Souvik Das, Anirudh Shankar, Vaneet Aggarwal

Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment.

PassGoodPool: Joint Passengers and Goods Fleet Management with Reinforcement Learning aided Pricing, Matching, and Route Planning

no code implementations17 Nov 2020 Kaushik Manchella, Marina Haliem, Vaneet Aggarwal, Bharat Bhargava

The ubiquitous growth of mobility-on-demand services for passenger and goods delivery has brought various challenges and opportunities within the realm of transportation systems.

Decision Making

Blind Decision Making: Reinforcement Learning with Delayed Observations

no code implementations16 Nov 2020 Mridul Agarwal, Vaneet Aggarwal

This paper proposes an approach, where the delay in the knowledge of the state can be used, and the decisions are made based on the available information which may not include the current state information.

Decision Making

A Distributed Model-Free Ride-Sharing Approach for Joint Matching, Pricing, and Dispatching using Deep Reinforcement Learning

no code implementations5 Oct 2020 Marina Haliem, Ganapathy Mani, Vaneet Aggarwal, Bharat Bhargava

In this paper, we present a dynamic, demand aware, and pricing-based vehicle-passenger matching and route planning framework that (1) dynamically generates optimal routes for each vehicle based on online demand, pricing associated with each ride, vehicle capacities and locations.

Decision Making

Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning

no code implementations27 Sep 2020 Ramkumar Raghu, Mahadesh Panju, Vaneet Aggarwal, Vinod Sharma

In this paper, we use deep reinforcement learning where we use function approximation of the Q-function via a deep neural network to obtain a power control policy that matches the optimal policy for a small network.

Stochastic Optimization

FlexPool: A Distributed Model-Free Deep Reinforcement Learning Algorithm for Joint Passengers & Goods Transportation

no code implementations27 Jul 2020 Kaushik Manchella, Abhishek K. Umrawal, Vaneet Aggarwal

Through simulations on a realistic multi-agent urban mobility platform, we demonstrate that FlexPool outperforms other model-free settings in serving the demands from passengers & goods.

Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks

1 code implementation18 Jul 2020 Seyyedali Hosseinalipour, Sheikh Shams Azam, Christopher G. Brinton, Nicolo Michelusi, Vaneet Aggarwal, David J. Love, Huaiyu Dai

We derive the upper bound of convergence for MH-FL with respect to parameters of the network topology (e. g., the spectral radius) and the learning algorithm (e. g., the number of D2D rounds in different clusters).

Federated Learning

Model-Free Algorithm and Regret Analysis for MDPs with Long-Term Constraints

no code implementations10 Jun 2020 Qinbo Bai, Vaneet Aggarwal, Ather Gattami

This paper uses concepts from constrained optimization and Q-learning to propose an algorithm for CMDP with long-term constraints.

Q-Learning

Efficient Large-Scale Gaussian Process Bandits by Believing only Informative Actions

no code implementations L4DC 2020 Amrit Singh Bedi, Dheeraj Peddireddy, Vaneet Aggarwal, Alec Koppel

Experimentally, we observe state of the art accuracy and complexity tradeoffs for GP bandit algorithms on various hyper-parameter tuning tasks, suggesting the merits of managing the complexity of GPs in bandit settings

From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks

no code implementations7 Jun 2020 Seyyedali Hosseinalipour, Christopher G. Brinton, Vaneet Aggarwal, Huaiyu Dai, Mung Chiang

There are several challenges with employing conventional federated learning in contemporary networks, due to the significant heterogeneity in compute and communication capabilities that exist across devices.

Federated Learning Recommendation Systems

Sublinear Regret and Belief Complexity in Gaussian Process Bandits via Information Thresholding

no code implementations23 Mar 2020 Amrit Singh Bedi, Dheeraj Peddireddy, Vaneet Aggarwal, Alec Koppel

Bayesian optimization is a framework for global search via maximum a posteriori updates rather than simulated annealing, and has gained prominence for decision-making under uncertainty.

Decision Making Decision Making Under Uncertainty

Provably Efficient Model-Free Algorithm for MDPs with Peak Constraints

no code implementations11 Mar 2020 Qinbo Bai, Vaneet Aggarwal, Ather Gattami

We propose a model-free algorithm that converts CMDP problem to an unconstrained problem and a Q-learning based approach is used.

Q-Learning

A Distributed Model-Free Algorithm for Multi-hop Ride-sharing using Deep Reinforcement Learning

no code implementations30 Oct 2019 Ashutosh Singh, Abubakr Alabbasi, Vaneet Aggarwal

The growth of autonomous vehicles, ridesharing systems, and self driving technology will bring a shift in the way ride hailing platforms plan out their services.

Autonomous Vehicles

Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning

no code implementations23 Oct 2019 Anis Elgabli, Jihong Park, Amrit S. Bedi, Chaouki Ben Issaid, Mehdi Bennis, Vaneet Aggarwal

In this article, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM).

Image Classification Quantization

Deep Reinforcement Learning Based Power control for Wireless Multicast Systems

no code implementations27 Sep 2019 Ramkumar Raghu, Pratheek Upadhyaya, Mahadesh Panju, Vaneet Aggarwal, Vinod Sharma

However for this system, obtaining optimal power control is intractable because of a very large state space.

Encoders and Decoders for Quantum Expander Codes Using Machine Learning

no code implementations6 Sep 2019 Sathwik Chadaga, Mridul Agarwal, Vaneet Aggarwal

However, large-scale design of quantum encoders and decoders have to depend on the channel characteristics and require look-up tables which require memory that is exponential in the number of qubits.

Q-Learning

Reinforcement Learning for Joint Optimization of Multiple Rewards

no code implementations6 Sep 2019 Mridul Agarwal, Vaneet Aggarwal

Reinforcement Learning (RL) algorithms such as DQN owe their success to Markov Decision Processes, and the fact that maximizing the sum of rewards allows using backward induction and reduce to the Bellman optimality equation.

Decision Making Fairness

GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning

no code implementations30 Aug 2019 Anis Elgabli, Jihong Park, Amrit S. Bedi, Mehdi Bennis, Vaneet Aggarwal

When the data is distributed across multiple servers, lowering the communication cost between the servers (or workers) while solving the distributed learning problem is an important problem and is the focus of this paper.

Reinforcement Learning for Mean Field Game

no code implementations30 May 2019 Mridul Agarwal, Vaneet Aggarwal, Arnob Ghosh, Nilay Tiwari

This paper focuses on finding a mean-field equilibrium (MFE) in an action coupled stochastic game setting in an episodic framework.

DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learning

no code implementations9 Mar 2019 Abubakr Alabbasi, Arnob Ghosh, Vaneet Aggarwal

The success of modern ride-sharing platforms crucially depends on the profit of the ride-sharing fleet operating companies, and how efficiently the resources are managed.

A Proximal Jacobian ADMM Approach for Fast Massive MIMO Signal Detection in Low-Latency Communications

1 code implementation2 Mar 2019 Anis Elgabli, Ali Elghariani, Vaneet Aggarwal, *Mehdi Bennis, Mark R. Bell

We introduce an objective function that is a sum of strictly convex and separable functions based on decomposing the received vector into multiple vectors.

Information Theory Information Theory

Stochastic Top-$K$ Subset Bandits with Linear Space and Non-Linear Feedback

no code implementations29 Nov 2018 Mridul Agarwal, Vaneet Aggarwal, Christopher J. Quinn, Abhishek K. Umrawal

Many real-world problems like Social Influence Maximization face the dilemma of choosing the best $K$ out of $N$ options at a given time instant.

Multi-Armed Bandits

Covfefe: A Computer Vision Approach For Estimating Force Exertion

no code implementations25 Sep 2018 Vaneet Aggarwal, Hamed Asadi, Mayank Gupta, Jae Joong Lee, Denny Yu

We note that the PPG signals can be obtained from the face videos, thus giving an efficient classification algorithm for the force exertion levels using face videos.

General Classification Photoplethysmography (PPG)

FastScan: Robust Low-Complexity Rate Adaptation Algorithm for Video Streaming over HTTP

1 code implementation7 Jun 2018 Anis Elgabli, Vaneet Aggarwal

For example, on an experiment conducted over 100 real cellular bandwidth traces of a public dataset that spans different bandwidth regimes, our proposed algorithm (FastScan) achieves the minimum re-buffering (stall) time and the maximum average playback rate in every single trace as compared to the original dash. js rate adaptation scheme, Festive, BBA, RB, and FastMPC algorithms.

Networking and Internet Architecture Multimedia

LBP: Robust Rate Adaptation Algorithm for SVC Video Streaming

no code implementations30 Apr 2018 Anis Elgabli, Vaneet Aggarwal, Shuai Hao, Feng Qian, Subhabrata Sen

The objective is to optimize a novel QoE metric that models a combination of the three objectives of minimizing the stall/skip duration of the video, maximizing the playback quality of every chunk, and minimizing the number of quality switches.

Networking and Internet Architecture Multimedia

Principal Component Analysis with Tensor Train Subspace

no code implementations13 Mar 2018 Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron

Tensor train is a hierarchical tensor network structure that helps alleviate the curse of dimensionality by parameterizing large-scale multidimensional data via a set of network of low-rank tensors.

Wide Compression: Tensor Ring Nets

no code implementations CVPR 2018 Wenqi Wang, Yifan Sun, Brian Eriksson, Wenlin Wang, Vaneet Aggarwal

Deep neural networks have demonstrated state-of-the-art performance in a variety of real-world applications.

Image Classification

On Deterministic Sampling Patterns for Robust Low-Rank Matrix Completion

no code implementations5 Dec 2017 Morteza Ashraphijuo, Vaneet Aggarwal, Xiaodong Wang

In this letter, we study the deterministic sampling patterns for the completion of low rank matrix, when corrupted with a sparse noise, also known as robust matrix completion.

Low-Rank Matrix Completion

Tensor Train Neighborhood Preserving Embedding

no code implementations3 Dec 2017 Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron

In this paper, we propose a Tensor Train Neighborhood Preserving Embedding (TTNPE) to embed multi-dimensional tensor data into low dimensional tensor subspace.

Dimensionality Reduction General Classification

Efficient Low Rank Tensor Ring Completion

no code implementations ICCV 2017 Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron

Using the matrix product state (MPS) representation of the recently proposed tensor ring decompositions, in this paper we propose a tensor completion algorithm, which is an alternating minimization algorithm that alternates over the factors in the MPS representation.

Matrix Completion

Rank Determination for Low-Rank Data Completion

no code implementations3 Jul 2017 Morteza Ashraphijuo, Xiaodong Wang, Vaneet Aggarwal

Moreover, for both single-view matrix and CP tensor, we are able to show that the obtained upper bound is exactly equal to the unknown rank if the lowest-rank completion is given.

Deterministic and Probabilistic Conditions for Finite Completability of Low-rank Multi-View Data

no code implementations3 Jan 2017 Morteza Ashraphijuo, Xiaodong Wang, Vaneet Aggarwal

We provide a deterministic necessary and sufficient condition on the sampling pattern for finite completability.

Matrix Completion

Deterministic and Probabilistic Conditions for Finite Completability of Low-Tucker-Rank Tensor

no code implementations6 Dec 2016 Morteza Ashraphijuo, Vaneet Aggarwal, Xiaodong Wang

We investigate the fundamental conditions on the sampling pattern, i. e., locations of the sampled entries, for finite completability of a low-rank tensor given some components of its Tucker rank.

Unsupervised clustering under the Union of Polyhedral Cones (UOPC) model

no code implementations15 Oct 2016 Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron

Similar to the Union of Subspaces (UOS) model where each data from each subspace is generated from a (unknown) basis, in the UOPC model each data from each cone is assumed to be generated from a finite number of (unknown) \emph{extreme rays}. To cluster data under this model, we consider several algorithms - (a) Sparse Subspace Clustering by Non-negative constraints Lasso (NCL), (b) Least squares approximation (LSA), and (c) K-nearest neighbor (KNN) algorithm to arrive at affinity between data points.

Low-tubal-rank Tensor Completion using Alternating Minimization

no code implementations5 Oct 2016 Xiao-Yang Liu, Shuchin Aeron, Vaneet Aggarwal, Xiaodong Wang

The low-tubal-rank tensor model has been recently proposed for real-world multidimensional data.

Low-Rank Matrix Completion

Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model

no code implementations19 Sep 2016 Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron

Using the matrix product state (MPS) representation of tensor train decompositions, in this paper we propose a tensor completion algorithm which alternates over the matrices (tensors) in the MPS representation.

Matrix Completion

On Deterministic Conditions for Subspace Clustering under Missing Data

no code implementations11 Jul 2016 Wenqi Wang, Shuchin Aeron, Vaneet Aggarwal

In this paper we present deterministic conditions for success of sparse subspace clustering (SSC) under missing data, when data is assumed to come from a Union of Subspaces (UoS) model.

On deterministic conditions for subspace clustering under missing data

no code implementations15 Apr 2016 Wenqi Wang, Shuchin Aeron, Vaneet Aggarwal

We provide extensive set of simulation results for clustering as well as completion of data under missing entries, under the UoS model.

Information-theoretic Bounds on Matrix Completion under Union of Subspaces Model

no code implementations14 Aug 2015 Vaneet Aggarwal, Shuchin Aeron

In this short note we extend some of the recent results on matrix completion under the assumption that the columns of the matrix can be grouped (clustered) into subspaces (not necessarily disjoint or independent).

Matrix Completion

Adaptive Sampling of RF Fingerprints for Fine-grained Indoor Localization

no code implementations10 Aug 2015 Xiao-Yang Liu, Shuchin Aeron, Vaneet Aggarwal, Xiaodong Wang, Min-You Wu

In contrast to several existing work that rely on random sampling, this paper shows that adaptivity in sampling can lead to significant improvements in localization accuracy.

Indoor Localization

Cannot find the paper you are looking for? You can Submit a new open access paper.