Search Results for author: Alvaro Velasquez

Found 34 papers, 5 papers with code

Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents

no code implementations6 Feb 2024 Yash Shukla, Tanushree Burman, Abhishek Kulkarni, Robert Wright, Alvaro Velasquez, Jivko Sinapov

In this work, we propose a novel approach, called Logical Specifications-guided Dynamic Task Sampling (LSTS), that learns a set of RL policies to guide an agent from an initial state to a goal state based on a high-level task specification, while minimizing the number of environmental interactions.

Continuous Control Decision Making +3

A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence

no code implementations6 Jan 2024 Justus Renkhoff, Ke Feng, Marc Meier-Doernberg, Alvaro Velasquez, Houbing Herbert Song

Since neurosymbolic AI combines the advantages of both symbolic and sub-symbolic AI, this survey explores how neurosymbolic applications can ease the V&V process.

Assume-Guarantee Reinforcement Learning

no code implementations15 Dec 2023 Milad Kazemi, Mateo Perez, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi, Alvaro Velasquez

We present a modular approach to \emph{reinforcement learning} (RL) in environments consisting of simpler components evolving in parallel.

reinforcement-learning Reinforcement Learning (RL)

LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agents

no code implementations14 Oct 2023 Yash Shukla, Wenchang Gao, Vasanth Sarathy, Alvaro Velasquez, Robert Wright, Jivko Sinapov

In this work, we propose LgTS (LLM-guided Teacher-Student learning), a novel approach that explores the planning abilities of LLMs to provide a graphical representation of the sub-goals to a reinforcement learning (RL) agent that does not have access to the transition dynamics of the environment.

Reinforcement Learning (RL)

Byzantine-Resilient Decentralized Multi-Armed Bandits

no code implementations11 Oct 2023 Jingxuan Zhu, Alec Koppel, Alvaro Velasquez, Ji Liu

In decentralized cooperative multi-armed bandits (MAB), each agent observes a distinct stream of rewards, and seeks to exchange information with others to select a sequence of arms so as to minimize its regret.

Multi-Armed Bandits Recommendation Systems

Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving

no code implementations28 Sep 2023 Sumit Kumar Jha, Susmit Jha, Patrick Lincoln, Nathaniel D. Bastian, Alvaro Velasquez, Rickard Ewetz, Sandeep Neema

We posit that we can use the satisfiability modulo theory (SMT) solvers as deductive reasoning engines to analyze the generated solutions from the LLMs, produce counterexamples when the solutions are incorrect, and provide that feedback to the LLMs exploiting the dialog capability of instruct-trained LLMs.

Hallucination Question Answering +1

Neural Stochastic Differential Equations for Robust and Explainable Analysis of Electromagnetic Unintended Radiated Emissions

no code implementations27 Sep 2023 Sumit Kumar Jha, Susmit Jha, Rickard Ewetz, Alvaro Velasquez

We provide an empirical demonstration of the fragility of ResNet-like models to Gaussian noise perturbations, where the model performance deteriorates sharply and its F1-score drops to near insignificance at 0. 008 with a Gaussian noise of only 0. 5 standard deviation.

Attribute Interpretable Machine Learning

SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments

no code implementations8 Sep 2023 Abhinav Rajvanshi, Karan Sikka, Xiao Lin, Bhoram Lee, Han-Pang Chiu, Alvaro Velasquez

We present SayNav, a new approach that leverages human knowledge from Large Language Models (LLMs) for efficient generalization to complex navigation tasks in unknown large-scale environments.

Common Sense Reasoning Navigate

Safety Margins for Reinforcement Learning

no code implementations25 Jul 2023 Alexander Grushin, Walt Woods, Alvaro Velasquez, Simon Khan

Proxy criticality metrics that are computable in real-time (i. e., without actually simulating the effects of random actions) can be compared to the true criticality, and we show how to leverage these proxy metrics to generate safety margins, which directly tie the consequences of potentially incorrect actions to an anticipated loss in overall performance.

reinforcement-learning

Model-Free Robust Average-Reward Reinforcement Learning

no code implementations17 May 2023 Yue Wang, Alvaro Velasquez, George Atia, Ashley Prater-Bennette, Shaofeng Zou

Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs.

Q-Learning reinforcement-learning

Automaton-Guided Curriculum Generation for Reinforcement Learning Agents

1 code implementation11 Apr 2023 Yash Shukla, Abhishek Kulkarni, Robert Wright, Alvaro Velasquez, Jivko Sinapov

Experiments in gridworld and physics-based simulated robotics domains show that the curricula produced by AGCL achieve improved time-to-threshold performance on a complex sequential decision-making problem relative to state-of-the-art curriculum learning (e. g, teacher-student, self-play) and automaton-guided reinforcement learning baselines (e. g, Q-Learning for Reward Machines).

Decision Making Q-Learning +2

A Resilient Distributed Algorithm for Solving Linear Equations

no code implementations1 Apr 2023 Jingxuan Zhu, Alvaro Velasquez, Ji Liu

This paper presents a resilient distributed algorithm for solving a system of linear algebraic equations over a multi-agent network in the presence of Byzantine agents capable of arbitrarily introducing untrustworthy information in communication.

Exploring Adversarial Attacks on Neural Networks: An Explainable Approach

1 code implementation8 Mar 2023 Justus Renkhoff, Wenkai Tan, Alvaro Velasquez, illiam Yichen Wang, Yongxin Liu, Jian Wang, Shuteng Niu, Lejla Begic Fazlic, Guido Dartmann, Houbing Song

Finally, we demonstrate that the layers $Block4\_conv1$ and $Block5\_cov1$ of the VGG-16 model are more susceptible to adversarial attacks.

Autonomous Driving

On the Robustness of AlphaFold: A COVID-19 Case Study

no code implementations10 Jan 2023 Ismail Alkhouri, Sumit Jha, Andre Beckus, George Atia, Alvaro Velasquez, Rickard Ewetz, Arvind Ramanathan, Susmit Jha

To measure the robustness of the predicted structures, we utilize (i) the root-mean-square deviation (RMSD) and (ii) the Global Distance Test (GDT) similarity measure between the predicted structure of the original sequence and the structure of its adversarially perturbed version.

Protein Folding

Robust Average-Reward Markov Decision Processes

no code implementations2 Jan 2023 Yue Wang, Alvaro Velasquez, George Atia, Ashley Prater-Bennette, Shaofeng Zou

We derive the robust Bellman equation for robust average-reward MDPs, prove that the optimal policy can be derived from its solution, and further design a robust relative value iteration algorithm that provably finds its solution, or equivalently, the optimal robust policy.

Resilient Constrained Consensus over Complete Graphs via Feasibility Redundancy

no code implementations26 Mar 2022 Jingxuan Zhu, Yixuan Lin, Alvaro Velasquez, Ji Liu

This paper considers a resilient high-dimensional constrained consensus problem and studies a resilient distributed algorithm for complete graphs.

A Differentiable Approach to Combinatorial Optimization using Dataless Neural Networks

no code implementations15 Mar 2022 Ismail R. Alkhouri, George K. Atia, Alvaro Velasquez

In particular, we reduce the combinatorial optimization problem to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest.

Combinatorial Optimization Community Detection +3

Protein Folding Neural Networks Are Not Robust

no code implementations9 Sep 2021 Sumit Kumar Jha, Arvind Ramanathan, Rickard Ewetz, Alvaro Velasquez, Susmit Jha

We define the robustness measure for the predicted structure of a protein sequence to be the inverse of the root-mean-square distance (RMSD) in the predicted structure and the structure of its adversarially perturbed sequence.

Adversarial Attack Protein Folding

Pulmonary Disease Classification Using Globally Correlated Maximum Likelihood: an Auxiliary Attention mechanism for Convolutional Neural Networks

1 code implementation1 Sep 2021 Edward Verenich, Tobias Martin, Alvaro Velasquez, Nazar Khan, Faraz Hussain

Two complementary generalization properties of CNNs, translation invariance and equivariance, are particularly useful in detecting manifested abnormalities associated with pulmonary disease, regardless of their spatial locations within the image.

Translation

BOSS: Bidirectional One-Shot Synthesis of Adversarial Examples

1 code implementation5 Aug 2021 Ismail R. Alkhouri, Alvaro Velasquez, George K. Atia

To this end, we present a problem that encodes objectives on the distance between the desired and output distributions of the trained model and the similarity between such inputs and the synthesized examples.

Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learning

no code implementations9 Jul 2021 Taylor Dohmen, Noah Topper, George Atia, Andre Beckus, Ashutosh Trivedi, Alvaro Velasquez

The success of reinforcement learning in typical settings is predicated on Markovian assumptions on the reward signal by which an agent learns optimal policies.

Decision Making reinforcement-learning +1

Controller Synthesis for Omega-Regular and Steady-State Specifications

no code implementations5 Jun 2021 Alvaro Velasquez, Ismail Alkhouri, Andre Beckus, Ashutosh Trivedi, George Atia

Given a Markov decision process (MDP) and a linear-time ($\omega$-regular or LTL) specification, the controller synthesis problem aims to compute the optimal policy that satisfies the specification.

Robust Ensembles of Neural Networks using Itô Processes

no code implementations1 Jan 2021 Sumit Kumar Jha, Susmit Jha, Rickard Ewetz, Alvaro Velasquez

We exploit this connection and the theory of stochastic dynamical systems to construct a novel ensemble of Itô processes as a new deep learning representation that is more robust than classical residual networks.

Steady-State Planning in Expected Reward Multichain MDPs

no code implementations3 Dec 2020 George K. Atia, Andre Beckus, Ismail Alkhouri, Alvaro Velasquez

In this paper, we explore this steady-state planning problem that consists of deriving a decision-making policy for an agent such that constraints on its steady-state behavior are satisfied.

Decision Making

Domain Wall Leaky Integrate-and-Fire Neurons with Shape-Based Configurable Activation Functions

no code implementations11 Nov 2020 Wesley H. Brigner, Naimul Hassan, Xuan Hu, Christopher H. Bennett, Felipe Garcia-Sanchez, Can Cui, Alvaro Velasquez, Matthew J. Marinella, Jean Anne C. Incorvia, Joseph S. Friedman

This work proposes modifications to these spintronic neurons that enable configuration of the activation functions through control of the shape of a magnetic domain wall track.

An Extension of Fano's Inequality for Characterizing Model Susceptibility to Membership Inference Attacks

no code implementations17 Sep 2020 Sumit Kumar Jha, Susmit Jha, Rickard Ewetz, Sunny Raj, Alvaro Velasquez, Laura L. Pullum, Ananthram Swami

We present a new extension of Fano's inequality and employ it to theoretically establish that the probability of success for a membership inference attack on a deep neural network can be bounded using the mutual information between its inputs and its activations.

Inference Attack Membership Inference Attack

Improving Explainability of Image Classification in Scenarios with Class Overlap: Application to COVID-19 and Pneumonia

no code implementations6 Aug 2020 Edward Verenich, Alvaro Velasquez, Nazar Khan, Faraz Hussain

Trust in predictions made by machine learning models is increased if the model generalizes well on previously unseen samples and when inference is accompanied by cogent explanations of the reasoning behind predictions.

Domain Generalization Image Classification +1

FlexServe: Deployment of PyTorch Models as Flexible REST Endpoints

no code implementations29 Feb 2020 Edward Verenich, Alvaro Velasquez, M. G. Sarwar Murshed, Faraz Hussain

The integration of artificial intelligence capabilities into modern software systems is increasingly being simplified through the use of cloud-based machine learning services and representational state transfer architecture design.

The Utility of Feature Reuse: Transfer Learning in Data-Starved Regimes

1 code implementation29 Feb 2020 Rashik Shadman, M. G. Sarwar Murshed, Edward Verenich, Alvaro Velasquez, Faraz Hussain

The use of transfer learning with deep neural networks has increasingly become widespread for deploying well-tested computer vision systems to newer domains, especially those with limited datasets.

Transfer Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.