Search Results for author:
Found 55 papers, 15 papers with code
Fairness for Workers Who Pull the Arms: An Index Based Policy for Allocation of Restless Bandit Tasks
The goal is to plan an intervention schedule that maximizes the expected reward while satisfying budget constraints on each worker as well as fairness in terms of the load assigned to each worker.
Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation
We consider the task of evaluating policies of algorithmic resource allocation through randomized controlled trials (RCTs).
Decision-Focused Evaluation: Analyzing Performance of Deployed Restless Multi-Arm Bandits
Restless multi-arm bandits (RMABs) is a popular decision-theoretic framework that has been used to model real-world sequential decision making problems in public health, wildlife conservation, communication systems, and beyond.
Artificial Intelligence and Life in 2030: The One Hundred Year Study on Artificial Intelligence
In September 2016, Stanford's "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the first report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society.
Indexability is Not Enough for Whittle: Improved, Near-Optimal Algorithms for Restless Bandits
Whittle index policies, which are based on Lagrangian relaxations, are widely used in these settings due to their simplicity and near-optimality under certain conditions.
Artificial Replay: A Meta-Algorithm for Harnessing Historical Data in Bandits
How best to incorporate historical data to "warm start" bandit algorithms is an open question: naively initializing reward estimates using all historical samples can suffer from spurious data and imbalanced data coverage, leading to computational and storage issues $\unicode{x2014}$ particularly salient in continuous action spaces.
Optimistic Whittle Index Policy: Online Learning for Restless Bandits
Restless multi-armed bandits (RMABs) extend multi-armed bandits to allow for stateful arms, where the state of each arm evolves restlessly with different transitions depending on whether that arm is pulled.
Ranked Prioritization of Groups in Combinatorial Bandit Allocation
Preventing poaching through ranger patrols protects endangered wildlife, directly contributing to the UN Sustainable Development Goal 15 of life on land.
Evolutionary Approach to Security Games with Signaling
Green Security Games have become a popular way to model scenarios involving the protection of natural resources, such as wildlife.
ADVISER: AI-Driven Vaccination Intervention Optimiser for Increasing Vaccine Uptake in Nigeria
More than 5 million children under five years die from largely preventable or treatable medical conditions every year, with an overwhelmingly large proportion of deaths occurring in under-developed countries with low vaccination uptake.
Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses
Decision-Focused Learning (DFL) is a paradigm for tailoring a predictive model to a downstream optimization task that uses its predictions in order to perform better on that specific task.
Proceedings of the Artificial Intelligence for Cyber Security (AICS) Workshop at AAAI 2022
These challenges are widely studied in enterprise networks, but there are many gaps in research and practice as well as novel problems in other domains.
Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health
To address this shortcoming, we propose a novel approach for decision-focused learning in RMAB that directly trains the predictive model to maximize the Whittle index solution quality.
Networked Restless Multi-Armed Bandits for Mobile Interventions
Motivated by a broad class of mobile intervention problems, we propose and study restless multi-armed bandits (RMABs) with network effects.
Facilitating human-wildlife cohabitation through conflict prediction
With increasing world population and expanded use of forests as cohabited regions, interactions and conflicts with wildlife are increasing, leading to large-scale loss of lives (animal and human) and livelihoods (economic).
Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health
Our second major contribution is evaluation of our RMAB system in collaboration with an NGO, via a real-world service quality improvement study.
Harnessing Heterogeneity: Learning from Decomposed Feedback in Bayesian Modeling
We introduce a novel decomposed GP regression to incorporate the subgroup decomposed feedback.
Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learning
Our approach uses a double oracle framework (oracles for \textit{agent} and \textit{nature}), which is often used for single-process robust planning but requires significant new techniques to accommodate the combinatorial nature of RMABs.
Q-Learning Lagrange Policies for Multi-Action Restless Bandits
Multi-action restless multi-armed bandits (RMABs) are a powerful framework for constrained resource allocation in which $N$ independent processes are managed.
Robust Reinforcement Learning Under Minimax Regret for Green Security
We formulate the problem as a game between the defender and nature who controls the parameter values of the adversarial behavior and design an algorithm MIRROR to find a robust policy.
Contingency-Aware Influence Maximization: A Reinforcement Learning Approach
Empirical results show that our method achieves influence as high as the state-of-the-art methods for contingency-aware IM, while having negligible runtime at test phase.
AI-driven Prices for Externalities and Sustainability in Production Markets
Traditional competitive markets do not account for negative externalities; indirect costs that some participants impose on others, such as the cost of over-appropriating a common-pool resource (which diminishes future stock, and thus harvest, for everyone).
Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Problems by Reinforcement Learning
In the predict-then-optimize framework, the objective is to train a predictive model, mapping from environment features to parameters of an optimization problem, which maximizes decision quality when the optimization is subsequently solved.
Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Making by Reinforcement Learning
In the predict-then-optimize framework, the objective is to train a predictive model, mapping from environment features to parameters of an optimization problem, which maximizes decision quality when the optimization is subsequently solved.
Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare
In many public health settings, it is important for patients to adhere to health programs, such as taking medications and periodic health checks.
Efficient Algorithms for Finite Horizon and Streaming Restless Multi-Armed Bandit Problems
Our contributions are as follows: (1) We derive conditions under which our problem satisfies indexability, a precondition that guarantees the existence and asymptotic optimality of the Whittle Index solution for RMABs.
Selective Intervention Planning using Restless Multi-Armed Bandits to Improve Maternal and Child Health Outcomes
India has a maternal mortality ratio of 113 and child mortality ratio of 2830 per 100, 000 live births.
Active Screening for Recurrent Diseases: A Reinforcement Learning Approach
However, given the limited number of health workers, only a small subset of the population can be visited in any given time period.
Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty
We therefore first propose a novel GSG model that combines defender allocation, patrolling, real-time drone notification to human patrollers, and drones sending warning signals to attackers.
Decision Making
Multiagent Systems
Efficient Reservoir Management through Deep Reinforcement Learning
Dams impact downstream river dynamics through flow regulation and disruption of upstream-downstream linkages.
Collapsing Bandits and Their Application to Public Health Intervention
Our main contributions are as follows: (i) Building on the Whittle index technique for RMABs, we derive conditions under which the Collapsing Bandits problem is indexable.
Enhancing Poaching Predictions for Under-Resourced Wildlife Conservation Parks Using Remote Sensing Imagery
To ensure under-resourced parks have access to meaningful poaching predictions, we introduce the use of publicly available remote sensing data to extract features for parks.
Measuring Data Collection Diligence for Community Healthcare
In this work, we define and test a data collection diligence score.
Dual-Mandate Patrols: Multi-Armed Bandits for Green Security
Conservation efforts in green security domains to protect wildlife and forests are constrained by the limited availability of defenders (i. e., patrollers), who must patrol vast areas to protect from attackers (e. g., poachers or illegal loggers).
Tracking disease outbreaks from sparse data with Bayesian inference
For example, case counts may be sparse when only a small fraction of infections are caught by a testing program.
Collapsing Bandits and Their Application to Public Health Interventions
(ii) We exploit the optimality of threshold policies to build fast algorithms for computing the Whittle index, including a closed-form.
Automatically Learning Compact Quality-aware Surrogates for Optimization Problems
Solving optimization problems with unknown parameters often requires learning a predictive model to predict the values of the unknown parameters and then solving the problem using these values.
Fair Influence Maximization: A Welfare Optimization Approach
Under this framework, the trade-off between fairness and efficiency can be controlled by a single inequality aversion design parameter.
Missed calls, Automated Calls and Health Support: Using AI to improve maternal health outcomes by increasing program engagement
India accounts for 11% of maternal deaths globally where a woman dies in childbirth every fifteen minutes.
AI for Social Impact: Learning and Planning in the Data-to-Deployment Pipeline
With the maturing of AI and multiagent systems research, we have a tremendous opportunity to direct these advances towards addressing complex societal problems.
Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learning
To solve the online problem with a hard bound on risk, we formulate it as a Reinforcement Learning (RL) problem with constraints on the action space (hard bound on risk).
MIPaaL: Mixed Integer Program as a Layer
It has been successfully applied to several limited combinatorial problem classes, such as those that can be expressed as linear programs (LP), and submodular optimization.
Influence maximization in unknown social networks: Learning Policies for Effective Graph Sampling
A serious challenge when finding influential actors in real-world social networks is the lack of knowledge about the structure of the underlying network.
End to end learning and optimization on graphs
However, graphs or related attributes are often only partially observed, introducing learning problems such as link prediction which must be solved prior to optimization.
Stay Ahead of Poachers: Illegal Wildlife Poaching Prediction and Patrol Planning Under Uncertainty with Field Test Evaluations
We evaluate our approach on real-world historical poaching data from Murchison Falls and Queen Elizabeth National Parks in Uganda and, for the first time, Srepok Wildlife Sanctuary in Cambodia.
Group-Fairness in Influence Maximization
Influence maximization is a widely used model for information dissemination in social networks.
Computer Science and Game Theory Social and Information Networks
End-to-End Game-Focused Learning of Adversary Behavior in Security Games
Stackelberg security games are a critical tool for maximizing the utility of limited defense resources to protect important targets from an intelligent adversary.
Learning to Prescribe Interventions for Tuberculosis Patients Using Digital Adherence Data
Digital Adherence Technologies (DATs) are an increasingly popular method for verifying patient adherence to many medications.
Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization
These components are typically approached separately: a machine learning model is first trained via a measure of predictive accuracy, and then its predictions are used as input into an optimization algorithm which produces a decision.
Mitigating the Curse of Correlation in Security Games by Entropy Maximization
To mitigate this issue, we propose to design entropy-maximizing defending strategies for spatio-temporal security games, which frequently suffer from CoC.
Pilot Testing an Artificial Intelligence Algorithm That Selects Homeless Youth Peer Leaders Who Promote HIV Testing
To pilot test an artificial intelligence (AI) algorithm that selects peer change agents (PCA) to disseminate HIV testing messaging in a population of homeless youth.
Using Social Networks to Aid Homeless Shelters: Dynamic Influence Maximization under Uncertainty - An Extended Version
This paper presents HEALER, a software agent that recommends sequential intervention plans for use by homeless shelters, who organize these interventions to raise awareness about HIV among homeless youth.
Learning Adversary Behavior in Security Games: A PAC Model Perspective
We provide four main contributions: (1) a PAC model of learning adversary response functions in SSGs; (2) PAC-model analysis of the learning of key, existing bounded rationality models in SSGs; (3) an entirely new approach to adversary modeling based on a non-parametric class of response functions with PAC-model analysis and (4) identification of conditions under which computing the best defender strategy against the learned adversary behavior is indeed the optimal strategy.
Security Games with Information Leakage: Modeling and Computation
Our experiments confirm the necessity of handling information leakage and the advantage of our algorithms.
Diverse Randomized Agents Vote to Win
We investigate the power of voting among diverse, randomized software agents.
