Search Results for author:
Found 85 papers, 20 papers with code
Rule-Bottleneck Reinforcement Learning: Joint Explanation and Decision Optimization for Resource Allocation with Language Agents
Deep Reinforcement Learning (RL) is remarkably effective in addressing sequential resource allocation problems in domains such as healthcare, public policy, and resource management.
On Sequential Fault-Intolerant Process Planning
We propose and study a planning problem we call Sequential Fault-Intolerant Process Planning (SFIPP).
Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness
In this work, we study the effects on both task performance and fairness when the DLM algorithm, a recent work on using LLMs to design reward functions for RMABs, is prompted with non-English language commands.
Finite-Horizon Single-Pull Restless Bandits: An Efficient Index Policy For Scarce Resource Allocation
However, in many practical settings with highly scarce resources, where each agent can only receive at most one resource, such as healthcare intervention programs, the standard RMAB framework falls short.
IRL for Restless Multi-Armed Bandits with Applications in Maternal and Child Health
To address this shortcoming, this paper is the first to present the use of inverse reinforcement learning (IRL) to learn desired rewards for RMABs, and we demonstrate improved outcomes in a maternal and child health telehealth program.
Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impact
AI for social impact (AI4SI) offers significant potential for addressing complex societal challenges in areas such as public health, agriculture, education, conservation, and public safety.
Contrasting local and global modeling with machine learning and satellite data: A case study estimating tree canopy height in African savannas
While advances in machine learning with satellite imagery (SatML) are facilitating environmental monitoring at a global scale, developing SatML models that are accurate and useful for local regions remains critical to understanding and acting on an ever-changing planet.
Bayesian Collaborative Bandits with Thompson Sampling for Improved Outreach in Maternal Health Program
Mobile health (mHealth) programs face a critical challenge in optimizing the timing of automated health information calls to beneficiaries.
On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow
We further find that, with deep learning approximation errors, fixed points can deviate from true states and the deviation is negatively correlated to the Jacobian rank.
Optimizing Vital Sign Monitoring in Resource-Constrained Maternal Care: An RL-Based Restless Bandit Approach
Maternal mortality remains a significant global public health challenge.
Find Rhinos without Finding Rhinos: Active Learning with Multimodal Imagery of South African Rhino Habitats
Much of Earth's charismatic megafauna is endangered by human activities, particularly the rhino, which is at risk of extinction due to the poaching crisis in Africa.
What is the Right Notion of Distance between Predict-then-Optimize Tasks?
Comparing datasets is a fundamental task in machine learning, essential for various learning paradigms; from evaluating train and test datasets for model generalization to using dataset similarity for detecting data drift.
Improving the Prediction of Individual Engagement in Recommendations Using Cognitive Models
For public health programs with limited resources, the ability to predict how behaviors change over time and in response to interventions is crucial for deciding when and to whom interventions should be allocated.
Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewards
In the presence of multiple agents, altering the reward function based on human preferences can impact subpopulations very differently, leading to complex tradeoffs and a multi-objective resource allocation problem.
The Bandit Whisperer: Communication Learning for Restless Bandits
Applying Reinforcement Learning (RL) to Restless Multi-Arm Bandits (RMABs) offers a promising avenue for addressing allocation problems with resource constraints and temporal dynamics.
Combining Diverse Information for Coordinated Action: Stochastic Bandit Algorithms for Heterogeneous Agents
Stochastic multi-agent multi-armed bandits typically assume that the rewards from each arm follow a fixed distribution, regardless of which agent pulls the arm.
Transcendence: Generative Models Can Outperform The Experts That Train Them
Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on.
Application-Driven Innovation in Machine Learning
As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly important.
Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning
However, the availability and time of these health workers are limited resources.
A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health
In this paper, we propose a Decision Language Model (DLM) for RMABs, enabling dynamic fine-tuning of RMAB policies in public health settings using human-language commands.
Social Environment Design
Artificial Intelligence (AI) holds promise as a technology that can be used to improve government and economic policy-making.
Evaluating the Effectiveness of Index-Based Treatment Allocation
When resources are scarce, an allocation policy is needed to decide who receives a resource.
Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits
Public health programs often provide interventions to encourage program adherence, and effectively allocating interventions is vital for producing the greatest overall health outcomes, especially in underserved communities where resources are limited.
Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shaping
Inverse reinforcement learning (IRL) is computationally challenging, with common approaches requiring the solution of multiple reinforcement learning (RL) sub-problems.
Analyzing and Predicting Low-Listenership Trends in a Large-Scale Mobile Health Program: A Preliminary Investigation
Mobile health programs are becoming an increasingly popular medium for dissemination of health information among beneficiaries in less privileged communities.
Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization
Restless multi-arm bandits (RMABs), a class of resource allocation problems with broad application in areas such as healthcare, online advertising, and anti-poaching, have recently been studied from a multi-agent reinforcement learning perspective.
Equitable Restless Multi-Armed Bandits: A General Framework Inspired By Digital Health
RMABs are increasingly being used for sensitive decisions such as in public health, treatment scheduling, anti-poaching, and -- the motivation for this work -- digital health.
Reflections from the Workshop on AI-Assisted Decision Making for Conservation
In this white paper, we synthesize key points made during presentations and discussions from the AI-Assisted Decision Making for Conservation workshop, hosted by the Center for Research on Computation and Society at Harvard University on October 20-21, 2022.
Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize
In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions.
Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare
The success of many healthcare programs depends on participants' adherence.
Fairness for Workers Who Pull the Arms: An Index Based Policy for Allocation of Restless Bandit Tasks
The goal is to plan an intervention schedule that maximizes the expected reward while satisfying budget constraints on each worker as well as fairness in terms of the load assigned to each worker.
Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation
We consider the task of evaluating policies of algorithmic resource allocation through randomized controlled trials (RCTs).
Decision-Focused Evaluation: Analyzing Performance of Deployed Restless Multi-Arm Bandits
Restless multi-arm bandits (RMABs) is a popular decision-theoretic framework that has been used to model real-world sequential decision making problems in public health, wildlife conservation, communication systems, and beyond.
Artificial Intelligence and Life in 2030: The One Hundred Year Study on Artificial Intelligence
In September 2016, Stanford's "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the first report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society.
Indexability is Not Enough for Whittle: Improved, Near-Optimal Algorithms for Restless Bandits
Whittle index policies, which are based on Lagrangian relaxations, are widely used in these settings due to their simplicity and near-optimality under certain conditions.
Artificial Replay: A Meta-Algorithm for Harnessing Historical Data in Bandits
We show that Artificial-Replay uses only a fraction of the historical data compared to a full warm-start approach, while still achieving identical regret for base algorithms that satisfy independence of irrelevant data (IIData), a novel and broadly applicable property that we introduce.
Optimistic Whittle Index Policy: Online Learning for Restless Bandits
Restless multi-armed bandits (RMABs) extend multi-armed bandits to allow for stateful arms, where the state of each arm evolves restlessly with different transitions depending on whether that arm is pulled.
Ranked Prioritization of Groups in Combinatorial Bandit Allocation
Preventing poaching through ranger patrols protects endangered wildlife, directly contributing to the UN Sustainable Development Goal 15 of life on land.
Evolutionary Approach to Security Games with Signaling
Green Security Games have become a popular way to model scenarios involving the protection of natural resources, such as wildlife.
ADVISER: AI-Driven Vaccination Intervention Optimiser for Increasing Vaccine Uptake in Nigeria
More than 5 million children under five years die from largely preventable or treatable medical conditions every year, with an overwhelmingly large proportion of deaths occurring in under-developed countries with low vaccination uptake.
Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses
Decision-Focused Learning (DFL) is a paradigm for tailoring a predictive model to a downstream optimization task that uses its predictions in order to perform better on that specific task.
Proceedings of the Artificial Intelligence for Cyber Security (AICS) Workshop at AAAI 2022
These challenges are widely studied in enterprise networks, but there are many gaps in research and practice as well as novel problems in other domains.
Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health
To address this shortcoming, we propose a novel approach for decision-focused learning in RMAB that directly trains the predictive model to maximize the Whittle index solution quality.
Networked Restless Multi-Armed Bandits for Mobile Interventions
Motivated by a broad class of mobile intervention problems, we propose and study restless multi-armed bandits (RMABs) with network effects.
Facilitating human-wildlife cohabitation through conflict prediction
With increasing world population and expanded use of forests as cohabited regions, interactions and conflicts with wildlife are increasing, leading to large-scale loss of lives (animal and human) and livelihoods (economic).
Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health
Our second major contribution is evaluation of our RMAB system in collaboration with an NGO, via a real-world service quality improvement study.
Harnessing Heterogeneity: Learning from Decomposed Feedback in Bayesian Modeling
We introduce a novel decomposed GP regression to incorporate the subgroup decomposed feedback.
Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learning
Our approach uses a double oracle framework (oracles for \textit{agent} and \textit{nature}), which is often used for single-process robust planning but requires significant new techniques to accommodate the combinatorial nature of RMABs.
Deep Reinforcement Learning
Multi-agent Reinforcement Learning
+2
Q-Learning Lagrange Policies for Multi-Action Restless Bandits
Multi-action restless multi-armed bandits (RMABs) are a powerful framework for constrained resource allocation in which $N$ independent processes are managed.
Robust Reinforcement Learning Under Minimax Regret for Green Security
We formulate the problem as a game between the defender and nature who controls the parameter values of the adversarial behavior and design an algorithm MIRROR to find a robust policy.
Contingency-Aware Influence Maximization: A Reinforcement Learning Approach
Empirical results show that our method achieves influence as high as the state-of-the-art methods for contingency-aware IM, while having negligible runtime at test phase.
AI-driven Prices for Externalities and Sustainability in Production Markets
Traditional competitive markets do not account for negative externalities; indirect costs that some participants impose on others, such as the cost of over-appropriating a common-pool resource (which diminishes future stock, and thus harvest, for everyone).
Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Problems by Reinforcement Learning
In the predict-then-optimize framework, the objective is to train a predictive model, mapping from environment features to parameters of an optimization problem, which maximizes decision quality when the optimization is subsequently solved.
Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Making by Reinforcement Learning
In the predict-then-optimize framework, the objective is to train a predictive model, mapping from environment features to parameters of an optimization problem, which maximizes decision quality when the optimization is subsequently solved.
Learn to Intervene: An Adaptive Learning Policy for Restless Bandits in Application to Preventive Healthcare
In many public health settings, it is important for patients to adhere to health programs, such as taking medications and periodic health checks.
Efficient Algorithms for Finite Horizon and Streaming Restless Multi-Armed Bandit Problems
Our contributions are as follows: (1) We derive conditions under which our problem satisfies indexability, a precondition that guarantees the existence and asymptotic optimality of the Whittle Index solution for RMABs.
Selective Intervention Planning using Restless Multi-Armed Bandits to Improve Maternal and Child Health Outcomes
India has a maternal mortality ratio of 113 and child mortality ratio of 2830 per 100, 000 live births.
Active Screening for Recurrent Diseases: A Reinforcement Learning Approach
However, given the limited number of health workers, only a small subset of the population can be visited in any given time period.
Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty
We therefore first propose a novel GSG model that combines defender allocation, patrolling, real-time drone notification to human patrollers, and drones sending warning signals to attackers.
Decision Making
Multiagent Systems
Efficient Reservoir Management through Deep Reinforcement Learning
Dams impact downstream river dynamics through flow regulation and disruption of upstream-downstream linkages.
Collapsing Bandits and Their Application to Public Health Intervention
Our main contributions are as follows: (i) Building on the Whittle index technique for RMABs, we derive conditions under which the Collapsing Bandits problem is indexable.
Enhancing Poaching Predictions for Under-Resourced Wildlife Conservation Parks Using Remote Sensing Imagery
To ensure under-resourced parks have access to meaningful poaching predictions, we introduce the use of publicly available remote sensing data to extract features for parks.
Measuring Data Collection Diligence for Community Healthcare
In this work, we define and test a data collection diligence score.
Dual-Mandate Patrols: Multi-Armed Bandits for Green Security
Conservation efforts in green security domains to protect wildlife and forests are constrained by the limited availability of defenders (i. e., patrollers), who must patrol vast areas to protect from attackers (e. g., poachers or illegal loggers).
Tracking disease outbreaks from sparse data with Bayesian inference
For example, case counts may be sparse when only a small fraction of infections are caught by a testing program.
Collapsing Bandits and Their Application to Public Health Interventions
(ii) We exploit the optimality of threshold policies to build fast algorithms for computing the Whittle index, including a closed-form.
Automatically Learning Compact Quality-aware Surrogates for Optimization Problems
Solving optimization problems with unknown parameters often requires learning a predictive model to predict the values of the unknown parameters and then solving the problem using these values.
Fair Influence Maximization: A Welfare Optimization Approach
Under this framework, the trade-off between fairness and efficiency can be controlled by a single inequality aversion design parameter.
Missed calls, Automated Calls and Health Support: Using AI to improve maternal health outcomes by increasing program engagement
India accounts for 11% of maternal deaths globally where a woman dies in childbirth every fifteen minutes.
AI for Social Impact: Learning and Planning in the Data-to-Deployment Pipeline
With the maturing of AI and multiagent systems research, we have a tremendous opportunity to direct these advances towards addressing complex societal problems.
Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learning
To solve the online problem with a hard bound on risk, we formulate it as a Reinforcement Learning (RL) problem with constraints on the action space (hard bound on risk).
MIPaaL: Mixed Integer Program as a Layer
It has been successfully applied to several limited combinatorial problem classes, such as those that can be expressed as linear programs (LP), and submodular optimization.
Influence maximization in unknown social networks: Learning Policies for Effective Graph Sampling
A serious challenge when finding influential actors in real-world social networks is the lack of knowledge about the structure of the underlying network.
End to end learning and optimization on graphs
However, graphs or related attributes are often only partially observed, introducing learning problems such as link prediction which must be solved prior to optimization.
Stay Ahead of Poachers: Illegal Wildlife Poaching Prediction and Patrol Planning Under Uncertainty with Field Test Evaluations
We evaluate our approach on real-world historical poaching data from Murchison Falls and Queen Elizabeth National Parks in Uganda and, for the first time, Srepok Wildlife Sanctuary in Cambodia.
Group-Fairness in Influence Maximization
Influence maximization is a widely used model for information dissemination in social networks.
Computer Science and Game Theory Social and Information Networks
End-to-End Game-Focused Learning of Adversary Behavior in Security Games
Stackelberg security games are a critical tool for maximizing the utility of limited defense resources to protect important targets from an intelligent adversary.
Learning to Prescribe Interventions for Tuberculosis Patients Using Digital Adherence Data
Digital Adherence Technologies (DATs) are an increasingly popular method for verifying patient adherence to many medications.
Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization
These components are typically approached separately: a machine learning model is first trained via a measure of predictive accuracy, and then its predictions are used as input into an optimization algorithm which produces a decision.
Mitigating the Curse of Correlation in Security Games by Entropy Maximization
To mitigate this issue, we propose to design entropy-maximizing defending strategies for spatio-temporal security games, which frequently suffer from CoC.
Pilot Testing an Artificial Intelligence Algorithm That Selects Homeless Youth Peer Leaders Who Promote HIV Testing
To pilot test an artificial intelligence (AI) algorithm that selects peer change agents (PCA) to disseminate HIV testing messaging in a population of homeless youth.
Using Social Networks to Aid Homeless Shelters: Dynamic Influence Maximization under Uncertainty - An Extended Version
This paper presents HEALER, a software agent that recommends sequential intervention plans for use by homeless shelters, who organize these interventions to raise awareness about HIV among homeless youth.
Learning Adversary Behavior in Security Games: A PAC Model Perspective
We provide four main contributions: (1) a PAC model of learning adversary response functions in SSGs; (2) PAC-model analysis of the learning of key, existing bounded rationality models in SSGs; (3) an entirely new approach to adversary modeling based on a non-parametric class of response functions with PAC-model analysis and (4) identification of conditions under which computing the best defender strategy against the learned adversary behavior is indeed the optimal strategy.
Security Games with Information Leakage: Modeling and Computation
Our experiments confirm the necessity of handling information leakage and the advantage of our algorithms.
Diverse Randomized Agents Vote to Win
We investigate the power of voting among diverse, randomized software agents.
