Search Results for author:
Found 104 papers, 19 papers with code
Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning
We study a heterogeneous agent macroeconomic model with an infinite number of households and firms competing in a labor market.
A Unifying Perspective on Multi-Calibration: Unleashing Game Dynamics for Multi-Objective Learning
We provide a unifying framework for the design and analysis of multi-calibrated and moment-multi-calibrated predictors.
Deterministic Nonsmooth Nonconvex Optimization
In particular, we prove a lower bound of $\Omega(d)$ for any deterministic algorithm.
Accelerated First-Order Optimization under Nonlinear Constraints
We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization.
Online Learning in Stackelberg Games with an Omniscient Follower
We analyze the sample complexity of regret minimization in this repeated Stackelberg game.
Principled Reinforcement Learning with Human Feedback from Pairwise or $K$-wise Comparisons
Our analysis shows that when the true reward function is linear, the widely used maximum likelihood estimator (MLE) converges under both the Bradley-Terry-Luce (BTL) model and the Plackett-Luce (PL) model.
Prediction-Powered Inference
We introduce prediction-powered inference $\unicode{x2013}$ a framework for performing valid statistical inference when an experimental data set is supplemented with predictions from a machine-learning system.
Incentive-Aware Recommender Systems in Two-Sided Markets
Online platforms in the Internet Economy commonly incorporate recommender systems that recommend arms (e. g., products) to agents (e. g., users).
The Sample Complexity of Online Contract Design
This result shows that exponential-in-$m$ samples are sufficient and necessary to learn a near-optimal contract, resolving an open problem on the hardness of online contract design.
Nesterov Meets Optimism: Rate-Optimal Optimistic-Gradient-Based Method for Stochastic Bilinearly-Coupled Minimax Optimization
We provide a novel first-order optimization algorithm for bilinearly-coupled strongly-convex-concave minimax optimization called the AcceleratedGradient OptimisticGradient (AG-OG).
A Primal-dual Approach for Solving Variational Inequalities with General-form Constraints
We prove its convergence and show that the gap function of the last iterate of this inexact-ACVI method decreases at a rate of $\mathcal{O}(\frac{1}{\sqrt{K}})$ when the operator is $L$-Lipschitz and monotone, provided that the errors decrease at appropriate rates.
Explicit Second-Order Min-Max Optimization Methods with Optimal Convergence Guarantee
We propose and analyze exact and inexact regularized Newton-type methods for finding a global saddle point of a \textit{convex-concave} unconstrained min-max optimization problem.
On-Demand Sampling: Learning Optimally from Multiple Distributions
Social and real-world considerations such as robustness, fairness, social welfare and multi-agent tradeoffs have given rise to multi-distribution learning paradigms, such as collaborative, group distributionally robust, and fair federated learning.
A Reinforcement Learning Approach in Multi-Phase Second-Price Auction Design
First, from the seller's perspective, we need to efficiently explore the environment in the presence of potentially nontruthful bidders who aim to manipulates seller's policy.
QuTE: decentralized multiple testing on sensor networks with false discovery rate control
We consider the setting where distinct agents reside on the nodes of an undirected graph, and each agent possesses p-values corresponding to one or more hypotheses local to its node.
A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning
With the increasing need for handling large state and action spaces, general function approximation has become a key technique in reinforcement learning (RL).
Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization
Nonsmooth nonconvex optimization problems broadly emerge in machine learning and business decision making, whereas two core challenges impede the development of efficient solution methods with finite-time convergence guarantee: the lack of computationally tractable optimality criterion and the lack of computationally powerful oracles.
Competition, Alignment, and Equilibria in Digital Marketplaces
Nonetheless, the data sharing assumptions impact what mechanism drives misalignment and also affect the specific form of misalignment (e. g. the quality of the best-case and worst-case market outcomes).
Empirical Gateaux Derivatives for Causal Inference
We then derive requirements on the rates of numerical approximation in perturbation and smoothing that preserve the statistical benefits of one-step adjustments, such as rate-double-robustness.
Valid Inference after Causal Discovery
Causal discovery and causal effect estimation are two fundamental tasks in causal inference.
Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium
We consider learning Nash equilibria in two-player zero-sum Markov Games with nonlinear function approximation, where the action-value function is approximated by a function in a Reproducing Kernel Hilbert Space (RKHS).
Continuous-time Analysis for Variational Inequalities: An Overview and Desiderata
Algorithms that solve zero-sum games, multi-objective agent objectives, or, more generally, variational inequality (VI) problems are notoriously unstable on general problems.
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels
Leveraging this observation, we propose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first, learn features using off-the-shelf methods (e. g., FedAvg); then, optimize a convexified problem obtained from the network's empirical neural tangent kernel approximation.
Mechanisms that Incentivize Data Sharing in Federated Learning
Federated learning is typically considered a beneficial technology which allows multiple agents to collaborate with each other, improve the accuracy of their models, and solve problems which are otherwise too data-intensive / expensive to be solved individually.
Recommendation Systems with Distribution-Free Reliability Guarantees
Building from a pre-trained ranking model, we show how to return a set of items that is rigorously guaranteed to contain mostly good items.
Breaking Feedback Loops in Recommender Systems with Causal Inference
Our main observation is that a recommender system does not suffer from feedback loops if it reasons about causal quantities, namely the intervention distributions of recommendations on user ratings.
NumS: Scalable Array Programming for the Cloud
LSHS is a local search method which optimizes operator placement by minimizing maximum memory and network load on any given node within a distributed system.
Modeling Content Creator Incentives on Algorithm-Curated Platforms
We prove that seemingly innocuous algorithmic choices -- e. g., non-negative vs. unconstrained factorization -- significantly affect the existence and character of (Nash) equilibria in exposure games.
Solving Constrained Variational Inequalities via a First-order Interior Point-based Method
We provide convergence guarantees for ACVI in two general classes of problems: (i) when the operator is $\xi$-monotone, and (ii) when it is monotone, some constraints are active and the game is not purely rotational.
Optimal Extragradient-Based Bilinearly-Coupled Saddle-Point Optimization
We consider the smooth convex-concave bilinearly-coupled saddle-point problem, $\min_{\mathbf{x}}\max_{\mathbf{y}}~F(\mathbf{x}) + H(\mathbf{x},\mathbf{y}) - G(\mathbf{y})$, where one has access to stochastic first-order oracles for $F$, $G$ as well as the bilinear coupling function $H$.
Robust Calibration with Multi-domain Temperature Scaling
Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains.
First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces
From optimal transport to robust dimensionality reduction, a plethora of machine learning applications can be cast into the min-max optimization problems over Riemannian manifolds.
Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees
In contrast to prior work, our proposed protocols improve the dimension dependence and achieve a tight statistical rate in terms of all the parameters for strongly convex losses.
Online Nonsubmodular Minimization with Delayed Costs: From Full Information to Bandit Feedback
Motivated by applications to online learning in sparse estimation and Bayesian optimization, we consider the problem of online unconstrained nonsubmodular minimization with delayed costs in both full information and bandit feedback settings.
Principal-Agent Hypothesis Testing
The pharmaceutical company wishes to sell a product to make a profit, and the FDA wishes to ensure that only efficacious drugs are released to the public.
First-Order Algorithms for Nonlinear Generalized Nash Equilibrium Problems
We consider the problem of computing an equilibrium in a class of \textit{nonlinear generalized Nash equilibrium problems (NGNEPs)} in which the strategy sets for each player are defined by equality and inequality constraints that may depend on the choices of rival players.
Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making.
Learn to Match with No Regret: Reinforcement Learning in Markov Matching Markets
We study a Markov matching market involving a planner and a set of strategic agents on the two sides of the market.
Learning Dynamic Mechanisms in Unknown Environments: A Reinforcement Learning Approach
Dynamic mechanism design studies how mechanism designers should allocate resources among agents in a time-varying environment.
Off-Policy Evaluation with Policy-Dependent Optimization Response
Under this framework, a decision-maker's utility depends on the policy-dependent optimization, which introduces a fundamental challenge of \textit{optimization} bias even for the case of policy evaluation.
Partial Identification with Noisy Covariates: A Robust Optimization Approach
Directly adjusting for these imperfect measurements of the covariates can lead to biased causal estimates.
Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning
This paper proposes a novel model of sequential information design, namely the Markov persuasion processes (MPPs), where a sender, with informational advantage, seeks to persuade a stream of myopic receivers to take actions that maximizes the sender's cumulative utilities in a finite horizon Markovian environment with varying prior and utility functions.
No-Regret Learning in Partially-Informed Auctions
We formalize this problem as an online learning task where the goal is to have low regret with respect to a myopic oracle that has perfect knowledge of the distribution over items and the seller's masking function.
Improving Generalization via Uncertainty Driven Perturbations
We show that UDP is guaranteed to achieve the maximum margin decision boundary on linear models and that it notably increases it on challenging simulated datasets.
Transferred Q-learning
We consider $Q$-learning with knowledge transfer, using samples from a target reinforcement learning (RL) task as well as source samples from different but related RL tasks.
Conformal prediction for the design problem
This is challenging because of a characteristic type of distribution shift between the training and test data in the design setting -- one in which the training and test data are statistically dependent, as the latter is chosen based on the former.
Robust Estimation for Nonparametric Families via Generative Adversarial Networks
Prior work focus on the problem of robust mean and covariance estimation when the true distribution lies in the family of Gaussian distributions or elliptical distributions, and analyze depth or scoring rule based GAN losses for the problem.
Reinforcement Learning with Heterogeneous Data: Estimation and Inference
Reinforcement Learning (RL) has the promise of providing data-driven support for decision-making in a wide range of problems in healthcare, education, business, and other domains.
Dynamic Thresholding for Online Distributed Data Selection
The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets.
Optimal variance-reduced stochastic approximation in Banach spaces
We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space.
Instance-Dependent Confidence and Early Stopping for Reinforcement Learning
As a consequence, we propose a data-dependent stopping rule for instance-optimal algorithms.
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning
We study Q-learning with Polyak-Ruppert averaging in a discounted Markov decision process in synchronous and tabular settings.
Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems
Motivated by the problem of online canonical correlation analysis, we propose the \emph{Stochastic Scaled-Gradient Descent} (SSGD) algorithm for minimizing the expectation of a stochastic function over a generic Riemannian manifold.
Wasserstein Flow Meets Replicator Dynamics: A Mean-Field Analysis of Representation Learning in Actor-Critic
Specifically, we consider a version of AC where the actor and critic are represented by overparameterized two-layer neural networks and are updated with two-timescale learning rates.
Last-Iterate Convergence of Saddle Point Optimizers via High-Resolution Differential Equations
However, the convergence properties of these methods are qualitatively different even on simple bilinear games.
Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?
We develop sample-efficient reinforcement learning (RL) algorithms for solving for an SNE in both online and offline settings.
Meta-Analysis of Randomized Experiments with Applications to Heavy-Tailed Response Data
The key insight of our procedure is that the noisy (but unbiased) difference-of-means estimate can be used as a ground truth ``label" on a portion of the RCT, to test the performance of an estimator trained on the other portion.
ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning
In this paper, we present a scalable and elastic library ElegantRL-podracer for cloud-native deep reinforcement learning, which efficiently supports millions of GPU cores to carry out massively parallel training at multiple levels.
An Instance-Dependent Analysis for the Cooperative Multi-Player Multi-Armed Bandit
We study the problem of information sharing and cooperation in Multi-Player Multi-Armed bandits.
Cluster-and-Conquer: A Framework For Time-Series Forecasting
Our framework -- which we refer to as "cluster-and-conquer" -- is highly general, allowing for any time-series forecasting and clustering method to be used in each step.
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
(2) The best ranked PTM can either be fine-tuned and deployed if we have no preference for the model's architecture or the target PTM can be tuned by the top $K$ ranked PTMs via a Bayesian procedure that we propose.
On the Self-Penalization Phenomenon in Feature Selection
We describe an implicit sparsity-inducing mechanism based on minimization over a family of kernels: \begin{equation*} \min_{\beta, f}~\widehat{\mathbb{E}}[L(Y, f(\beta^{1/q} \odot X)] + \lambda_n \|f\|_{\mathcal{H}_q}^2~~\text{subject to}~~\beta \ge 0, \end{equation*} where $L$ is the loss, $\odot$ is coordinate-wise multiplication and $\mathcal{H}_q$ is the reproducing kernel Hilbert space based on the kernel $k_q(x, x') = h(\|x-x'\|_q^q)$, where $\|\cdot\|_q$ is the $\ell_q$ norm.
Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees.
SOUL: An Energy-Efficient Unsupervised Online Learning Seizure Detection Classifier
Implantable devices that record neural activity and detect seizures have been adopted to issue warnings or trigger neurostimulation to suppress epileptic seizures.
Desiderata for Representation Learning: A Causal Perspective
Representation learning constructs low-dimensional representations to summarize essential features of high-dimensional data.
Learning Equilibria in Matching Markets from Bandit Feedback
Large-scale, two-sided matching platforms must find market outcomes that align with user preferences while simultaneously learning these preferences from data.
Optimization on manifolds: A symplectic approach
There has been great interest in using tools from dynamical systems and numerical analysis of differential equations to understand and construct new optimization methods.
Data Sharing Markets
We model bilateral sharing as a network formation game and show the existence of strongly stable outcome under the top agents property by allowing limited complementarity.
On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems
We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems.
Robust Learning of Optimal Auctions
The proposed algorithms operate beyond the setting of bounded distributions that have been studied in prior works, and are guaranteed to obtain a fraction $1-O(\alpha)$ of the optimal revenue under the true distribution when the distributions are MHR.
Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics
Bayesian models based on the Dirichlet process and other stick-breaking priors have been proposed as core ingredients for clustering, topic modeling, and other unsupervised learning tasks.
On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averaging
We study the stochastic bilinear minimax optimization problem, presenting an analysis of the same-sample Stochastic ExtraGradient (SEG) method with constant step size, and presenting variations of the method that yield favorable convergence.
Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence
We thus propose a novel combination of optimization and sampling techniques for approximate Bayesian inference by constructing an IS proposal distribution through the minimization of a forward KL (FKL) divergence.
On component interactions in two-stage recommender systems
Thanks to their scalability, two-stage recommenders are used by many of today's largest online platforms, including YouTube, LinkedIn, and Pinterest.
Instance-optimality in optimal value estimation: Adaptivity via variance-reduced Q-learning
Various algorithms in reinforcement learning exhibit dramatic variability in their convergence rates and ultimate accuracy as a function of the problem structure.
The Stereotyping Problem in Collaboratively Filtered Recommender Systems
First, we introduce a notion of joint accessibility, which measures the extent to which a set of items can jointly be accessed by users.
Who Leads and Who Follows in Strategic Classification?
In particular, by generalizing the standard model to allow both players to learn over time, we show that a decision-maker that makes updates faster than the agents can reverse the order of play, meaning that the agents lead and the decision-maker follows.
Test-time Collective Prediction
An increasingly common setting in machine learning involves multiple parties, each with their own data, who want to jointly make predictions on future test points.
Taming Nonconvexity in Kernel Feature Selection -- Favorable Properties of the Laplace Kernel
Kernel-based feature selection is an important tool in nonparametric statistics.
Learning Competitive Equilibria in Exchange Economies with Bandit Feedback
The allocations at a CE are Pareto efficient and fair.
PAC Best Arm Identification Under a Deadline
In this work, the decision-maker is given a deadline of $T$ rounds, where, on each round, it can adaptively choose which arms to pull and how many times to pull them; this distinguishes the number of decisions made (i. e., time or number of rounds) from the number of samples acquired (cost).
On the Theory of Reinforcement Learning with Once-per-Episode Feedback
We study a theory of reinforcement learning (RL) in which the learner receives binary feedback only once at the end of an episode.
Parallelizing Contextual Bandits
Standard approaches to decision-making under uncertainty focus on sequential exploration of the space of decisions.
Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization
Distributionally robust supervised learning (DRSL) is emerging as a key paradigm for building reliable machine learning systems for real-world applications -- reflecting the need for classifiers and predictive models that are robust to the distribution shifts that arise from phenomena such as selection bias or nonstationarity.
Multi-Source Causal Inference Using Control Variates
Across simulations and two case studies with real data, we show that this control variate can significantly reduce the variance of the ATE estimate.
On the Stability of Nonlinear Receding Horizon Control: A Geometric Perspective
The widespread adoption of nonlinear Receding Horizon Control (RHC) strategies by industry has led to more than 30 years of intense research efforts to provide stability guarantees for these methods.
A Variational Inequality Approach to Bayesian Regression Games
Bayesian regression games are a special class of two-player general-sum Bayesian games in which the learner is partially informed about the adversary's objective through a Bayesian prior.
Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
Collecting more diverse and representative training data is often touted as a remedy for the disparate performance of machine learning predictors across subpopulations.
Learning in Multi-Stage Decentralized Matching Markets
Matching markets are often organized in a multi-stage and decentralized manner.
Private Prediction Sets
Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine.
Tactical Optimism and Pessimism for Deep Reinforcement Learning
In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control.
Distribution-Free, Risk-Controlling Prediction Sets
While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making.
Stochastic Approximation for Online Tensorial Independent Component Analysis
For estimating one component, we provide a dynamics-based analysis to prove that our online tensorial ICA algorithm with a specific choice of stepsize achieves a sharp finite-sample error bound.
Bandit Learning in Decentralized Matching Markets
We study two-sided matching markets in which one side of the market (the players) does not have a priori knowledge about its preferences for the other side (the arms) and is required to learn its preferences from experience.
Optimal Mean Estimation without a Variance
Concretely, given a sample $\mathbf{X} = \{X_i\}_{i = 1}^n$ from a distribution $\mathcal{D}$ over $\mathbb{R}^d$ with mean $\mu$ which satisfies the following \emph{weak-moment} assumption for some ${\alpha \in [0, 1]}$: \begin{equation*} \forall \|v\| = 1: \mathbb{E}_{X \thicksim \mathcal{D}}[\lvert \langle X - \mu, v\rangle \rvert^{1 + \alpha}] \leq 1, \end{equation*} and given a target failure probability, $\delta$, our goal is to design an estimator which attains the smallest possible confidence interval as a function of $n, d,\delta$.
On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
The classical theory of reinforcement learning (RL) has focused on tabular and linear representations of value functions.
Do Offline Metrics Predict Online Performance in Recommender Systems?
We observe that offline metrics are correlated with online performance over a range of environments.
Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization
The use of min-max optimization in adversarial training of deep neural network classifiers and training of generative adversarial networks has motivated the study of nonconvex-nonconcave optimization objectives, which frequently arise in these applications.
Resource Allocation in Multi-armed Bandit Exploration: Overcoming Sublinear Scaling with Adaptive Parallelism
Second, we present an algorithm for a fixed deadline setting, where we are given a time deadline and need to maximize the probability of finding the best arm.
Learning Strategies in Decentralized Matching Markets under Uncertain Preferences
We study the problem of decision-making in the setting of a scarcity of shared resources when the preferences of agents are unknown a priori and must be learned from data.
Behavior-Guided Reinforcement Learning
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space.
Posterior Distribution for the Number of Clusters in Dirichlet Process Mixture Models
In this work, we provide a rigorous study for the posterior distribution of the number of clusters in DPMM under different prior distributions on the parameters and constraints on the distributions of the data.
Bayesian Nonparametric Inference of Switching Linear Dynamical Systems
Many complex dynamical phenomena can be effectively modeled by a system that switches among a set of conditionally linear dynamical modes.
Latent Dirichlet Allocation
Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities.
