Search Results for author: Andreas Krause

Found 256 papers, 102 papers with code

Residual Deep Gaussian Processes on Manifolds

1 code implementation31 Oct 2024 Kacper Wyrwal, Andreas Krause, Viacheslav Borovitskiy

We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks.

Bayesian Optimisation Gaussian Processes

ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning

no code implementations12 Oct 2024 Yarden As, Bhavya Sukhija, Lenart Treven, Carmelo Sferrazza, Stelian Coros, Andreas Krause

Under regularity assumptions on the constraints and dynamics, we show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time.

Efficient Exploration reinforcement-learning +3

Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs

1 code implementation10 Oct 2024 Jonas Hübotter, Sascha Bongni, Ido Hakimi, Andreas Krause

To address this, we introduce SIFT, a data selection algorithm designed to reduce uncertainty about the model's response given a prompt, which unifies ideas from retrieval and active learning.

Active Learning Language Modelling +1

Amortized SHAP values via sparse Fourier function approximation

no code implementations8 Oct 2024 Ali Gorji, Andisheh Amrollahi, Andreas Krause

Our algorithm's first step harnesses recent results showing that many real-world predictors have a spectral bias that allows us to either exactly represent (in the case of ensembles of decision trees), or efficiently approximate them (in the case of neural networks) using a compact Fourier representation.

Active Fine-Tuning of Generalist Policies

no code implementations7 Oct 2024 Marco Bagatella, Jonas Hübotter, Georg Martius, Andreas Krause

We study this multi-task problem and explore an interactive framework in which the agent adaptively selects the tasks to be demonstrated.

Imitation Learning

Optimistic Games for Combinatorial Bayesian Optimization with Application to Protein Design

no code implementations27 Sep 2024 Melis Ilayda Bal, Pier Giuseppe Sessa, Mojmir Mutny, Andreas Krause

Crucially, this allows us to efficiently break down the complexity of the combinatorial domain into individual decision sets, making $\textbf{GameOpt}$ scalable to large combinatorial spaces.

Bayesian Optimization Drug Discovery +2

Towards safe and tractable Gaussian process-based MPC: Efficient sampling within a sequential quadratic programming framework

1 code implementation13 Sep 2024 Manish Prajapat, Amon Lahr, Johannes Köhler, Andreas Krause, Melanie N. Zeilinger

Learning uncertain dynamics models using Gaussian process~(GP) regression has been demonstrated to enable high-performance and safety-aware control strategies for challenging real-world applications.

Model Predictive Control

Directed Exploration in Reinforcement Learning from Linear Temporal Logic

no code implementations18 Aug 2024 Marco Bagatella, Andreas Krause, Georg Martius

Linear temporal logic (LTL) is a powerful language for task specification in reinforcement learning, as it allows describing objectives beyond the expressivity of conventional discounted return formulations.

reinforcement-learning Reinforcement Learning

Transductive Active Learning with Application to Safe Bayesian Optimization

1 code implementation ICML Workshop on Aligning Reinforcement Learning Experimentalists and Theorists 2024 Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, Andreas Krause

We analyze Safe BO under the lens of a generalization of active learning with concrete prediction targets where sampling is restricted to an accessible region of the domain, while prediction targets may lie outside this region.

Active Learning Bayesian Optimization +4

Bandits with Preference Feedback: A Stackelberg Game Perspective

1 code implementation24 Jun 2024 Barna Pásztor, Parnian Kassraie, Andreas Krause

Bandits with preference feedback present a powerful tool for optimizing unknown target functions when only pairwise comparisons are allowed instead of direct value queries.

valid

Standardizing Structural Causal Models

no code implementations17 Jun 2024 Weronika Ormaniec, Scott Sussex, Lars Lorch, Bernhard Schölkopf, Andreas Krause

Moreover, contrary to the post-hoc standardization of data generated by standard SCMs, we prove that linear iSCMs are less identifiable from prior knowledge on the weights and do not collapse to deterministic relationships in large systems, which may make iSCMs a useful model in causal inference beyond the benchmarking problem studied here.

Benchmarking Causal Inference

Breeding Programs Optimization with Reinforcement Learning

no code implementations6 Jun 2024 Omar G. Younis, Luca Corinzia, Ioannis N. Athanasiadis, Andreas Krause, Joachim M. Buhmann, Matteo Turchetta

Crop breeding is crucial in improving agricultural productivity while potentially decreasing land usage, greenhouse gas emissions, and water consumption.

reinforcement-learning Reinforcement Learning +1

Stochastic Bilevel Optimization with Lower-Level Contextual Markov Decision Processes

no code implementations3 Jun 2024 Vinzenz Thoma, Barna Pasztor, Andreas Krause, Giorgia Ramponi, Yifan Hu

In various applications, the optimal policy in a strategic decision-making problem depends both on the environmental configuration and exogenous events.

Bilevel Optimization Decision Making

NeoRL: Efficient Exploration for Nonepisodic RL

no code implementations3 Jun 2024 Bhavya Sukhija, Lenart Treven, Florian Dörfler, Stelian Coros, Andreas Krause

We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i. e., without resets.

Efficient Exploration Reinforcement Learning (RL)

Safe Exploration Using Bayesian World Models and Log-Barrier Optimization

no code implementations9 May 2024 Yarden As, Bhavya Sukhija, Andreas Krause

A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process.

Safe Exploration

A PAC-Bayesian Framework for Optimal Control with Stability Guarantees

1 code implementation26 Mar 2024 Mahrokh Ghoddousi Boroujeni, Clara Lucía Galimberti, Andreas Krause, Giancarlo Ferrari-Trecate

Based on these bounds, we propose a new method for designing optimal controllers, offering a principled way to incorporate prior knowledge into the synthesis process, which aids in improving the control policy and mitigating overfitting.

Generalization Bounds

Transductive Active Learning: Theory and Applications

2 code implementations13 Feb 2024 Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, Andreas Krause

We study a generalization of classical active learning to real-world settings with concrete prediction targets where sampling is restricted to an accessible region of the domain, while prediction targets may lie outside this region.

Active Learning Bayesian Optimization +2

Active Few-Shot Fine-Tuning

no code implementations13 Feb 2024 Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, Andreas Krause

We study the question: How can we select the right data for fine-tuning to a specific task?

Active Learning Generalization Bounds +1

Safe Guaranteed Exploration for Non-linear Systems

1 code implementation9 Feb 2024 Manish Prajapat, Johannes Köhler, Matteo Turchetta, Andreas Krause, Melanie N. Zeilinger

Based on this framework we propose an efficient algorithm, SageMPC, SAfe Guaranteed Exploration using Model Predictive Control.

Efficient Exploration Model Predictive Control

Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL

1 code implementation8 Feb 2024 Jiawei Huang, Niao He, Andreas Krause

We study the sample complexity of reinforcement learning (RL) in Mean-Field Games (MFGs) with model-based function approximation that requires strategic exploration to find a Nash Equilibrium policy.

Computational Efficiency Reinforcement Learning (RL)

Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach

no code implementations16 Jan 2024 Mahrokh Ghoddousi Boroujeni, Andreas Krause, Giancarlo Ferrari Trecate

Personalized federated learning (PFL) goes one step further by adapting the global model to each client, enhancing the model's fit for different clients.

Personalized Federated Learning

3DReact: Geometric deep learning for chemical reactions

1 code implementation13 Dec 2023 Puck van Gerwen, Ksenia R. Briling, Charlotte Bunne, Vignesh Ram Somnath, Ruben Laplaza, Andreas Krause, Clemence Corminboeuf

We show that, compared to existing models for reaction property prediction, 3DReact offers a flexible framework that exploits atom-mapping information, if available, as well as geometries of reactants and products (in an invariant or equivariant fashion).

Deep Learning Property Prediction

Sinkhorn Flow: A Continuous-Time Framework for Understanding and Generalizing the Sinkhorn Algorithm

no code implementations28 Nov 2023 Mohammad Reza Karimi, Ya-Ping Hsieh, Andreas Krause

Many problems in machine learning can be formulated as solving entropy-regularized optimal transport on the space of probability measures.

Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning

no code implementations13 Nov 2023 Arjun Bhardwaj, Jonas Rothfuss, Bhavya Sukhija, Yarden As, Marco Hutter, Stelian Coros, Andreas Krause

We introduce PACOH-RL, a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics.

Meta-Learning Meta Reinforcement Learning +2

Implicit Manifold Gaussian Process Regression

1 code implementation NeurIPS 2023 Bernardo Fichera, Viacheslav Borovitskiy, Andreas Krause, Aude Billard

Gaussian process regression is widely used because of its ability to provide well-calibrated uncertainty estimates and handle small or sparse datasets.

regression

Intrinsic Gaussian Vector Fields on Manifolds

1 code implementation28 Oct 2023 Daniel Robert-Nicoud, Andreas Krause, Viacheslav Borovitskiy

Various applications ranging from robotics to climate science require modeling signals on non-Euclidean domains, such as the sphere.

Uncertainty Quantification

DockGame: Cooperative Games for Multimeric Rigid Protein Docking

no code implementations9 Oct 2023 Vignesh Ram Somnath, Pier Giuseppe Sessa, Maria Rodriguez Martinez, Andreas Krause

Most traditional and deep learning methods for docking have focused mainly on binary docking, following either a search-based, regression-based, or generative modeling paradigm.

Protein Design

Distributionally Robust Model-based Reinforcement Learning with Large State Spaces

no code implementations5 Sep 2023 Shyam Sundhar Ramesh, Pier Giuseppe Sessa, Yifan Hu, Andreas Krause, Ilija Bogunovic

Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment.

Gaussian Processes Model-based Reinforcement Learning +2

Model-based Causal Bayesian Optimization

no code implementations31 Jul 2023 Scott Sussex, Pier Giuseppe Sessa, Anastasiia Makarova, Andreas Krause

We formalize this generalization of CBO as Adversarial Causal Bayesian Optimization (ACBO) and introduce the first algorithm for ACBO with bounded regret: Causal Bayesian Optimization with Multiplicative Weights (CBO-MW).

Bayesian Optimization counterfactual

Submodular Reinforcement Learning

1 code implementation25 Jul 2023 Manish Prajapat, Mojmír Mutný, Melanie N. Zeilinger, Andreas Krause

In many important applications, such as coverage control, experiment design and informative path planning, rewards naturally have diminishing returns, i. e., their value decreases in light of similar states visited previously.

reinforcement-learning Reinforcement Learning +1

Anytime Model Selection in Linear Bandits

1 code implementation NeurIPS 2023 Parnian Kassraie, Nicolas Emmenegger, Andreas Krause, Aldo Pacchiano

This allows us to develop ALEXP, which has an exponentially improved ($\log M$) dependence on $M$ for its regret.

Model Selection

Safe Risk-averse Bayesian Optimization for Controller Tuning

no code implementations23 Jun 2023 Christopher Koenig, Miks Ozols, Anastasia Makarova, Efe C. Balta, Andreas Krause, Alisa Rupenyan

Controller tuning and parameter optimization are crucial in system design to improve both the controller and underlying system performance.

Bayesian Optimization

Unbalanced Diffusion Schrödinger Bridge

1 code implementation15 Jun 2023 Matteo Pariset, Ya-Ping Hsieh, Charlotte Bunne, Andreas Krause, Valentin De Bortoli

Schr\"odinger bridges (SBs) provide an elegant framework for modeling the temporal evolution of populations in physical, chemical, or biological systems.

Provably Learning Nash Policies in Constrained Markov Potential Games

no code implementations13 Jun 2023 Pragnya Alatur, Giorgia Ramponi, Niao He, Andreas Krause

Multi-agent reinforcement learning (MARL) addresses sequential decision-making problems with multiple agents, where each agent optimizes its own objective.

Decision Making Multi-agent Reinforcement Learning +1

Tuning Legged Locomotion Controllers via Safe Bayesian Optimization

1 code implementation12 Jun 2023 Daniel Widmer, Dongho Kang, Bhavya Sukhija, Jonas Hübotter, Andreas Krause, Stelian Coros

This paper presents a data-driven strategy to streamline the deployment of model-based controllers in legged robotic hardware platforms.

Bayesian Optimization Efficient Exploration

A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networks

no code implementations16 May 2023 Ali Gorji, Andisheh Amrollahi, Andreas Krause

We show how this spectral bias towards low-degree frequencies can in fact hurt the neural network's generalization on real-world datasets.

Safe Deep RL for Intraoperative Planning of Pedicle Screw Placement

no code implementations9 May 2023 Yunke Ao, Hooman Esfandiari, Fabio Carrillo, Yarden As, Mazda Farshad, Benjamin F. Grewe, Andreas Krause, Philipp Fuernstahl

Spinal fusion surgery requires highly accurate implantation of pedicle screw implants, which must be conducted in critical proximity to vital structures with a limited view of anatomy.

Anatomy

Hallucinated Adversarial Control for Conservative Offline Policy Evaluation

1 code implementation2 Mar 2023 Jonas Rothfuss, Bhavya Sukhija, Tobias Birchler, Parnian Kassraie, Andreas Krause

We study the problem of conservative off-policy evaluation (COPE) where given an offline dataset of environment interactions, collected by other agents, we seek to obtain a (tight) lower bound on a policy's performance.

continuous-control Continuous Control +2

Aligned Diffusion Schrödinger Bridges

2 code implementations22 Feb 2023 Vignesh Ram Somnath, Matteo Pariset, Ya-Ping Hsieh, Maria Rodriguez Martinez, Andreas Krause, Charlotte Bunne

Diffusion Schr\"odinger bridges (DSB) have recently emerged as a powerful framework for recovering stochastic dynamics via their marginal observations at different time points.

Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applications

no code implementations7 Feb 2023 Johannes Kirschner, Tor Lattimore, Andreas Krause

Partial monitoring is an expressive framework for sequential decision-making with an abundance of applications, including graph-structured and dueling bandits, dynamic pricing and transductive feedback models.

Decision Making

Near-optimal Policy Identification in Active Reinforcement Learning

no code implementations19 Dec 2022 Xiang Li, Viraj Mehta, Johannes Kirschner, Ian Char, Willie Neiswanger, Jeff Schneider, Andreas Krause, Ilija Bogunovic

Many real-world reinforcement learning tasks require control of complex dynamical systems that involve both costly data acquisition processes and large state spaces.

Bayesian Optimization reinforcement-learning +2

Model-based Causal Bayesian Optimization

1 code implementation18 Nov 2022 Scott Sussex, Anastasiia Makarova, Andreas Krause

How should we intervene on an unknown structural equation model to maximize a downstream variable of interest?

Bayesian Optimization

Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice

no code implementations14 Nov 2022 Jonas Rothfuss, Martin Josifoski, Vincent Fortuin, Andreas Krause

Meta-Learning aims to speed up the learning process on new tasks by acquiring useful inductive biases from datasets of related learning tasks.

Gaussian Processes Meta-Learning +1

Isotropic Gaussian Processes on Finite Spaces of Graphs

3 code implementations3 Nov 2022 Viacheslav Borovitskiy, Mohammad Reza Karimi, Vignesh Ram Somnath, Andreas Krause

We propose a principled way to define Gaussian process priors on various sets of unweighted graphs: directed or undirected, with or without loops.

Gaussian Processes Molecular Property Prediction +1

Instance-Dependent Generalization Bounds via Optimal Transport

no code implementations2 Nov 2022 Songyan Hou, Parnian Kassraie, Anastasis Kratsios, Andreas Krause, Jonas Rothfuss

Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks.

Generalization Bounds Inductive Bias

Lifelong Bandit Optimization: No Prior and No Regret

no code implementations27 Oct 2022 Felix Schur, Parnian Kassraie, Jonas Rothfuss, Andreas Krause

Our algorithm can be paired with any kernelized or linear bandit algorithm and guarantees oracle optimal performance, meaning that as more tasks are solved, the regret of LIBO on each task converges to the regret of the bandit algorithm with oracle knowledge of the true kernel.

Leveraging Demonstrations with Latent Space Priors

1 code implementation26 Oct 2022 Jonas Gehring, Deepak Gopinath, Jungdam Won, Andreas Krause, Gabriel Synnaeve, Nicolas Usunier

Starting with a learned joint latent space, we separately train a generative model of demonstration sequences and an accompanying low-level policy.

Offline RL

A Dynamical System View of Langevin-Based Non-Convex Sampling

no code implementations NeurIPS 2023 Mohammad Reza Karimi, Ya-Ping Hsieh, Andreas Krause

Non-convex sampling is a key challenge in machine learning, central to non-convex optimization in deep learning as well as to approximate probabilistic inference.

MARS: Meta-Learning as Score Matching in the Function Space

1 code implementation24 Oct 2022 Krunoslav Lehman Pavasovic, Jonas Rothfuss, Andreas Krause

To circumvent these issues, we approach meta-learning through the lens of functional Bayesian neural network inference, which views the prior as a stochastic process and performs inference in the function space.

Meta-Learning

Movement Penalized Bayesian Optimization with Application to Wind Energy Systems

no code implementations14 Oct 2022 Shyam Sundhar Ramesh, Pier Giuseppe Sessa, Andreas Krause, Ilija Bogunovic

Contextual Bayesian optimization (CBO) is a powerful framework for sequential decision-making given side information, with important applications, e. g., in wind energy systems.

Bayesian Optimization Decision Making

Near-Optimal Multi-Agent Learning for Safe Coverage Control

1 code implementation12 Oct 2022 Manish Prajapat, Matteo Turchetta, Melanie N. Zeilinger, Andreas Krause

In this paper, we aim to efficiently learn the density to approximately solve the coverage problem while preserving the agents' safety.

Diversity Navigate +1

Replicable Bandits

no code implementations4 Oct 2022 Hossein Esfandiari, Alkis Kalavasis, Amin Karbasi, Andreas Krause, Vahab Mirrokni, Grigoris Velegkas

Similarly, for stochastic linear bandits (with finitely and infinitely many arms) we develop replicable policies that achieve the best-known problem-independent regret bounds with an optimal dependency on the replicability parameter.

Multi-Armed Bandits

Meta-Learning Priors for Safe Bayesian Optimization

no code implementations3 Oct 2022 Jonas Rothfuss, Christopher Koenig, Alisa Rupenyan, Andreas Krause

In the presence of unknown safety constraints, it is crucial to choose reliable model hyper-parameters to avoid safety violations.

Bayesian Optimization Meta-Learning +1

Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning

2 code implementations21 Jul 2022 Ilnura Usmanova, Yarden As, Maryam Kamgarpour, Andreas Krause

We introduce a general approach for seeking a stationary point in high dimensional non-linear stochastic optimization problems in which maintaining safety during learning is crucial.

reinforcement-learning Reinforcement Learning (RL) +2

Active Exploration for Inverse Reinforcement Learning

1 code implementation18 Jul 2022 David Lindner, Andreas Krause, Giorgia Ramponi

We propose a novel IRL algorithm: Active exploration for Inverse Reinforcement Learning (AceIRL), which actively explores an unknown environment and expert policy to quickly learn the expert's reward function and identify a good policy.

reinforcement-learning Reinforcement Learning +1

Graph Neural Network Bandits

no code implementations13 Jul 2022 Parnian Kassraie, Andreas Krause, Ilija Bogunovic

By establishing a novel connection between such kernels and the graph neural tangent kernel (GNTK), we introduce the first GNN confidence bound and use it to design a phased-elimination algorithm with sublinear regret.

Drug Discovery Graph Neural Network

Active Exploration via Experiment Design in Markov Chains

no code implementations29 Jun 2022 Mojmír Mutný, Tadeusz Janik, Andreas Krause

A key challenge in science and engineering is to design experiments to learn about some unknown quantity of interest.

Experimental Design

Supervised Training of Conditional Monge Maps

1 code implementation28 Jun 2022 Charlotte Bunne, Andreas Krause, Marco Cuturi

To account for that context in OT estimation, we introduce CondOT, a multi-task approach to estimate a family of OT maps conditioned on a context variable, using several pairs of measures $\left(\mu_i, \nu_i\right)$ tagged with a context label $c_i$.

Learning To Cut By Looking Ahead: Cutting Plane Selection via Imitation Learning

no code implementations27 Jun 2022 Max B. Paulus, Giulia Zarpellon, Andreas Krause, Laurent Charlin, Chris J. Maddison

Cutting planes are essential for solving mixed-integer linear problems (MILPs), because they facilitate bound improvements on the optimal solution value.

Imitation Learning

Invariant Causal Mechanisms through Distribution Matching

1 code implementation23 Jun 2022 Mathieu Chevalley, Charlotte Bunne, Andreas Krause, Stefan Bauer

Learning representations that capture the underlying data generating process is a key problem for data efficient and robust use of neural networks.

Domain Generalization

Riemannian stochastic approximation algorithms

no code implementations14 Jun 2022 Mohammad Reza Karimi, Ya-Ping Hsieh, Panayotis Mertikopoulos, Andreas Krause

We examine a wide class of stochastic approximation algorithms for solving (stochastic) nonlinear problems on Riemannian manifolds.

Riemannian optimization

Interactively Learning Preference Constraints in Linear Bandits

1 code implementation10 Jun 2022 David Lindner, Sebastian Tschiatschek, Katja Hofmann, Andreas Krause

We provide an instance-dependent lower bound for constrained linear best-arm identification and show that ACOL's sample complexity matches the lower bound in the worst-case.

Decision Making

Active Bayesian Causal Inference

1 code implementation4 Jun 2022 Christian Toth, Lars Lorch, Christian Knoll, Andreas Krause, Franz Pernkopf, Robert Peharz, Julius von Kügelgen

In this work, we propose Active Bayesian Causal Inference (ABCI), a fully-Bayesian active learning framework for integrated causal discovery and reasoning, which jointly infers a posterior over causal models and queries of interest.

Active Learning Causal Discovery +2

Experimental Design for Linear Functionals in Reproducing Kernel Hilbert Spaces

no code implementations26 May 2022 Mojmír Mutný, Andreas Krause

In this work, we investigate the optimal design of experiments for {\em estimation of linear functionals in reproducing kernel Hilbert spaces (RKHSs)}.

Experimental Design

Amortized Inference for Causal Structure Learning

1 code implementation25 May 2022 Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, Bernhard Schölkopf

Rather than searching over structures, we train a variational inference model to directly predict the causal structure from observational or interventional data.

Causal Discovery Inductive Bias +1

Gradient-Based Trajectory Optimization With Learned Dynamics

no code implementations9 Apr 2022 Bhavya Sukhija, Nathanael Köhler, Miguel Zamora, Simon Zimmermann, Sebastian Curi, Andreas Krause, Stelian Coros

In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car, and gives good performance in combination with trajectory optimization methods.

Multi-Scale Representation Learning on Proteins

1 code implementation NeurIPS 2021 Vignesh Ram Somnath, Charlotte Bunne, Andreas Krause

This paper introduces a multi-scale graph construction of a protein -- HoloProt -- connecting surface to structure and sequence.

graph construction Protein Function Prediction +3

Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation

no code implementations14 Mar 2022 Pier Giuseppe Sessa, Maryam Kamgarpour, Andreas Krause

We consider model-based multi-agent reinforcement learning, where the environment transition model is unknown and can only be learned via expensive interactions with the environment.

Autonomous Driving Gaussian Processes +4

The Schrödinger Bridge between Gaussian Measures has a Closed Form

no code implementations11 Feb 2022 Charlotte Bunne, Ya-Ping Hsieh, Marco Cuturi, Andreas Krause

The static optimal transport $(\mathrm{OT})$ problem between Gaussians seeks to recover an optimal map, or more generally a coupling, to morph a Gaussian into another.

Gaussian Processes MORPH

A Robust Phased Elimination Algorithm for Corruption-Tolerant Gaussian Process Bandits

no code implementations3 Feb 2022 Ilija Bogunovic, Zihan Li, Andreas Krause, Jonathan Scarlett

We consider the sequential optimization of an unknown, continuous, and expensive to evaluate reward function, from noisy and adversarially corrupted observed rewards.

Meta-Learning Hypothesis Spaces for Sequential Decision-making

no code implementations1 Feb 2022 Parnian Kassraie, Jonas Rothfuss, Andreas Krause

We demonstrate our approach on the kernelized bandit problem (a. k. a.~Bayesian optimization), where we establish regret bounds competitive with those given the true kernel.

Bayesian Optimization Decision Making +3

GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems

1 code implementation24 Jan 2022 Bhavya Sukhija, Matteo Turchetta, David Lindner, Andreas Krause, Sebastian Trimpe, Dominik Baumann

Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage.

Safe Exploration

Constrained Policy Optimization via Bayesian World Models

1 code implementation ICLR 2022 Yarden As, Ilnura Usmanova, Sebastian Curi, Andreas Krause

Improving sample-efficiency and safety are crucial challenges when deploying reinforcement learning in high-stakes real world applications.

reinforcement-learning Reinforcement Learning (RL)

Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking

1 code implementation ICLR 2022 Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi Jaakkola, Andreas Krause

Protein complex formation is a central problem in biology, being involved in most of the cell's processes, and essential for applications, e. g. drug design or protein engineering.

Graph Matching Translation

Misspecified Gaussian Process Bandit Optimization

no code implementations NeurIPS 2021 Ilija Bogunovic, Andreas Krause

Instead, we introduce a \emph{misspecified} kernelized bandit setting where the unknown function can be $\epsilon$--uniformly approximated by a function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS).

Risk-averse Heteroscedastic Bayesian Optimization

1 code implementation NeurIPS 2021 Anastasiia Makarova, Ilnura Usmanova, Ilija Bogunovic, Andreas Krause

We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori.

Bayesian Optimization

Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems

1 code implementation NeurIPS 2021 Andreas Schlaginhaufen, Philippe Wenk, Andreas Krause, Florian Dörfler

To this end, neural ODEs regularized with neural Lyapunov functions are a promising approach when states are fully observed.

Diversified Sampling for Batched Bayesian Optimization with Determinantal Point Processes

no code implementations22 Oct 2021 Elvis Nava, Mojmír Mutný, Andreas Krause

In Bayesian Optimization (BO) we study black-box function optimization with noisy point evaluations and Bayesian priors.

Bayesian Optimization Diversity +2

Sensing Cox Processes via Posterior Sampling and Positive Bases

1 code implementation21 Oct 2021 Mojmír Mutný, Andreas Krause

We study adaptive sensing of Cox point processes, a widely used model from spatial statistics.

Experimental Design Point Processes

Hierarchical Skills for Efficient Exploration

1 code implementation NeurIPS 2021 Jonas Gehring, Gabriel Synnaeve, Andreas Krause, Nicolas Usunier

We alleviate the need for prior knowledge by proposing a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.

continuous-control Continuous Control +6

Data Summarization via Bilevel Optimization

no code implementations26 Sep 2021 Zalán Borsos, Mojmír Mutný, Marco Tagliasacchi, Andreas Krause

We show the effectiveness of our framework for a wide range of models in various settings, including training non-convex models online and batch active learning.

Active Learning Bilevel Optimization +2

Contextual Games: Multi-Agent Learning with Side Information

no code implementations NeurIPS 2020 Pier Giuseppe Sessa, Ilija Bogunovic, Andreas Krause, Maryam Kamgarpour

We formulate the novel class of contextual games, a type of repeated games driven by contextual information at each round.

Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning

no code implementations8 Jul 2021 Barna Pásztor, Ilija Bogunovic, Andreas Krause

Learning in multi-agent systems is highly challenging due to several factors including the non-stationarity introduced by agents' interactions and the combinatorial nature of their state and action spaces.

Gaussian Processes Model-based Reinforcement Learning +3

Neural Contextual Bandits without Regret

1 code implementation7 Jul 2021 Parnian Kassraie, Andreas Krause

Contextual bandits are a rich model for sequential decision making given side information, with important applications, e. g., in recommender systems.

Decision Making Multi-Armed Bandits +1

PopSkipJump: Decision-Based Attack for Probabilistic Classifiers

1 code implementation14 Jun 2021 Carl-Johann Simon-Gabriel, Noman Ahmed Sheikh, Andreas Krause

Most current classifiers are vulnerable to adversarial examples, small input perturbations that change the classification output.

Proximal Optimal Transport Modeling of Population Dynamics

2 code implementations11 Jun 2021 Charlotte Bunne, Laetitia Meng-Papaxanthos, Andreas Krause, Marco Cuturi

We propose to model these trajectories as collective realizations of a causal Jordan-Kinderlehrer-Otto (JKO) flow of measures: The JKO scheme posits that the new configuration taken by a population at time $t+1$ is one that trades off an improvement, in the sense that it decreases an energy, while remaining close (in Wasserstein distance) to the previous configuration observed at $t$.

Meta-Learning Reliable Priors in the Function Space

no code implementations NeurIPS 2021 Jonas Rothfuss, Dominique Heyn, Jinfan Chen, Andreas Krause

When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks.

Bayesian Optimization Decision Making +2

Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning

no code implementations ICLR 2022 Yatao Bian, Yu Rong, Tingyang Xu, Jiaxiang Wu, Andreas Krause, Junzhou Huang

By running fixed point iteration for multiple steps, we achieve a trajectory of the valuations, among which we define the valuation with the best conceivable decoupling error as the Variational Index.

Data Valuation Variational Inference

Learning Graph Models for Template-Free Retrosynthesis

no code implementations arXiv 2021 Vignesh Ram Somnath, Charlotte Bunne, Connor W. Coley, Andreas Krause, Regina Barzilay

Retrosynthesis prediction is a fundamental problem in organic synthesis, where the task is to identify precursor molecules that can be used to synthesize a target molecule.

Retrosynthesis Single-step retrosynthesis

Addressing the Long-term Impact of ML Decisions via Policy Regret

1 code implementation2 Jun 2021 David Lindner, Hoda Heidari, Andreas Krause

To capture the long-term effects of ML-based allocation decisions, we study a setting in which the reward from each arm evolves every time the decision-maker pulls that arm.

Multi-Armed Bandits

DiBS: Differentiable Bayesian Structure Learning

2 code implementations NeurIPS 2021 Lars Lorch, Jonas Rothfuss, Bernhard Schölkopf, Andreas Krause

In this work, we propose a general, fully differentiable framework for Bayesian structure learning (DiBS) that operates in the continuous space of a latent probabilistic graph representation.

Causal Discovery Variational Inference

Bias-Robust Bayesian Optimization via Dueling Bandits

no code implementations25 May 2021 Johannes Kirschner, Andreas Krause

We consider Bayesian optimization in settings where observations can be adversarially biased, for example by an uncontrolled hidden confounder.

Bayesian Optimization

A note on the CAPM with endogenously consistent market returns

no code implementations21 May 2021 Andreas Krause

I demonstrate that with the market return determined by the equilibrium returns of the CAPM, expected returns of an asset are affected by the risks of all assets jointly.

Regret Bounds for Gaussian-Process Optimization in Large Domains

1 code implementation NeurIPS 2021 Manuel Wüthrich, Bernhard Schölkopf, Andreas Krause

These regret bounds illuminate the relationship between the number of evaluations, the domain size (i. e. cardinality of finite domains / Lipschitz constant of the covariance function in continuous domains), and the optimality of the retrieved function value.

Automatic Termination for Hyperparameter Optimization

1 code implementation16 Apr 2021 Anastasia Makarova, Huibin Shen, Valerio Perrone, Aaron Klein, Jean Baptiste Faddoul, Andreas Krause, Matthias Seeger, Cedric Archambeau

Across an extensive range of real-world HPO problems and baselines, we show that our termination criterion achieves a better trade-off between the test performance and optimization time.

Bayesian Optimization Hyperparameter Optimization

Risk-Averse Offline Reinforcement Learning

1 code implementation ICLR 2021 Núria Armengol Urpí, Sebastian Curi, Andreas Krause

We demonstrate empirically that in the presence of natural distribution-shifts, O-RAAC learns policies with good average performance.

reinforcement-learning Reinforcement Learning +1

Efficient Pure Exploration for Combinatorial Bandits with Semi-Bandit Feedback

no code implementations21 Jan 2021 Marc Jourdan, Mojmír Mutný, Johannes Kirschner, Andreas Krause

Combinatorial bandits with semi-bandit feedback generalize multi-armed bandits, where the agent chooses sets of arms and observes a noisy reward for each arm contained in the chosen set.

Multi-Armed Bandits

Meta-Learning Bayesian Neural Network Priors Based on PAC-Bayesian Theory

no code implementations1 Jan 2021 Jonas Rothfuss, Martin Josifoski, Andreas Krause

Bayesian deep learning is a promising approach towards improved uncertainty quantification and sample efficiency.

Meta-Learning Uncertainty Quantification +1

Logistic Q-Learning

no code implementations21 Oct 2020 Joan Bas-Serrano, Sebastian Curi, Andreas Krause, Gergely Neu

We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs.

Q-Learning Reinforcement Learning (RL)

Online Active Model Selection for Pre-trained Classifiers

1 code implementation19 Oct 2020 Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlaš, Johannes Rausch, Ce Zhang, Andreas Krause

Given $k$ pre-trained classifiers and a stream of unlabeled data examples, how can we actively decide when to query a label so that we can distinguish the best model from the rest while making a small number of queries?

Model Selection

Semi-supervised Batch Active Learning via Bilevel Optimization

1 code implementation19 Oct 2020 Zalán Borsos, Marco Tagliasacchi, Andreas Krause

Active learning is an effective technique for reducing the labeling cost by improving data efficiency.

Active Learning Bilevel Optimization +1

Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator

5 code implementations ICLR 2021 Max B. Paulus, Chris J. Maddison, Andreas Krause

Gradient estimation in models with discrete latent variables is a challenging problem, because the simplest unbiased estimators tend to have high variance.

Learning Set Functions that are Sparse in Non-Orthogonal Fourier Bases

3 code implementations1 Oct 2020 Chris Wendler, Andisheh Amrollahi, Bastian Seifert, Andreas Krause, Markus Püschel

Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain.

Recommendation Systems

Learning to Play Sequential Games versus Unknown Opponents

1 code implementation NeurIPS 2020 Pier Giuseppe Sessa, Ilija Bogunovic, Maryam Kamgarpour, Andreas Krause

We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action.

Bilevel Optimization

Stochastic Linear Bandits Robust to Adversarial Attacks

no code implementations7 Jul 2020 Ilija Bogunovic, Arpan Losalka, Andreas Krause, Jonathan Scarlett

We consider a stochastic linear bandit problem in which the rewards are not only subject to random noise, but also adversarial attacks subject to a suitable budget $C$ (i. e., an upper bound on the sum of corruption magnitudes across the time horizon).

Continuous Submodular Function Maximization

no code implementations24 Jun 2020 Yatao Bian, Joachim M. Buhmann, Andreas Krause

We start by a thorough characterization of the class of continuous submodular functions, and show that continuous submodularity is equivalent to a weak version of the diminishing returns (DR) property.

Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning

1 code implementation NeurIPS 2020 Sebastian Curi, Felix Berkenkamp, Andreas Krause

Based on this theoretical foundation, we show how optimistic exploration can be easily combined with state-of-the-art reinforcement learning algorithms and different probabilistic models.

Model-based Reinforcement Learning reinforcement-learning +2

Learning Graph Models for Retrosynthesis Prediction

2 code implementations NeurIPS 2021 Vignesh Ram Somnath, Charlotte Bunne, Connor W. Coley, Andreas Krause, Regina Barzilay

Retrosynthesis prediction is a fundamental problem in organic synthesis, where the task is to identify precursor molecules that can be used to synthesize a target molecule.

Retrosynthesis

Safe non-smooth black-box optimization with application to policy search

no code implementations L4DC 2020 Ilnura Usmanova, Andreas Krause, Maryam Kamgarpour

For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points.

Hierarchical Image Classification using Entailment Cone Embeddings

1 code implementation2 Apr 2020 Ankit Dhall, Anastasia Makarova, Octavian Ganea, Dario Pavllo, Michael Greeff, Andreas Krause

Image classification has been studied extensively, but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training.

Classification General Classification +2

SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives

1 code implementation5 Mar 2020 Emmanouil Angelis, Philippe Wenk, Bernhard Schölkopf, Stefan Bauer, Andreas Krause

Gaussian processes are an important regression tool with excellent analytic properties which allow for direct integration of derivative observations.

Gaussian Processes regression

Corruption-Tolerant Gaussian Process Bandit Optimization

no code implementations4 Mar 2020 Ilija Bogunovic, Andreas Krause, Jonathan Scarlett

We consider the problem of optimizing an unknown (typically non-convex) function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS), based on noisy bandit feedback.

Mixed Strategies for Robust Optimization of Unknown Objectives

no code implementations28 Feb 2020 Pier Giuseppe Sessa, Ilija Bogunovic, Maryam Kamgarpour, Andreas Krause

We consider robust optimization problems, where the goal is to optimize an unknown objective function against the worst-case realization of an uncertain parameter.

Autonomous Vehicles Gaussian Processes +1

Information Directed Sampling for Linear Partial Monitoring

no code implementations25 Feb 2020 Johannes Kirschner, Tor Lattimore, Andreas Krause

Partial monitoring is a rich framework for sequential decision making under uncertainty that generalizes many well known bandit models, including linear, combinatorial and dueling bandits.

Decision Making Decision Making Under Uncertainty

Distributionally Robust Bayesian Optimization

no code implementations20 Feb 2020 Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, Andreas Krause

Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate.

Bayesian Optimization

Efficiently Learning Fourier Sparse Set Functions

1 code implementation NeurIPS 2019 Andisheh Amrollahi, Amir Zandieh, Michael Kapralov, Andreas Krause

In this paper we consider the problem of efficiently learning set functions that are defined over a ground set of size $n$ and that are sparse (say $k$-sparse) in the Fourier domain.

A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness

no code implementations8 Nov 2019 Mohammad Yaghini, Andreas Krause, Hoda Heidari

Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases.

Decision Making Fairness

Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization

no code implementations29 Oct 2019 Matteo Turchetta, Andreas Krause, Sebastian Trimpe

In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment.

Bayesian Optimization reinforcement-learning +2

Adaptive Sampling for Stochastic Risk-Averse Learning

1 code implementation NeurIPS 2020 Sebastian Curi, Kfir. Y. Levy, Stefanie Jegelka, Andreas Krause

In high-stakes machine learning applications, it is crucial to not only perform well on average, but also when restricted to difficult examples.

Point Processes

Noise Regularization for Conditional Density Estimation

1 code implementation21 Jul 2019 Jonas Rothfuss, Fabio Ferreira, Simon Boehm, Simon Walther, Maxim Ulrich, Tamim Asfour, Andreas Krause

To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training.

Density Estimation

Structured Variational Inference in Unstable Gaussian Process State Space Models

1 code implementation16 Jul 2019 Silvan Melchior, Sebastian Curi, Felix Berkenkamp, Andreas Krause

Finally, we show experimentally that our learning algorithm performs well in stable and unstable real systems with hidden states.

Gaussian Processes State Space Models +1

Mixed-Variable Bayesian Optimization

no code implementations2 Jul 2019 Erik Daxberger, Anastasia Makarova, Matteo Turchetta, Andreas Krause

However, few methods exist for mixed-variable domains and none of them can handle discrete constraints that arise in many real-world applications.

Bayesian Optimization Thompson Sampling

Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning

no code implementations28 Jun 2019 Marcello Fiducioso, Sebastian Curi, Benedikt Schumacher, Markus Gwerder, Andreas Krause

Furthermore, this successful attempt paves the way for further use at different levels of HVAC systems, with promising energy, operational, and commissioning costs savings, and it is a practical demonstration of the positive effects that Artificial Intelligence can have on environmental sustainability.

Bayesian Optimization

Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning

1 code implementation27 Jun 2019 Torsten Koller, Felix Berkenkamp, Matteo Turchetta, Joschka Boedecker, Andreas Krause

We evaluate the resulting algorithm to safely explore the dynamics of an inverted pendulum and to solve a reinforcement learning task on a cart-pole system with safety constraints.

Model Predictive Control reinforcement-learning +3

Stochastic Bandits with Context Distributions

1 code implementation NeurIPS 2019 Johannes Kirschner, Andreas Krause

We introduce a stochastic contextual bandit model where at each time step the environment chooses a distribution over a context set and samples the context from this distribution.

Learning Generative Models across Incomparable Spaces

no code implementations14 May 2019 Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety.

Relational Reasoning

Evaluating GANs via Duality

no code implementations ICLR 2019 Paulina Grnarova, Kfir. Y. Levy, Aurelien Lucchi, Nathanael Perraudin, Thomas Hofmann, Andreas Krause

Generative Adversarial Networks (GANs) have shown great results in accurately modeling complex distributions, but their training is known to be difficult due to instabilities caused by a challenging minimax optimization problem.

Online Variance Reduction with Mixtures

1 code implementation29 Mar 2019 Zalán Borsos, Sebastian Curi, Kfir. Y. Levy, Andreas Krause

Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction.

Stochastic Optimization

Multi-Player Bandits: The Adversarial Case

no code implementations21 Feb 2019 Pragnya Alatur, Kfir. Y. Levy, Andreas Krause

We consider a setting where multiple players sequentially choose among a common set of actions (arms).

ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems

2 code implementations17 Feb 2019 Philippe Wenk, Gabriele Abbati, Michael A. Osborne, Bernhard Schölkopf, Andreas Krause, Stefan Bauer

Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting.

Gaussian Processes Model Selection +1

Adaptive Sequence Submodularity

1 code implementation NeurIPS 2019 Marko Mitrovic, Ehsan Kazemi, Moran Feldman, Andreas Krause, Amin Karbasi

In many machine learning applications, one needs to interactively select a sequence of items (e. g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e. g., guiding an agent through a series of states).

Decision Making Link Prediction +1

Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces

2 code implementations8 Feb 2019 Johannes Kirschner, Mojmír Mutný, Nicole Hiller, Rasmus Ischebeck, Andreas Krause

In order to scale the method and keep its benefits, we propose an algorithm (LineBO) that restricts the problem to a sequence of iteratively chosen one-dimensional sub-problems that can be solved efficiently.

Bayesian Optimization

No-Regret Bayesian Optimization with Unknown Hyperparameters

no code implementations10 Jan 2019 Felix Berkenkamp, Angela P. Schoellig, Andreas Krause

In this paper, we present the first BO algorithm that is provably no-regret and converges to the optimum without knowledge of the hyperparameters.

Bayesian Optimization