Search Results for author: Katja Hofmann

Found 60 papers, 26 papers with code

Trajectory VAE for multi-modal imitation

no code implementations ICLR 2019 Xiaoyu Lu, Jan Stuehmer, Katja Hofmann

In this paper, we use a generative model to capture different emergent playstyles in an unsupervised manner, enabling the imitation of a diverse range of distinct behaviours.

Continuous Control Imitation Learning

Toward Human-AI Alignment in Large-Scale Multi-Player Games

no code implementations5 Feb 2024 Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad

First, we analyze extensive human gameplay data from Xbox's Bleeding Edge (100K+ games), uncovering behavioral patterns in a complex task space.

AI Agent

Transformer Neural Autoregressive Flows

no code implementations3 Jan 2024 Massimiliano Patacchiola, Aliaksandra Shysheya, Katja Hofmann, Richard E. Turner

In this paper, we propose a novel solution to these challenges by exploiting transformers to define a new class of neural flows called Transformer Neural Autoregressive Flows (T-NAFs).

Density Estimation

Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation

1 code implementation23 Jun 2023 Massimiliano Patacchiola, Mingfei Sun, Katja Hofmann, Richard E. Turner

Despite its simplicity this baseline is competitive with meta-learning methods on a variety of conditions and is able to imitate target policies trained on unseen variations of the original environment.

Few-Shot Image Classification Few-Shot Imitation Learning +3

Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games

no code implementations2 Mar 2023 Stephanie Milani, Arthur Juliani, Ida Momennejad, Raluca Georgescu, Jaroslaw Rzpecki, Alison Shaw, Gavin Costello, Fei Fang, Sam Devlin, Katja Hofmann

We aim to understand how people assess human likeness in navigation produced by people and artificially intelligent (AI) agents in a video game.

AI Agent

Trust-Region-Free Policy Optimization for Stochastic Policies

no code implementations15 Feb 2023 Mingfei Sun, Benjamin Ellis, Anuj Mahajan, Sam Devlin, Katja Hofmann, Shimon Whiteson

In this paper, we show that the trust region constraint over policies can be safely substituted by a trust-region-free constraint without compromising the underlying monotonic improvement guarantee.

Imitating Human Behaviour with Diffusion Models

1 code implementation25 Jan 2023 Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin

This paper studies their application as observation-to-action models for imitating human behaviour in sequential environments.

UniMASK: Unified Inference in Sequential Decision Problems

1 code implementation20 Nov 2022 Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin

Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks.

Decision Making

Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification

1 code implementation20 Jun 2022 Massimiliano Patacchiola, John Bronskill, Aliaksandra Shysheya, Katja Hofmann, Sebastian Nowozin, Richard E. Turner

In this paper we push this Pareto frontier in the few-shot image classification setting with a key contribution: a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance with a single forward pass of the user data (context).

Few-Shot Image Classification Few-Shot Learning +1

Interactively Learning Preference Constraints in Linear Bandits

1 code implementation10 Jun 2022 David Lindner, Sebastian Tschiatschek, Katja Hofmann, Andreas Krause

We provide an instance-dependent lower bound for constrained linear best-arm identification and show that ACOL's sample complexity matches the lower bound in the worst-case.

Decision Making

You May Not Need Ratio Clipping in PPO

no code implementations31 Jan 2022 Mingfei Sun, Vitaly Kurin, Guoqing Liu, Sam Devlin, Tao Qin, Katja Hofmann, Shimon Whiteson

Furthermore, we show that ESPO can be easily scaled up to distributed training with many workers, delivering strong performance as well.

Continuous Control

Trust Region Bounds for Decentralized PPO Under Non-stationarity

no code implementations31 Jan 2022 Mingfei Sun, Sam Devlin, Jacob Beck, Katja Hofmann, Shimon Whiteson

We present trust region bounds for optimizing decentralized policies in cooperative Multi-Agent Reinforcement Learning (MARL), which holds even when the transition dynamics are non-stationary.

Multi-agent Reinforcement Learning

Strategically Efficient Exploration in Competitive Multi-agent Reinforcement Learning

1 code implementation30 Jul 2021 Robert Loftin, Aadirupa Saha, Sam Devlin, Katja Hofmann

High sample complexity remains a barrier to the application of reinforcement learning (RL), particularly in multi-agent systems.

Efficient Exploration Multi-agent Reinforcement Learning +2

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

no code implementations2 Jul 2021 Grgur Kovač, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer

In this paper, we argue that aiming towards human-level AI requires a broader set of key social skills: 1) language use in complex and variable social contexts; 2) beyond language, complex embodied communication in multimodal settings within constantly evolving social worlds.

Benchmarking reinforcement-learning +1

Grounding Spatio-Temporal Language with Transformers

1 code implementation NeurIPS 2021 Tristan Karch, Laetitia Teodorescu, Katja Hofmann, Clément Moulin-Frier, Pierre-Yves Oudeyer

While there is an extended literature studying how machines can learn grounded language, the topic of how to learn spatio-temporal linguistic concepts is still largely uncharted.

Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation

1 code implementation20 May 2021 Sam Devlin, Raluca Georgescu, Ida Momennejad, Jaroslaw Rzepecki, Evelyn Zuniga, Gavin Costello, Guy Leroy, Ali Shaw, Katja Hofmann

A key challenge on the path to developing agents that learn complex human-like behavior is the need to quickly and accurately quantify human-likeness.

SocialAI 0.1: Towards a Benchmark to Stimulate Research on Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

no code implementations27 Apr 2021 Grgur Kovač, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer

Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI.

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL

1 code implementation17 Mar 2021 Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer

Training autonomous agents able to generalize to multiple tasks is a key target of Deep Reinforcement Learning (DRL) research.

Evaluating the Robustness of Collaborative Agents

no code implementations14 Jan 2021 Paul Knott, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, A. D. Dragan, Rohin Shah

We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness.

Meta Automatic Curriculum Learning

no code implementations16 Nov 2020 Rémy Portelas, Clément Romac, Katja Hofmann, Pierre-Yves Oudeyer

In such complex task spaces, it is essential to rely on some form of Automatic Curriculum Learning (ACL) to adapt the task sampling distribution to a given learning agent, instead of randomly sampling tasks, as many could end up being either trivial or unfeasible.

Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

1 code implementation2 Oct 2020 Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson

To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep.

Meta-Learning Meta Reinforcement Learning +2

"It's Unwieldy and It Takes a Lot of Time." Challenges and Opportunities for Creating Agents in Commercial Games

no code implementations1 Sep 2020 Mikhail Jacob, Sam Devlin, Katja Hofmann

We compare with literature from the research community that address the challenges identified and conclude by highlighting promising directions for future research supporting agent creation in the games industry.

Guaranteeing Reproducibility in Deep Learning Competitions

no code implementations12 May 2020 Brandon Houghton, Stephanie Milani, Nicholay Topin, William Guss, Katja Hofmann, Diego Perez-Liebana, Manuela Veloso, Ruslan Salakhutdinov

To encourage the development of methods with reproducible and robust training behavior, we propose a challenge paradigm where competitors are evaluated directly on the performance of their learning procedures rather than pre-trained agents.

AMRL: Aggregated Memory For Reinforcement Learning

no code implementations ICLR 2020 Jacob Beck, Kamil Ciosek, Sam Devlin, Sebastian Tschiatschek, Cheng Zhang, Katja Hofmann

In many partially observable scenarios, Reinforcement Learning (RL) agents must rely on long-term memory in order to learn an optimal policy.

reinforcement-learning Reinforcement Learning (RL)

SpatialSim: Recognizing Spatial Configurations of Objects with Graph Neural Networks

no code implementations9 Apr 2020 Laetitia Teodorescu, Katja Hofmann, Pierre-Yves Oudeyer

Recognizing precise geometrical configurations of groups of objects is a key capability of human spatial cognition, yet little studied in the deep learning literature so far.

Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning

no code implementations7 Apr 2020 Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer

A major challenge in the Deep RL (DRL) community is to train agents able to generalize over unseen situations, which is often approached by training them on a diversity of tasks (or environments).

Automatic Curriculum Learning For Deep RL: A Short Survey

no code implementations10 Mar 2020 Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, Pierre-Yves Oudeyer

Automatic Curriculum Learning (ACL) has become a cornerstone of recent successes in Deep Reinforcement Learning (DRL). These methods shape the learning trajectories of agents by challenging them with tasks adapted to their capacities.

reinforcement-learning Reinforcement Learning (RL)

Better Exploration with Optimistic Actor Critic

1 code implementation NeurIPS 2019 Kamil Ciosek, Quan Vuong, Robert Loftin, Katja Hofmann

To address both of these phenomena, we introduce a new algorithm, Optimistic Actor Critic, which approximates a lower and upper confidence bound on the state-action value function.

Continuous Control Efficient Exploration

Better Exploration with Optimistic Actor-Critic

no code implementations28 Oct 2019 Kamil Ciosek, Quan Vuong, Robert Loftin, Katja Hofmann

To address both of these phenomena, we introduce a new algorithm, Optimistic Actor Critic, which approximates a lower and upper confidence bound on the state-action value function.

Continuous Control Efficient Exploration

Variational Integrator Networks for Physically Structured Embeddings

1 code implementation21 Oct 2019 Steindor Saemundsson, Alexander Terenin, Katja Hofmann, Marc Peter Deisenroth

Learning workable representations of dynamical systems is becoming an increasingly important problem in a number of application areas.

Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

2 code implementations16 Oct 2019 Rémy Portelas, Cédric Colas, Katja Hofmann, Pierre-Yves Oudeyer

We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments.

Combining No-regret and Q-learning

1 code implementation7 Oct 2019 Ian A. Kash, Michael Sullins, Katja Hofmann

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall.

counterfactual Q-Learning

Near-Optimal Online Egalitarian learning in General Sum Repeated Matrix Games

no code implementations4 Jun 2019 Aristide Tossou, Christos Dimitrakakis, Jaroslaw Rzepecki, Katja Hofmann

We study two-player general sum repeated finite games where the rewards of each player are generated from an unknown distribution.

The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors

1 code implementation22 Apr 2019 William H. Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, Phillip Wang

To that end, we introduce: (1) the Minecraft ObtainDiamond task, a sequential decision making environment requiring long-term planning, hierarchical control, and efficient exploration methods; and (2) the MineRL-v0 dataset, a large-scale collection of over 60 million state-action pairs of human demonstrations that can be resimulated into embodied trajectories with arbitrary modifications to game state and visuals.

Decision Making Efficient Exploration +2

The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition

2 code implementations23 Jan 2019 Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noburu Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita

Learning in multi-agent scenarios is a fruitful research direction, but current approaches still show scalability problems in multiple games with general reward settings and different opponent types.

Multi-agent Reinforcement Learning reinforcement-learning +1

Fast Context Adaptation via Meta-Learning

1 code implementation8 Oct 2018 Luisa M. Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson

We propose CAVIA for meta-learning, a simple extension to MAML that is less prone to meta-overfitting, easier to parallelise, and more interpretable.

General Classification Meta-Learning +3

CAML: Fast Context Adaptation via Meta-Learning

no code implementations27 Sep 2018 Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson

We propose CAML, a meta-learning method for fast adaptation that partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks.


Depth and nonlinearity induce implicit exploration for RL

no code implementations29 May 2018 Justas Dauparas, Ryota Tomioka, Katja Hofmann

The question of how to explore, i. e., take actions with uncertain outcomes to learn about possible future rewards, is a key question in reinforcement learning (RL).

Q-Learning reinforcement-learning +1

Variational Inference for Data-Efficient Model Learning in POMDPs

no code implementations23 May 2018 Sebastian Tschiatschek, Kai Arulkumaran, Jan Stühmer, Katja Hofmann

In this paper we propose DELIP, an approach to model learning for POMDPs that utilizes amortized structured variational inference.

Decision Making Decision Making Under Uncertainty +2

Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning

no code implementations9 May 2018 Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft

We use an adversarial discriminator and train our neural ranking model on a small set of domains.

Information Retrieval

Meta Reinforcement Learning with Latent Variable Gaussian Processes

no code implementations20 Mar 2018 Steindór Sæmundsson, Katja Hofmann, Marc Peter Deisenroth

Learning from small data sets is critical in many practical applications where data collection is time consuming or expensive, e. g., robotics, animal experiments or drug design.

Gaussian Processes Meta-Learning +5

The Atari Grand Challenge Dataset

2 code implementations31 May 2017 Vitaly Kurin, Sebastian Nowozin, Katja Hofmann, Lucas Beyer, Bastian Leibe

Recent progress in Reinforcement Learning (RL), fueled by its combination, with Deep Learning has enabled impressive results in learning to interact with complex virtual environments, yet real-world applications of RL are still scarce.

Imitation Learning Reinforcement Learning (RL)

Memory Lens: How Much Memory Does an Agent Use?

no code implementations21 Nov 2016 Christoph Dann, Katja Hofmann, Sebastian Nowozin

The study of memory as information that flows from the past to the current action opens avenues to understand and improve successful reinforcement learning algorithms.

reinforcement-learning Reinforcement Learning (RL)

A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games

no code implementations21 Nov 2016 Felix Leibfried, Nate Kushman, Katja Hofmann

Reinforcement learning is concerned with identifying reward-maximizing behaviour policies in environments that are initially unknown.

Atari Games Model-based Reinforcement Learning +2

Experimental and causal view on information integration in autonomous agents

no code implementations14 Jun 2016 Philipp Geiger, Katja Hofmann, Bernhard Schölkopf

The amount of digitally available but heterogeneous information about the world is remarkable, and new technologies such as self-driving cars, smart homes, or the internet of things may further increase it.

Decision Making Self-Driving Cars +1

Contextual Dueling Bandits

no code implementations23 Feb 2015 Miroslav Dudík, Katja Hofmann, Robert E. Schapire, Aleksandrs Slivkins, Masrour Zoghi

The first of these algorithms achieves particularly low regret, even when data is adversarial, although its time and space requirements are linear in the size of the policy space.

Cannot find the paper you are looking for? You can Submit a new open access paper.