1 code implementation • WS 2019 • Kris Korrel, Dieuwke Hupkes, Verna Dankers, Elia Bruni
While sequence-to-sequence models have shown remarkable generalization power across several natural language tasks, their construct of solutions are argued to be less compositional than human-like generalization.
1 code implementation • 22 Aug 2019 • Dieuwke Hupkes, Verna Dankers, Mathijs Mul, Elia Bruni
Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack of agreement about what it means for a neural model to be compositional.
1 code implementation • EMNLP 2020 • Oskar van der Wal, Silvan de Boer, Elia Bruni, Dieuwke Hupkes
In this paper, we consider the syntactic properties of languages emerged in referential games, using unsupervised grammar induction (UGI) techniques originally designed to analyse natural language.
1 code implementation • ACL 2022 • Verna Dankers, Elia Bruni, Dieuwke Hupkes
Obtaining human-like performance in NLP is often argued to require compositional generalisation.
1 code implementation • COLING 2022 • Xenia Ohmer, Marko Duda, Elia Bruni
We develop a novel communication game, the hierarchical reference game, to study the emergence of such reference systems in artificial agents.
3 code implementations • NAACL 2019 • Ravi Shekhar, Aashish Venkatesh, Tim Baumgärtner, Elia Bruni, Barbara Plank, Raffaella Bernardi, Raquel Fernández
We compare our approach to an alternative system which extends the baseline with reinforcement learning.
1 code implementation • EACL 2021 • Gautier Dagan, Dieuwke Hupkes, Elia Bruni
However, they do not take into account a second constraint considered to be fundamental for the shape of human language: that it must be learnable by new language learners.
1 code implementation • WS 2019 • Dennis Ulmer, Dieuwke Hupkes, Elia Bruni
Since their inception, encoder-decoder models have successfully been applied to a wide array of problems in computational linguistics.
1 code implementation • 18 Apr 2024 • Xenia Ohmer, Elia Bruni, Dieuwke Hupkes
The staggering pace with which the capabilities of large language models (LLMs) are increasing, as measured by a range of commonly used natural language understanding (NLU) benchmarks, raises many questions regarding what "understanding" means for a language model and how it compares to human understanding.
1 code implementation • COLING 2018 • Ravi Shekhar, Tim Baumgartner, Aashish Venkatesh, Elia Bruni, Raffaella Bernardi, Raquel Fernandez
We make initial steps towards this general goal by augmenting a task-oriented visual dialogue model with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess.
1 code implementation • WS 2019 • Rezka Leonandya, Elia Bruni, Dieuwke Hupkes, Germán Kruszewski
Learning to follow human instructions is a long-pursued goal in artificial intelligence.
no code implementations • 20 May 2018 • Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski, Elia Bruni
While neural network models have been successfully applied to domains that require substantial generalisation skills, recent studies have implied that they struggle when solving the task they are trained on requires inferring its underlying compositional structure.
no code implementations • 1 Mar 2016 • Korsuk Sirinukunwattana, Josien P. W. Pluim, Hao Chen, Xiaojuan Qi, Pheng-Ann Heng, Yun Bo Guo, Li Yang Wang, Bogdan J. Matuszewski, Elia Bruni, Urko Sanchez, Anton Böhm, Olaf Ronneberger, Bassem Ben Cheikh, Daniel Racoceanu, Philipp Kainz, Michael Pfeiffer, Martin Urschler, David R. J. Snead, Nasir M. Rajpoot
Colorectal adenocarcinoma originating in intestinal glandular structures is the most common form of colon cancer.
no code implementations • WS 2017 • Elia Bruni, Raquel Fern{\'a}ndez
We investigate the potential of adversarial evaluation methods for open-domain dialogue generation systems, comparing the performance of a discriminative agent to that of humans on the same task.
no code implementations • ACL 2019 • Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, Raquel Fernández
This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation.
no code implementations • WS 2019 • Joris Baan, Jana Leible, Mitja Nikolaus, David Rau, Dennis Ulmer, Tim Baumgärtner, Dieuwke Hupkes, Elia Bruni
We present a detailed comparison of two types of sequence to sequence models trained to conduct a compositional task.
no code implementations • 14 Aug 2019 • Mathijs Mul, Diane Bouchacourt, Elia Bruni
A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language instructions.
no code implementations • WS 2019 • Benjamin Kolb, Leon Lang, Henning Bartsch, Arwin Gansekoele, Raymond Koopmanschap, Leonardo Romor, David Speck, Mathijs Mul, Elia Bruni
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent.
no code implementations • ACL 2020 • Yann Dubois, Gautier Dagan, Dieuwke Hupkes, Elia Bruni
We hypothesize that models with a separate content- and location-based attention are more likely to extrapolate than those with common attention mechanisms.
no code implementations • 11 Dec 2019 • Benjamin Kolb, Leon Lang, Henning Bartsch, Arwin Gansekoele, Raymond Koopmanschap, Leonardo Romor, David Speck, Mathijs Mul, Elia Bruni
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent.
no code implementations • 6 Jan 2020 • Thomas A. Unger, Elia Bruni
We converted the recently developed BabyAI grid world platform to a sender/receiver setup in order to test the hypothesis that established deep reinforcement learning techniques are sufficient to incentivize the emergence of a grounded discrete communication protocol between generalized agents.
no code implementations • 13 Jan 2020 • Michiel van der Meer, Matteo Pirotta, Elia Bruni
In this work, we present an alternative approach to making an agent compositional through the use of a diagnostic classifier.
no code implementations • 23 Jan 2020 • Bence Keresztury, Elia Bruni
Recent findings in multi-agent deep learning systems point towards the emergence of compositional languages.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Diana Rodríguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, Elia Bruni
In previous work, artificial agents were shown to achieve almost perfect accuracy in referential games where they have to communicate to identify images.
no code implementations • EACL 2021 • Lucas Weber, Jaap Jumelet, Elia Bruni, Dieuwke Hupkes
In this paper, we propose to study language modelling as a multi-task problem, bringing together three strands of research: multi-task learning, linguistics, and interpretability.
1 code implementation • 19 May 2023 • Xenia Ohmer, Elia Bruni, Dieuwke Hupkes
At the staggering pace with which the capabilities of large language models (LLMs) are increasing, creating future-proof evaluation sets to assess their understanding becomes more and more challenging.
no code implementations • 23 Aug 2023 • Lucas Weber, Jaap Jumelet, Paul Michel, Elia Bruni, Dieuwke Hupkes
We present a number of different case studies with different common hand-crafted and automated CL approaches to illustrate this phenomenon, and we find that none of them outperforms optimisation with only Adam with well-chosen hyperparameters.
no code implementations • 20 Oct 2023 • Lucas Weber, Elia Bruni, Dieuwke Hupkes
Finding the best way of adapting pre-trained language models to a task is a big challenge in current NLP.
no code implementations • 15 Nov 2023 • Serwan Jassim, Mario Holubar, Annika Richter, Cornelius Wolff, Xenia Ohmer, Elia Bruni
Our evaluation reveals significant shortcomings in the language grounding and intuitive physics capabilities of these models.
no code implementations • 8 Dec 2023 • Lucas Weber, Elia Bruni, Dieuwke Hupkes
Just like the previous generation of task-tuned models, large language models (LLMs) that are adapted to tasks via prompt-based methods like in-context-learning (ICL) perform well in some setups but not in others.