Search Results for author:
Found 6 papers, 4 papers with code
Bidirectional Emergent Language in Situated Environments
Emergent language research has made significant progress in recent years, but still largely fails to explore how communication emerges in more complex and situated multi-agent systems.
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
The staggering pace with which the capabilities of large language models (LLMs) are increasing, as measured by a range of commonly used natural language understanding (NLU) benchmarks, raises many questions regarding what "understanding" means for a language model and how it compares to human understanding.
GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language models
Our evaluation reveals significant shortcomings in the language grounding and intuitive physics capabilities of these models.
On the Relationship between Skill Neurons and Robustness in Prompt Tuning
We show that prompts tuned for a specific task are transferable to tasks of the same type but are not very robust to adversarial data.
Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses
At the staggering pace with which the capabilities of large language models (LLMs) are increasing, creating future-proof evaluation sets to assess their understanding becomes more and more challenging.
Emergence of hierarchical reference systems in multi-agent communication
We develop a novel communication game, the hierarchical reference game, to study the emergence of such reference systems in artificial agents.
