Search Results for author:
Found 31 papers, 20 papers with code
Ad-hoc Concept Forming in the Game Codenames as a Means for Evaluating Large Language Models
This study utilizes the game Codenames as a benchmarking tool to evaluate large language models (LLMs) with respect to specific linguistic and cognitive skills.
Plant in Cupboard, Orange on Table, Book on Shelf. Benchmarking Practical Reasoning and Situation Modelling in a Text-Simulated Situated Environment
Large language models (LLMs) have risen to prominence as 'chatbots' for users to interact via natural language.
Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programming
While there has been a lot of research recently on robots in household environments, at the present time, most robots in existence can be found on shop floors, and most interactions between humans and robots happen there.
Free-text Rationale Generation under Readability Level Control
Free-text rationales justify model decisions in natural language and thus become likable and accessible among approaches to explanation across many tasks.
Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft
In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks.
Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models
On further analysis, we find that the exceptional deep captioning capabilities of the largest models drive some of the performance.
clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents
In this paper, we take one of the proposed frameworks for setting up such game-play environments, and further test its usefulness as an evaluation instrument, along a number of dimensions: We show that it can easily keep up with new developments while avoiding data contamination, we show that the tests implemented within it are not yet saturated (human performance is substantially higher than that of even the best models), and we show that it lends itself to investigating additional questions, such as the impact of the prompting language on performance.
M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets
In recent years, multimodal natural language processing, aimed at learning from diverse data types, has garnered significant attention.
Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies
In collaborative goal-oriented settings, the participants are not only interested in achieving a successful outcome, but do also implicitly negotiate the effort they put into the interaction (by adapting to each other).
Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game
Albrecht and Stone (2018) state that modeling of changing behaviors remains an open problem "due to the essentially unconstrained nature of what other agents may do".
Unveiling Global Narratives: A Multilingual Twitter Dataset of News Media on the Russo-Ukrainian Conflict
The ongoing Russo-Ukrainian conflict has been a subject of intense media coverage worldwide.
Improving Generalization for Multimodal Fake News Detection
To reduce the bias and improve model generalization, we suggest training data augmentation to conduct more meaningful experiments for fake news detection on social media.
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks
Specifically, we investigate the performance of open-source, open-access language models against GPT-3 on five vision-language tasks when given textually-encoded visual information.
Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents"-agents that operate in rich linguistic and non-linguistic contexts-through testing them in carefully constructed interactive settings.
Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers
The ability to pick up on language signals in an ongoing interaction is crucial for future machine learning models to collaborate and interact with humans naturally.
MM-Locate-News: Multimodal Focus Location Estimation in News
In this paper, a novel dataset called Multimodal Focus Location of News (MM-Locate-News) is introduced.
MM-Claims: A Dataset for Multimodal Claim Detection in Social Media
In recent years, the problem of misinformation on the web has become widespread across languages, countries, and various social media platforms.
TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes
The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis.
Combining Textual Features for the Detection of Hateful and Offensive Language
The detection of offensive, hateful and profane language has become a critical challenge since many users in social networks are exposed to cyberbullying activities on a daily basis.
A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods
Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts.
Unsupervised Video Summarization via Multi-source Features
Our evaluation shows that we obtain state-of-the-art results on both datasets, while also highlighting the shortcomings of previous work with regard to the evaluation methodology.
GeoWINE: Geolocation based Wiki, Image,News and Event Retrieval
The first module is a state-of-the-art model for geolocation estimation of images.
Supervised Video Summarization via Multiple Feature Sets with Parallel Attention
The proposed architecture utilizes an attention mechanism before fusing motion features and features representing the (static) visual content, i. e., derived from an image classification model.
Ranked #1 on
Supervised Video Summarization
on TvSum
On the Role of Images for Analyzing Claims in Social Media
Fake news is a severe problem in social media.
Ontology-driven Event Type Classification in Images
Event classification can add valuable information for semantic search and the increasingly important topic of fact validation in news.
Classification of Important Segments in Educational Videos using Multimodal Features
Our experiments investigate the impact of visual and temporal information, as well as the combination of multimodal features on importance prediction.
Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Features
In this digital age of news consumption, a news reader has the ability to react, express and share opinions with others in a highly interactive and fast manner.
A Feature Analysis for Multimodal News Retrieval
Content-based information retrieval is based on the information contained in documents rather than using metadata such as keywords.
Evaluating Architectural Choices for Deep Learning Approaches for Question Answering over Knowledge Bases
We address the task of answering simple questions, consisting in predicting the subject and predicate of a triple given a question.
AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data
We present the first multilingual QALD pipeline that induces a model from training data for mapping a natural language question into logical form as probabilistic inference.
