Search Results for author:
Found 17 papers, 1 papers with code
Decoding, Fast and Slow: A Case Study on Balancing Trade-Offs in Incremental, Character-level Pragmatic Reasoning
Recent work has adopted models of pragmatic reasoning for the generation of informative language in, e. g., image captioning.
Challenges in Designing Natural Language Interfaces for Complex Visual Models
Intuitive interaction with visual models becomes an increasingly important task in the field of Visualization (VIS) and verbal interaction represents a significant aspect of it.
The Why and The How: A Survey on Natural Language Interaction in Visualization
In this survey, we provide an overview of natural language-based interaction in the research area of visualization.
Decoupling Pragmatics: Discriminative Decoding for Referring Expression Generation
The shift to neural models in Referring Expression Generation (REG) has enabled more natural set-ups, but at the cost of interpretability.
What Did This Castle Look like before? Exploring Referential Relations in Naturally Occurring Multimodal Texts
Multi-modal texts are abundant and diverse in structure, yet Language & Vision research of these naturally occurring texts has mostly focused on genres that are comparatively light on text, like tweets.
Exploring Semantic Spaces for Detecting Clustering and Switching in Verbal Fluency
In this work, we explore the fitness of various word/concept representations in analyzing an experimental verbal fluency dataset providing human responses to 10 different category enumeration tasks.
Exploring Text Recombination for Automatic Narrative Level Detection
Automatizing the process of understanding the global narrative structure of long texts and stories is still a major challenge for state-of-the-art natural language understanding systems, particularly because annotated data is scarce and existing annotation workflows do not scale well to the annotation of complex narrative phenomena.
Diversity as a By-Product: Goal-oriented Language Generation Leads to Linguistic Variation
The ability for variation in language use is necessary for speakers to achieve their conversational goals, for instance when referring to objects in visual environments.
From “Before” to “After”: Generating Natural Language Instructions from Image Pairs in a Simple Visual Domain
We find that it is, and investigate several simple baselines, taking these from the related task of image captioning.
Resilience through Scene Context in Visual Referring Expression Generation
Scene context is well known to facilitate humans' perception of visible objects.
Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions
Existing language and vision models achieve impressive performance in image-text understanding.
Enabling Robots to Draw and Tell: Towards Visually Grounded Multimodal Description Generation
Socially competent robots should be equipped with the ability to perceive the world that surrounds them and communicate about it in a human-like manner.
MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment
Building computer systems that can converse about their visual environment is one of the oldest concerns of research in Artificial Intelligence and Computational Linguistics (see, for example, Winograd's 1972 SHRDLU system).
Know What You Don't Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories
Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories.
The Code2Text Challenge: Text Generation in Source Code Libraries
We propose a new shared task for tactical data-to-text generation in the domain of source code libraries.
