Search Results for author:
Found 23 papers, 6 papers with code
Analyzing Speaker Strategy in Referential Communication
We analyze a corpus of referential communication through the lens of quantitative models of speaker reasoning.
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
The ability to interact with machines using natural human language is becoming not just commonplace, but expected.
Socially Cognizant Robotics for a Technology Enhanced Society
Emerging applications of robotics, and concerns about their impact, require the research community to put human-centric objectives front-and-center.
Investigating Reinforcement Learning for Communication Strategies in a Task-Initiative Setting
Many conversational domains require the system to present nuanced information to users.
Zero-shot Cross-Linguistic Learning of Event Semantics
Typologically diverse languages offer systems of lexical and grammatical aspect that allow speakers to focus on facets of event structure in ways that comport with the specific communicative setting and discourse constraints they face.
Cross-Modal Coherence for Text-to-Image Retrieval
Common image-text joint understanding techniques presume that images and the associated text can universally be characterized by a single implicit model.
COSMic: A Coherence-Aware Generation Metric for Image Descriptions
Developers of text generation models rely on automated evaluation metrics as a stand-in for slow and expensive manual evaluations.
Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue
In many domains, dialogue systems need to work collaboratively with users to successfully reconstruct the meaning the user had in mind.
Aspectuality Across Genre: A Distributional Semantics Approach
The interpretation of the lexical aspect of verbs in English plays a crucial role for recognizing textual entailment and learning discourse-level inferences.
Discourse Coherence, Reference Grounding and Goal Oriented Dialogue
Prior approaches to realizing mixed-initiative human--computer referential communication have adopted information-state or collaborative problem-solving approaches.
Cross-modal Coherence Modeling for Caption Generation
We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.
Achieving Common Ground in Multi-modal Dialogue
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality.
Clue: Cross-modal Coherence Modeling for Caption Generation
We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.
That and There: Judging the Intent of Pointing Actions with Robotic Arms
This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature.
AI2D-RST: A multimodal corpus of 1000 primary school science diagrams
This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology.
``Caption'' as a Coherence Relation: Evidence and Implications
We study verbs in image{--}text corpora, contrasting \textit{caption} corpora, where texts are explicitly written to characterize image content, with \textit{depiction} corpora, where texts and images may stand in more general relations.
CITE: A Corpus of Image-Text Discourse Relations
This paper presents a novel crowd-sourced resource for multimodal discourse: our resource characterizes inferences in image-text contexts in the domain of cooking recipes in the form of coherence relations.
Syntactic realization with data-driven neural tree grammars
A key component in surface realization in natural language generation is to choose concrete syntactic relationships to express a target meaning.
A Bayesian Model of Grounded Color Semantics
Natural language meanings allow speakers to encode important real-world distinctions, but corpora of grounded language use also reveal that speakers categorize the world in different ways and describe situations with different terminology.
