Search Results for author: Matthew Stone

Found 17 papers, 4 papers with code

COSMic: A Coherence-Aware Generation Metric for Image Descriptions

no code implementations11 Sep 2021 Mert İnan, Piyush Sharma, Baber Khalid, Radu Soricut, Matthew Stone, Malihe Alikhani

Developers of text generation models rely on automated evaluation metrics as a stand-in for slow and expensive manual evaluations.

Image Captioning Text Generation

Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue

no code implementations COLING 2020 Baber Khalid, Malihe Alikhani, Matthew Stone

In many domains, dialogue systems need to work collaboratively with users to successfully reconstruct the meaning the user had in mind.

Aspectuality Across Genre: A Distributional Semantics Approach

no code implementations COLING 2020 Thomas Kober, Malihe Alikhani, Matthew Stone, Mark Steedman

The interpretation of the lexical aspect of verbs in English plays a crucial role for recognizing textual entailment and learning discourse-level inferences.

Natural Language Inference

Discourse Coherence, Reference Grounding and Goal Oriented Dialogue

no code implementations8 Jul 2020 Baber Khalid, Malihe Alikhani, Michael Fellner, Brian McMahan, Matthew Stone

Prior approaches to realizing mixed-initiative human--computer referential communication have adopted information-state or collaborative problem-solving approaches.

Achieving Common Ground in Multi-modal Dialogue

no code implementations ACL 2020 Malihe Alikhani, Matthew Stone

All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality.

Cross-modal Coherence Modeling for Caption Generation

no code implementations ACL 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

Image Captioning

Clue: Cross-modal Coherence Modeling for Caption Generation

no code implementations2 May 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

Image Captioning

That and There: Judging the Intent of Pointing Actions with Robotic Arms

1 code implementation13 Dec 2019 Malihe Alikhani, Baber Khalid, Rahul Shome, Chaitanya Mitash, Kostas Bekris, Matthew Stone

This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature.

Common Sense Reasoning

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams

no code implementations9 Dec 2019 Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman

This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology.

Question Answering Visual Question Answering

``Caption'' as a Coherence Relation: Evidence and Implications

no code implementations WS 2019 Malihe Alikhani, Matthew Stone

We study verbs in image{--}text corpora, contrasting \textit{caption} corpora, where texts are explicitly written to characterize image content, with \textit{depiction} corpora, where texts and images may stand in more general relations.

Image Retrieval

CITE: A Corpus of Image-Text Discourse Relations

1 code implementation NAACL 2019 Malihe Alikhani, Sreyasi Nag Chowdhury, Gerard de Melo, Matthew Stone

This paper presents a novel crowd-sourced resource for multimodal discourse: our resource characterizes inferences in image-text contexts in the domain of cooking recipes in the form of coherence relations.

Common Sense Reasoning

Arrows are the Verbs of Diagrams

1 code implementation COLING 2018 Malihe Alikhani, Matthew Stone

Arrows are a key ingredient of schematic pictorial communication.

Syntactic realization with data-driven neural tree grammars

1 code implementation COLING 2016 Brian McMahan, Matthew Stone

A key component in surface realization in natural language generation is to choose concrete syntactic relationships to express a target meaning.

Language Modelling Text Generation

A Bayesian Model of Grounded Color Semantics

no code implementations TACL 2015 Brian McMahan, Matthew Stone

Natural language meanings allow speakers to encode important real-world distinctions, but corpora of grounded language use also reveal that speakers categorize the world in different ways and describe situations with different terminology.

Cannot find the paper you are looking for? You can Submit a new open access paper.