Search Results for author: Matthew Stone

Found 22 papers, 6 papers with code

CITE: A Corpus of Image-Text Discourse Relations

1 code implementation NAACL 2019 Malihe Alikhani, Sreyasi Nag Chowdhury, Gerard de Melo, Matthew Stone

This paper presents a novel crowd-sourced resource for multimodal discourse: our resource characterizes inferences in image-text contexts in the domain of cooking recipes in the form of coherence relations.

Common Sense Reasoning

Syntactic realization with data-driven neural tree grammars

1 code implementation COLING 2016 Brian McMahan, Matthew Stone

A key component in surface realization in natural language generation is to choose concrete syntactic relationships to express a target meaning.

Language Modelling Text Generation

That and There: Judging the Intent of Pointing Actions with Robotic Arms

1 code implementation13 Dec 2019 Malihe Alikhani, Baber Khalid, Rahul Shome, Chaitanya Mitash, Kostas Bekris, Matthew Stone

This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature.

Common Sense Reasoning

Cross-Modal Coherence for Text-to-Image Retrieval

1 code implementation22 Sep 2021 Malihe Alikhani, Fangda Han, Hareesh Ravi, Mubbasir Kapadia, Vladimir Pavlovic, Matthew Stone

Common image-text joint understanding techniques presume that images and the associated text can universally be characterized by a single implicit model.

Image Retrieval Retrieval

A Bayesian Model of Grounded Color Semantics

no code implementations TACL 2015 Brian McMahan, Matthew Stone

Natural language meanings allow speakers to encode important real-world distinctions, but corpora of grounded language use also reveal that speakers categorize the world in different ways and describe situations with different terminology.

``Caption'' as a Coherence Relation: Evidence and Implications

no code implementations WS 2019 Malihe Alikhani, Matthew Stone

We study verbs in image{--}text corpora, contrasting \textit{caption} corpora, where texts are explicitly written to characterize image content, with \textit{depiction} corpora, where texts and images may stand in more general relations.

Image Retrieval Relation +1

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams

no code implementations9 Dec 2019 Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman

This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology.

Question Answering Visual Question Answering

Clue: Cross-modal Coherence Modeling for Caption Generation

no code implementations2 May 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

controllable image captioning Relation

Cross-modal Coherence Modeling for Caption Generation

no code implementations ACL 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

controllable image captioning Relation

Achieving Common Ground in Multi-modal Dialogue

no code implementations ACL 2020 Malihe Alikhani, Matthew Stone

All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality.

Discourse Coherence, Reference Grounding and Goal Oriented Dialogue

no code implementations8 Jul 2020 Baber Khalid, Malihe Alikhani, Michael Fellner, Brian McMahan, Matthew Stone

Prior approaches to realizing mixed-initiative human--computer referential communication have adopted information-state or collaborative problem-solving approaches.

reinforcement-learning Reinforcement Learning (RL)

Aspectuality Across Genre: A Distributional Semantics Approach

no code implementations COLING 2020 Thomas Kober, Malihe Alikhani, Matthew Stone, Mark Steedman

The interpretation of the lexical aspect of verbs in English plays a crucial role for recognizing textual entailment and learning discourse-level inferences.

Natural Language Inference

Analyzing Speaker Strategy in Referential Communication

no code implementations SIGDIAL (ACL) 2020 Brian McMahan, Matthew Stone

We analyze a corpus of referential communication through the lens of quantitative models of speaker reasoning.

Zero-shot Cross-Linguistic Learning of Event Semantics

no code implementations5 Jul 2022 Malihe Alikhani, Thomas Kober, Bashar Alhafni, Yue Chen, Mert Inan, Elizabeth Nielsen, Shahab Raji, Mark Steedman, Matthew Stone

Typologically diverse languages offer systems of lexical and grammatical aspect that allow speakers to focus on facets of event structure in ways that comport with the specific communicative setting and discourse constraints they face.

Socially Cognizant Robotics for a Technology Enhanced Society

no code implementations27 Oct 2023 Kristin J. Dana, Clinton Andrews, Kostas Bekris, Jacob Feldman, Matthew Stone, Pernille Hemmer, Aaron Mazzeo, Hal Salzman, Jingang Yi

Emerging applications of robotics, and concerns about their impact, require the research community to put human-centric objectives front-and-center.

Cannot find the paper you are looking for? You can Submit a new open access paper.