Search Results for author: Yoav Artzi

Found 58 papers, 35 papers with code

Training RNNs as Fast as CNNs

1 code implementation ICLR 2018 Tao Lei, Yu Zhang, Yoav Artzi

Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.

General Classification Language Modelling +4

A Corpus for Reasoning About Natural Language Grounded in Photographs

1 code implementation ACL 2019 Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, Yoav Artzi

We crowdsource the data using sets of visually rich images and a compare-and-contrast task to elicit linguistically diverse language.

Visual Reasoning

NLVR2 Visual Bias Analysis

1 code implementation23 Sep 2019 Alane Suhr, Yoav Artzi

We show that the performance of existing models (Li et al., 2019; Tan and Bansal 2019) is relatively robust to this potential bias.

Sentence

Revisiting Few-sample BERT Fine-tuning

1 code implementation ICLR 2021 Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, Yoav Artzi

We empirically test the impact of these factors, and identify alternative practices that resolve the commonly observed instability of the process.

Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies

6 code implementations NAACL 2018 Max Grusky, Mor Naaman, Yoav Artzi

We present NEWSROOM, a summarization dataset of 1. 3 million articles and summaries written by authors and editors in newsrooms of 38 major news publications.

Cornell SPF: Cornell Semantic Parsing Framework

1 code implementation13 Nov 2013 Yoav Artzi

The Cornell Semantic Parsing Framework (SPF) is a learning and inference framework for mapping natural language to formal representation of its meaning.

Semantic Parsing

Learning to Map Context-Dependent Sentences to Executable Formal Queries

1 code implementation NAACL 2018 Alane Suhr, Srinivasan Iyer, Yoav Artzi

We propose a context-dependent model to map utterances within an interaction to executable formal queries.

CHALET: Cornell House Agent Learning Environment

2 code implementations23 Jan 2018 Claudia Yan, Dipendra Misra, Andrew Bennnett, Aaron Walsman, Yonatan Bisk, Yoav Artzi

We present CHALET, a 3D house simulator with support for navigation and manipulation.

A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution

1 code implementation12 Jul 2021 Valts Blukis, Chris Paxton, Dieter Fox, Animesh Garg, Yoav Artzi

Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents.

Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning

1 code implementation31 May 2018 Valts Blukis, Nataly Brukhim, Andrew Bennett, Ross A. Knepper, Yoav Artzi

We introduce a method for following high-level navigation instructions by mapping directly from images, instructions and pose estimates to continuous low-level velocity commands for real-time control.

Imitation Learning Instruction Following

Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction

1 code implementation10 Nov 2018 Valts Blukis, Dipendra Misra, Ross A. Knepper, Yoav Artzi

We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone.

Continuous Control Imitation Learning +2

Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight

1 code implementation21 Oct 2019 Valts Blukis, Yannick Terme, Eyvind Niklasson, Ross A. Knepper, Yoav Artzi

Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control.

Continuous Control Instruction Following +2

Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following

1 code implementation14 Nov 2020 Valts Blukis, Ross A. Knepper, Yoav Artzi

We study the problem of learning a robot policy to follow natural language instructions that can be easily extended to reason about new objects.

Continuous Control Instruction Following

CB2: Collaborative Natural Language Interaction Research Platform

1 code implementation14 Mar 2023 Jacob Sharf, Mustafa Omer Gul, Yoav Artzi

CB2 is a multi-agent platform to study collaborative natural language interaction in a grounded task-oriented scenario.

Instruction Following

Who's Waldo? Linking People Across Text and Images

1 code implementation ICCV 2021 Claire Yuqing Cui, Apoorv Khandelwal, Yoav Artzi, Noah Snavely, Hadar Averbuch-Elor

We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.

 Ranked #1 on Person-centric Visual Grounding on Who’s Waldo (using extra training data)

Person-centric Visual Grounding

Continually Improving Extractive QA via Human Feedback

1 code implementation21 May 2023 Ge Gao, Hung-Ting Chen, Yoav Artzi, Eunsol Choi

We study continually improving an extractive question answering (QA) system via human user feedback.

Domain Adaptation Extractive Question-Answering +1

Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation

1 code implementation ACL 2018 Alane Suhr, Yoav Artzi

We propose a learning approach for mapping context-dependent sequential instructions to actions.

IncDSI: Incrementally Updatable Document Retrieval

1 code implementation19 Jul 2023 Varsha Kishore, Chao Wan, Justin Lovelace, Yoav Artzi, Kilian Q. Weinberger

Differentiable Search Index is a recently proposed paradigm for document retrieval, that encodes information about a corpus of documents within the parameters of a neural network and directly maps queries to corresponding documents.

Retrieval

A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models

1 code implementation6 Sep 2023 Noriyuki Kojima, Hadar Averbuch-Elor, Yoav Artzi

Key to tasks that require reasoning about natural language in visual contexts is grounding words and phrases to image regions.

Phrase Grounding

Visual Reasoning with Natural Language

no code implementations2 Oct 2017 Stephanie Zhou, Alane Suhr, Yoav Artzi

To understand language in complex environments, agents must reason about the full range of language inputs and their correspondence to the world.

Descriptive Visual Reasoning

Early Fusion for Goal Directed Robotic Vision

no code implementations21 Nov 2018 Aaron Walsman, Yonatan Bisk, Saadia Gabriel, Dipendra Misra, Yoav Artzi, Yejin Choi, Dieter Fox

Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline.

Imitation Learning Retrieval

A Corpus of Natural Language for Visual Reasoning

no code implementations ACL 2017 Alane Suhr, Mike Lewis, James Yeh, Yoav Artzi

We present a new visual reasoning language dataset, containing 92, 244 pairs of examples of natural statements grounded in synthetic images with 3, 962 unique sentences.

Question Answering Visual Question Answering (VQA) +1

Executing Instructions in Situated Collaborative Interactions

no code implementations IJCNLP 2019 Alane Suhr, Claudia Yan, Charlotte Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, Yoav Artzi

We study a collaborative scenario where a user not only instructs a system to complete tasks, but also acts alongside it.

Evaluating NLP Models via Contrast Sets

no code implementations1 Oct 2020 Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, A. Zhang, Ben Zhou

Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.

Reading Comprehension Sentiment Analysis

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior

no code implementations10 Aug 2021 Noriyuki Kojima, Alane Suhr, Yoav Artzi

We study continual learning for natural language instruction generation, by observing human users' instruction execution.

Continual Learning

Analysis of Language Change in Collaborative Instruction Following

1 code implementation Findings (EMNLP) 2021 Anna Effenberger, Eva Yan, Rhia Singh, Alane Suhr, Yoav Artzi

We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise.

Instruction Following

Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection

no code implementations EMNLP (ACL) 2021 Alane Suhr, Clara Vania, Nikita Nangia, Maarten Sap, Mark Yatskar, Samuel R. Bowman, Yoav Artzi

Even though it is such a fundamental tool in NLP, crowdsourcing use is largely guided by common practices and the personal experience of researchers.

Abstract Visual Reasoning with Tangram Shapes

no code implementations29 Nov 2022 Anya Ji, Noriyuki Kojima, Noah Rush, Alane Suhr, Wai Keen Vong, Robert D. Hawkins, Yoav Artzi

We introduce KiloGram, a resource for studying abstract visual reasoning in humans and machines.

Visual Reasoning

Continual Learning for Instruction Following from Realtime Feedback

1 code implementation NeurIPS 2023 Alane Suhr, Yoav Artzi

We propose and deploy an approach to continually train an instruction-following agent from feedback provided by users during collaborative interactions.

Continual Learning Instruction Following

Semantic uncertainty guides the extension of conventions to new referents

no code implementations11 May 2023 Ron Eliav, Anya Ji, Yoav Artzi, Robert D. Hawkins

A long tradition of studies in psycholinguistics has examined the formation and generalization of ad hoc conventions in reference games, showing how newly acquired conventions for a given target transfer to new referential contexts.

A Surprising Failure? Multimodal LLMs and the NLVR Challenge

no code implementations26 Feb 2024 Anne Wu, Kianté Brantley, Yoav Artzi

This study evaluates three state-of-the-art MLLMs -- GPT-4V, Gemini Pro, and the open-source model IDEFICS -- on the compositional natural language vision reasoning task NLVR.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.