Learning Action-Effect Dynamics from Pairs of Scene-graphs

no code implementations • 7 Dec 2022 • Shailaja Keyur Sampat, Pratyay Banerjee, Yezhou Yang, Chitta Baral

'Actions' play a vital role in how humans interact with the world.

Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task

1 code implementation • 7 Dec 2022 • Shailaja Keyur Sampat, Pratyay Banerjee, Yezhou Yang, Chitta Baral

'Actions' play a vital role in how humans interact with the world.

Graph Question Answering Question Answering

Reasoning about Actions over Visual and Linguistic Modalities: A Survey

no code implementations • 15 Jul 2022 • Shailaja Keyur Sampat, Maitreya Patel, Subhasish Das, Yezhou Yang, Chitta Baral

'Actions' play a vital role in how humans interact with the world and enable them to achieve desired goals.

Common Sense Reasoning

Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

7 code implementations • 16 Apr 2022 • Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi, Daniel Khashabi

This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.

Benchmarking Instruction Following

CLEVR\_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images

1 code implementation • NAACL 2021 • Shailaja Keyur Sampat, Akshay Kumar, Yezhou Yang, Chitta Baral

Most existing research on visual question answering (VQA) is limited to information explicitly present in an image or a video.

Question Answering Visual Question Answering

CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images

1 code implementation • 13 Apr 2021 • Shailaja Keyur Sampat, Akshay Kumar, Yezhou Yang, Chitta Baral

Most existing research on visual question answering (VQA) is limited to information explicitly present in an image or a video.

Question Answering Visual Question Answering

`Just because you are right, doesn't mean I am wrong': Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks

no code implementations • EACL 2021 • Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral

GQA (CITATION) is a dataset for real-world visual reasoning and compositional question answering.

Question Answering Visual Question Answering +1

'Just because you are right, doesn't mean I am wrong': Overcoming a Bottleneck in the Development and Evaluation of Open-Ended Visual Question Answering (VQA) Tasks

1 code implementation • 28 Mar 2021 • Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral

GQA~\citep{hudson2019gqa} is a dataset for real-world visual reasoning and compositional question answering.

Question Answering Visual Question Answering +1

Visuo-Linguistic Question Answering (VLQA) Challenge

1 code implementation • Findings of the Association for Computational Linguistics 2020 • Shailaja Keyur Sampat, Yezhou Yang, Chitta Baral

Understanding images and text together is an important aspect of cognition and building advanced Artificial Intelligence (AI) systems.

Question Answering Reading Comprehension +1