Search Results for author: Sanjay Subramanian

Found 18 papers, 8 papers with code

Pose Priors from Language Models

no code implementations6 May 2024 Sanjay Subramanian, Evonne Ng, Lea Müller, Dan Klein, Shiry Ginosar, Trevor Darrell

We present a zero-shot pose optimization method that enforces accurate physical contact constraints when estimating the 3D pose of humans.

Pose Estimation

TraveLER: A Multi-LMM Agent Framework for Video Question-Answering

no code implementations1 Apr 2024 Chuyi Shang, Amos You, Sanjay Subramanian, Trevor Darrell, Roei Herzig

Specifically, we propose TraveLER, a model that can create a plan to "Traverse" through the video, ask questions about individual frames to "Locate" and store key information, and then "Evaluate" if there is enough information to answer the question.

Question Answering Video Question Answering

Recursive Visual Programming

no code implementations4 Dec 2023 Jiaxin Ge, Sanjay Subramanian, Baifeng Shi, Roei Herzig, Trevor Darrell

Visual Programming (VP) has emerged as a powerful framework for Visual Question Answering (VQA).

Code Generation Question Answering +1

From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation

no code implementations21 Nov 2023 Jiaxin Ge, Sanjay Subramanian, Trevor Darrell, Boyi Li

Addressing the challenge of adapting pre-trained vision-language models for generating insightful explanations for visual reasoning tasks with limited annotations, we present ReVisE: a $\textbf{Re}$cursive $\textbf{Vis}$ual $\textbf{E}$xplanation algorithm.

Explanation Generation Visual Question Answering (VQA) +1

Can Language Models Learn to Listen?

no code implementations ICCV 2023 Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, Shiry Ginosar

We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words.

Language Modelling Large Language Model

ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension

2 code implementations ACL 2022 Sanjay Subramanian, William Merrill, Trevor Darrell, Matt Gardner, Sameer Singh, Anna Rohrbach

Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain.

Image Classification Referring Expression +1

Evaluating NLP Models via Contrast Sets

no code implementations1 Oct 2020 Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, A. Zhang, Ben Zhou

Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.

Reading Comprehension Sentiment Analysis

Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering

1 code implementation1 Jul 2020 Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan Berant

However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples.

Inductive Bias Question Answering +1

Obtaining Faithful Interpretations from Compositional Neural Networks

1 code implementation ACL 2020 Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner

Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the compositional structure of the problem in the network architecture.

AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models

1 code implementation IJCNLP 2019 Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Subramanian, Matt Gardner, Sameer Singh

Neural NLP models are increasingly accurate but are imperfect and opaque---they break in counterintuitive ways and leave end users puzzled at their behavior.

Language Modelling Masked Language Modeling +1

An Improved Neural Baseline for Temporal Relation Extraction

no code implementations IJCNLP 2019 Qiang Ning, Sanjay Subramanian, Dan Roth

Determining temporal relations (e. g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data.

Common Sense Reasoning Natural Language Understanding +3

Correlation Clustering with Same-Cluster Queries Bounded by Optimal Cost

1 code implementation14 Aug 2019 Barna Saha, Sanjay Subramanian

An algorithm in this setting has access to an oracle with full knowledge of an optimal clustering, and the algorithm can ask the oracle queries of the form, "Does the optimal clustering put vertices $ u $ and $ v $ in the same cluster?"

Clustering Graph Clustering

Improving Generalization in Coreference Resolution via Adversarial Training

no code implementations SEMEVAL 2019 Sanjay Subramanian, Dan Roth

In order for coreference resolution systems to be useful in practice, they must be able to generalize to new text.


Evaluation of named entity coreference

no code implementations WS 2019 Oshin Agarwal, Sanjay Subramanian, Ani Nenkova, Dan Roth

It is therefore important that coreference resolution systems are able to link these different types of mentions to the correct entity name.


Named Person Coreference in English News

no code implementations26 Oct 2018 Oshin Agarwal, Sanjay Subramanian, Ani Nenkova, Dan Roth

Here, we evaluate two state of the art coreference resolution systems on the subtask of Named Person Coreference, in which we are interested in identifying a person mentioned by name, along with all other mentions of the person, by pronoun or generic noun phrase.

coreference-resolution named-entity-recognition +2

Cannot find the paper you are looking for? You can Submit a new open access paper.