Search Results for author:
Found 40 papers, 17 papers with code
Relevant CommonSense Subgraphs for “What if...” Procedural Reasoning
We study the challenge of learning causal reasoning over procedural text to answer “What if...” questions when external commonsense knowledge is required.
Explicit Object Relation Alignment for Vision and Language Navigation
In this paper, we investigate the problem of vision and language navigation.
GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning
In this work, we propose GIP-COL (Graph-Injected Soft Prompting for COmpositional Learning) to better explore the compositional zero-shot learning (CZSL) ability of VLMs within the prompt-based learning framework.
Syntax-Guided Transformers: Elevating Compositional Generalization and Grounding in Multimodal Environments
Compositional generalization, the ability of intelligent models to extrapolate understanding of components to novel compositions, is a fundamental yet challenging facet in AI research, especially within multimodal environments.
MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition
Inspired by this observation, in this paper, we propose MetaReVision, a retrieval-enhanced meta-learning model to address the visually grounded compositional concept learning problem.
Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning
Spatial reasoning over text is challenging as the models not only need to extract the direct spatial information from the text but also reason over those and infer implicit spatial relations.
Teaching Probabilistic Logical Reasoning to Transformers
In this work, we analyze the use of probabilistic logical rules in transformer-based language models.
VLN-Trans: Translator for the Vision and Language Navigation Agent
The mentioned landmarks are not recognizable by the navigation agent due to the different vision abilities of the instructor and the modeled agent.
GLUECons: A Generic Benchmark for Learning Under Constraints
Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models.
The Role of Semantic Parsing in Understanding Procedural Text
In this paper, we investigate whether symbolic semantic representations, extracted from deep semantic parsers, can help reasoning over the states of involved entities in a procedural text.
Using Persuasive Writing Strategies to Explain and Detect Health Misinformation
The spread of misinformation is a prominent problem in today's society, and many researchers in academia and industry are trying to combat it.
Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning
This work explores the zero-shot compositional learning ability of large pre-trained vision-language models(VLMs) within the prompt-based learning framework and propose a model (\textit{PromptCompVL}) to solve the compositonal zero-shot learning (CZSL) problem.
Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning
Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains.
LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation
Understanding spatial and visual information is essential for a navigation agent who follows natural language instructions.
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering
DRGN operates on a given KG subgraph based on the question and answers entities and uses the relevance scores between the nodes to establish new edges dynamically for learning node representations in the graph network.
Relevant CommonSense Subgraphs for "What if..." Procedural Reasoning
We study the challenge of learning causal reasoning over procedural text to answer "What if..." questions when external commonsense knowledge is required.
DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning
We demonstrate a library for the integration of domain knowledge in deep learning architectures.
Zero-Shot Compositional Concept Learning
In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework.
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning
This paper proposes a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models (LM).
Relational Gating for "What If" Reasoning
We propose a novel relational gating network that learns to filter the key entities and relationships and learns contextual and cross representations of both procedure and question for finding the answer.
Towards Navigation by Reasoning over Spatial Configurations
Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.
Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events
This enables us to use pre-trained transformer-based language models on other QA benchmarks by adapting those to the procedural text understanding.
Ranked #1 on
Procedural Text Understanding
on ProPara
Latent Alignment of Procedural Concepts in Multimodal Recipes
We propose a novel alignment mechanism to deal with procedural reasoning on a newly released multimodal QA dataset, named RecipeQA.
Ranked #1 on
Question Answering
on RecipeQA
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks
Understating spatial semantics expressed in natural language can become highly complex in real-world applications.
SRLGRN: Semantic Role Labeling Graph Reasoning Network
This work deals with the challenge of learning and reasoning over multi-hop question answering (QA).
From Spatial Relations to Spatial Configurations
To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.
Cross-Modality Relevance for Reasoning on Language and Vision
This work deals with the challenge of learning and reasoning over language and vision data for the related downstream tasks such as visual question answering (VQA) and natural language for visual reasoning (NLVR).
Declarative Learning-Based Programming as an Interface to AI Systems
Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry.
Anaphora Resolution for Improving Spatial Relation Extraction from Text
Spatial relation extraction from generic text is a challenging problem due to the ambiguity of the prepositions spatial meaning as well as the nesting structure of the spatial descriptions.
Visually Guided Spatial Relation Extraction from Text
Extraction of spatial relations from sentences with complex/nesting relationships is very challenging as often needs resolving inherent semantic ambiguities.
Spatial Language Understanding with Multimodal Graphs using Declarative Learning based Programming
The DeLBP framework facilitates combining modalities and representing various data in a unified graph.
Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks
In particular, we provide an initial prototype for a relational and graph traversal query language where queries are directly used as relational features for structured machine learning models.
Better call Saul: Flexible Programming for Learning and Inference in NLP
We present a novel way for designing complex joint inference and learning models using Saul (Kordjamshidi et al., 2015), a recently-introduced declarative learning-based programming language (DeLBP).
EDISON: Feature Extraction for NLP, Simplified
We present EDISON, a Java library of feature generation functions used in a suite of state-of-the-art NLP tools, based on a set of generic NLP data structures.
Deep Embedding for Spatial Role Labeling
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description.
HiEve: A Corpus for Extracting Event Hierarchies from News Stories
In this work, we present HiEve, a corpus for recognizing relations of spatiotemporal containment between events.
