Search Results for author:
Found 44 papers, 20 papers with code
HiEve: A Corpus for Extracting Event Hierarchies from News Stories
In this work, we present HiEve, a corpus for recognizing relations of spatiotemporal containment between events.
Deep Embedding for Spatial Role Labeling
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description.
EDISON: Feature Extraction for NLP, Simplified
We present EDISON, a Java library of feature generation functions used in a suite of state-of-the-art NLP tools, based on a set of generic NLP data structures.
Better call Saul: Flexible Programming for Learning and Inference in NLP
We present a novel way for designing complex joint inference and learning models using Saul (Kordjamshidi et al., 2015), a recently-introduced declarative learning-based programming language (DeLBP).
Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks
In particular, we provide an initial prototype for a relational and graph traversal query language where queries are directly used as relational features for structured machine learning models.
Spatial Language Understanding with Multimodal Graphs using Declarative Learning based Programming
The DeLBP framework facilitates combining modalities and representing various data in a unified graph.
Anaphora Resolution for Improving Spatial Relation Extraction from Text
Spatial relation extraction from generic text is a challenging problem due to the ambiguity of the prepositions spatial meaning as well as the nesting structure of the spatial descriptions.
Visually Guided Spatial Relation Extraction from Text
Extraction of spatial relations from sentences with complex/nesting relationships is very challenging as often needs resolving inherent semantic ambiguities.
Declarative Learning-Based Programming as an Interface to AI Systems
Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry.
Cross-Modality Relevance for Reasoning on Language and Vision
This work deals with the challenge of learning and reasoning over language and vision data for the related downstream tasks such as visual question answering (VQA) and natural language for visual reasoning (NLVR).
From Spatial Relations to Spatial Configurations
To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.
SRLGRN: Semantic Role Labeling Graph Reasoning Network
This work deals with the challenge of learning and reasoning over multi-hop question answering (QA).
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks
Understating spatial semantics expressed in natural language can become highly complex in real-world applications.
Latent Alignment of Procedural Concepts in Multimodal Recipes
We propose a novel alignment mechanism to deal with procedural reasoning on a newly released multimodal QA dataset, named RecipeQA.
Ranked #1 on
Question Answering
on RecipeQA
Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events
This enables us to use pre-trained transformer-based language models on other QA benchmarks by adapting those to the procedural text understanding.
Ranked #1 on
Procedural Text Understanding
on ProPara
Towards Navigation by Reasoning over Spatial Configurations
Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.
Relational Gating for "What If" Reasoning
We propose a novel relational gating network that learns to filter the key entities and relationships and learns contextual and cross representations of both procedure and question for finding the answer.
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning
This paper proposes a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models (LM).
Zero-Shot Compositional Concept Learning
In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework.
DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning
We demonstrate a library for the integration of domain knowledge in deep learning architectures.
Relevant CommonSense Subgraphs for "What if..." Procedural Reasoning
We study the challenge of learning causal reasoning over procedural text to answer "What if..." questions when external commonsense knowledge is required.
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering
DRGN operates on a given KG subgraph based on the question and answers entities and uses the relevance scores between the nodes to establish new edges dynamically for learning node representations in the graph network.
LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation
Understanding spatial and visual information is essential for a navigation agent who follows natural language instructions.
Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning
Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains.
Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning
This work explores the zero-shot compositional learning ability of large pre-trained vision-language models(VLMs) within the prompt-based learning framework and propose a model (\textit{PromptCompVL}) to solve the compositonal zero-shot learning (CZSL) problem.
Using Persuasive Writing Strategies to Explain and Detect Health Misinformation
We evaluate fine-tuning and prompt-engineering techniques with pre-trained language models of the BERT family and the generative large language models of the GPT family using persuasive strategies as an additional source of information.
The Role of Semantic Parsing in Understanding Procedural Text
In this paper, we investigate whether symbolic semantic representations, extracted from deep semantic parsers, can help reasoning over the states of involved entities in a procedural text.
GLUECons: A Generic Benchmark for Learning Under Constraints
Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models.
VLN-Trans: Translator for the Vision and Language Navigation Agent
The mentioned landmarks are not recognizable by the navigation agent due to the different vision abilities of the instructor and the modeled agent.
Teaching Probabilistic Logical Reasoning to Transformers
In this paper, we evaluate the capability of transformer-based language models in making inferences over uncertain text that includes uncertain rules of reasoning.
Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning
Spatial reasoning over text is challenging as the models not only need to extract the direct spatial information from the text but also reason over those and infer implicit spatial relations.
MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition
Inspired by this observation, in this paper, we propose MetaReVision, a retrieval-enhanced meta-learning model to address the visually grounded compositional concept learning problem.
Syntax-Guided Transformers: Elevating Compositional Generalization and Grounding in Multimodal Environments
Compositional generalization, the ability of intelligent models to extrapolate understanding of components to novel compositions, is a fundamental yet challenging facet in AI research, especially within multimodal environments.
GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning
In this work, we propose GIP-COL (Graph-Injected Soft Prompting for COmpositional Learning) to better explore the compositional zero-shot learning (CZSL) ability of VLMs within the prompt-based learning framework.
NavHint: Vision and Language Navigation Agent with a Hint Generator
The hint generator assists the navigation agent in developing a global understanding of the visual environment.
Consistent Joint Decision-Making with Heterogeneous Learning Models
This paper introduces a novel decision-making framework that promotes consistency among decisions made by diverse models while utilizing external knowledge.
Probabilistic Reasoning in Generative Large Language Models
This paper considers the challenges that Large Language Models (LLMs) face when reasoning over text that includes information involving uncertainty explicitly quantified via probability values.
Find The Gap: Knowledge Base Reasoning For Visual Question Answering
2) How do task-specific and LLM-based models perform in the integration of visual and external knowledge, and multi-hop reasoning over both sources of information?
Relevant CommonSense Subgraphs for “What if...” Procedural Reasoning
We study the challenge of learning causal reasoning over procedural text to answer “What if...” questions when external commonsense knowledge is required.
