1 code implementation • ACL 2022 • Yue Zhang, Parisa Kordjamshidi
In this paper, we investigate the problem of vision and language navigation.
no code implementations • Findings (ACL) 2022 • Chen Zheng, Parisa Kordjamshidi
We study the challenge of learning causal reasoning over procedural text to answer “What if...” questions when external commonsense knowledge is required.
no code implementations • 4 Feb 2025 • Akshar Tumu, Parisa Kordjamshidi
In this work, we propose using the Referring Expression Comprehension task instead as a platform for the evaluation of spatial reasoning by VLMs.
1 code implementation • 20 Dec 2024 • Danial Kamali, Elham J. Barezi, Parisa Kordjamshidi
Compositional generalization is crucial for artificial intelligence agents to solve complex vision-language reasoning tasks.
Ranked #1 on
Visual Question Answering
on CLEVR
1 code implementation • 22 Oct 2024 • Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, Ziqiao Ma
Spatial expressions in situated communication can be ambiguous, as their meanings vary depending on the frames of reference (FoR) adopted by speakers and listeners.
no code implementations • 4 Oct 2024 • Yue Zhang, Zhiyang Xu, Ying Shen, Parisa Kordjamshidi, Lifu Huang
2) the architectures of existing 3D-based LLMs lack explicit alignment between the spatial representations of 3D scenes and natural language, limiting their performance in tasks requiring precise spatial reasoning.
1 code implementation • 6 Sep 2024 • Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
In this work, we propose a framework for evaluating in-context learning mechanisms, which we claim are a combination of retrieving internal knowledge and learning from in-context examples by focusing on regression tasks.
1 code implementation • 19 Aug 2024 • Yue Zhang, Parisa Kordjamshidi
First, VLN-CE agents that discretize the visual environment are primarily trained with high-level view selection, which causes them to ignore crucial spatial reasoning within the low-level action movements.
no code implementations • 30 Jul 2024 • Hossein Rajaby Faghihi, Aliakbar Nafar, Andrzej Uszok, Hamid Karimian, Parisa Kordjamshidi
This approach empowers domain experts, even those not well-versed in ML/AI, to formally declare their knowledge to be incorporated in customized neural models in the DomiKnowS framework.
1 code implementation • 9 Jul 2024 • Yue Zhang, Ziqiao Ma, Jialu Li, Yanyuan Qiao, Zun Wang, Joyce Chai, Qi Wu, Mohit Bansal, Parisa Kordjamshidi
Vision-and-Language Navigation (VLN) has gained increasing attention over recent years and many approaches have emerged to advance their development.
1 code implementation • 6 Jul 2024 • Zixu Cheng, Yujiang Pu, Shaogang Gong, Parisa Kordjamshidi, Yu Kong
Temporal grounding, also known as video moment retrieval, aims at locating video segments corresponding to a given query sentence.
no code implementations • 27 Jun 2024 • Elham J. Barezi, Parisa Kordjamshidi
We study the Knowledge-Based visual question-answering problem, for which given a question, the models need to ground it into the visual modality to find the answer.
1 code implementation • 19 Jun 2024 • Tanawan Premsri, Parisa Kordjamshidi
Recent research shows that more data and larger models can provide more accurate solutions to natural language problems requiring reasoning.
no code implementations • 13 Jun 2024 • Sania Sinha, Tanawan Premsri, Parisa Kordjamshidi
Compositional learning, mastering the ability to combine basic concepts and construct more intricate ones, is crucial for human cognition, especially in human language comprehension and visual perception.
no code implementations • 16 Apr 2024 • Elham J. Barezi, Parisa Kordjamshidi
2) How do task-specific and LLM-based models perform in the integration of visual and external knowledge, and multi-hop reasoning over both sources of information?
1 code implementation • 14 Feb 2024 • Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
This paper considers the challenges Large Language Models (LLMs) face when reasoning over text that includes information involving uncertainty explicitly quantified via probability values.
no code implementations • 6 Feb 2024 • Hossein Rajaby Faghihi, Parisa Kordjamshidi
This paper introduces a novel decision-making framework that promotes consistency among decisions made by diverse models while utilizing external knowledge.
1 code implementation • 4 Feb 2024 • Yue Zhang, Quan Guo, Parisa Kordjamshidi
The hint generator assists the navigation agent in developing a global understanding of the visual environment.
1 code implementation • 9 Nov 2023 • Guangyue Xu, Joyce Chai, Parisa Kordjamshidi
In this work, we propose GIP-COL (Graph-Injected Soft Prompting for COmpositional Learning) to better explore the compositional zero-shot learning (CZSL) ability of VLMs within the prompt-based learning framework.
no code implementations • 7 Nov 2023 • Danial Kamali, Parisa Kordjamshidi
Compositional generalization, the ability of intelligent models to extrapolate understanding of components to novel compositions, is a fundamental yet challenging facet in AI research, especially within multimodal environments.
Ranked #2 on
Compositional Generalization (AVG)
on ReaSCAN
1 code implementation • 2 Nov 2023 • Guangyue Xu, Parisa Kordjamshidi, Joyce Chai
Inspired by this observation, in this paper, we propose MetaReVision, a retrieval-enhanced meta-learning model to address the visually grounded compositional concept learning problem.
1 code implementation • 25 Oct 2023 • Roshanak Mirzaee, Parisa Kordjamshidi
Spatial reasoning over text is challenging as the models not only need to extract the direct spatial information from the text but also reason over those and infer implicit spatial relations.
1 code implementation • 22 May 2023 • Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
In this paper, we evaluate the capability of transformer-based language models in making inferences over uncertain text that includes uncertain rules of reasoning.
1 code implementation • 18 Feb 2023 • Yue Zhang, Parisa Kordjamshidi
The mentioned landmarks are not recognizable by the navigation agent due to the different vision abilities of the instructor and the modeled agent.
1 code implementation • 16 Feb 2023 • Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, Parisa Kordjamshidi
Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models.
1 code implementation • 14 Feb 2023 • Hossein Rajaby Faghihi, Parisa Kordjamshidi, Choh Man Teng, James Allen
In this paper, we investigate whether symbolic semantic representations, extracted from deep semantic parsers, can help reasoning over the states of involved entities in a procedural text.
1 code implementation • 11 Nov 2022 • Danial Kamali, Joseph Romain, Huiyi Liu, Wei Peng, Jingbo Meng, Parisa Kordjamshidi
We evaluate fine-tuning and prompt-engineering techniques with pre-trained language models of the BERT family and the generative large language models of the GPT family using persuasive strategies as an additional source of information.
no code implementations • 9 Nov 2022 • Guangyue Xu, Parisa Kordjamshidi, Joyce Chai
This work explores the zero-shot compositional learning ability of large pre-trained vision-language models(VLMs) within the prompt-based learning framework and propose a model (\textit{PromptCompVL}) to solve the compositonal zero-shot learning (CZSL) problem.
1 code implementation • 30 Oct 2022 • Roshanak Mirzaee, Parisa Kordjamshidi
Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains.
1 code implementation • COLING 2022 • Yue Zhang, Parisa Kordjamshidi
Understanding spatial and visual information is essential for a navigation agent who follows natural language instructions.
1 code implementation • COLING 2022 • Chen Zheng, Parisa Kordjamshidi
DRGN operates on a given KG subgraph based on the question and answers entities and uses the relevance scores between the nodes to establish new edges dynamically for learning node representations in the graph network.
1 code implementation • 21 Mar 2022 • Chen Zheng, Parisa Kordjamshidi
We study the challenge of learning causal reasoning over procedural text to answer "What if..." questions when external commonsense knowledge is required.
1 code implementation • EMNLP (ACL) 2021 • Hossein Rajaby Faghihi, Quan Guo, Andrzej Uszok, Aliakbar Nafar, Elaheh Raisi, Parisa Kordjamshidi
We demonstrate a library for the integration of domain knowledge in deep learning architectures.
no code implementations • ACL (MetaNLP) 2021 • Guangyue Xu, Parisa Kordjamshidi, Joyce Y. Chai
In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework.
2 code implementations • NAACL 2021 • Roshanak Mirzaee, Hossein Rajaby Faghihi, Qiang Ning, Parisa Kordjamshidi
This paper proposes a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models (LM).
1 code implementation • 27 May 2021 • Chen Zheng, Parisa Kordjamshidi
We propose a novel relational gating network that learns to filter the key entities and relationships and learns contextual and cross representations of both procedure and question for finding the answer.
no code implementations • ACL (splurobonlp) 2021 • Yue Zhang, Quan Guo, Parisa Kordjamshidi
Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.
1 code implementation • NAACL 2021 • Hossein Rajaby Faghihi, Parisa Kordjamshidi
This enables us to use pre-trained transformer-based language models on other QA benchmarks by adapting those to the procedural text understanding.
Ranked #1 on
Procedural Text Understanding
on ProPara
1 code implementation • WS 2020 • Hossein Rajaby Faghihi, Roshanak Mirzaee, Sudarshan Paliwal, Parisa Kordjamshidi
We propose a novel alignment mechanism to deal with procedural reasoning on a newly released multimodal QA dataset, named RecipeQA.
Ranked #1 on
Question Answering
on RecipeQA
no code implementations • EMNLP 2020 • Parisa Kordjamshidi, James Pustejovsky, Marie-Francine Moens
Understating spatial semantics expressed in natural language can become highly complex in real-world applications.
1 code implementation • EMNLP 2020 • Chen Zheng, Parisa Kordjamshidi
This work deals with the challenge of learning and reasoning over multi-hop question answering (QA).
no code implementations • LREC 2020 • Soham Dan, Parisa Kordjamshidi, Julia Bonn, Archna Bhatia, Jon Cai, Martha Palmer, Dan Roth
To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.
Abstract Meaning Representation
Natural Language Understanding
+1
1 code implementation • ACL 2020 • Chen Zheng, Quan Guo, Parisa Kordjamshidi
This work deals with the challenge of learning and reasoning over language and vision data for the related downstream tasks such as visual question answering (VQA) and natural language for visual reasoning (NLVR).
no code implementations • 18 Jun 2019 • Parisa Kordjamshidi, Dan Roth, Kristian Kersting
Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry.
no code implementations • WS 2018 • Umar Manzoor, Parisa Kordjamshidi
Spatial relation extraction from generic text is a challenging problem due to the ambiguity of the prepositions spatial meaning as well as the nesting structure of the spatial descriptions.
no code implementations • NAACL 2018 • Taher Rahgooy, Umar Manzoor, Parisa Kordjamshidi
Extraction of spatial relations from sentences with complex/nesting relationships is very challenging as often needs resolving inherent semantic ambiguities.
no code implementations • WS 2017 • Parisa Kordjamshidi, Taher Rahgooy, Umar Manzoor
The DeLBP framework facilitates combining modalities and representing various data in a unified graph.
no code implementations • 25 Jul 2017 • Parisa Kordjamshidi, Sameer Singh, Daniel Khashabi, Christos Christodoulopoulos, Mark Summons, Saurabh Sinha, Dan Roth
In particular, we provide an initial prototype for a relational and graph traversal query language where queries are directly used as relational features for structured machine learning models.
1 code implementation • COLING 2016 • Parisa Kordjamshidi, Daniel Khashabi, Christos Christodoulopoulos, Bhargav Mangipudi, Sameer Singh, Dan Roth
We present a novel way for designing complex joint inference and learning models using Saul (Kordjamshidi et al., 2015), a recently-introduced declarative learning-based programming language (DeLBP).
no code implementations • LREC 2016 • Mark Sammons, Christos Christodoulopoulos, Parisa Kordjamshidi, Daniel Khashabi, Vivek Srikumar, Dan Roth
We present EDISON, a Java library of feature generation functions used in a suite of state-of-the-art NLP tools, based on a set of generic NLP data structures.
1 code implementation • 28 Mar 2016 • Oswaldo Ludwig, Xiao Liu, Parisa Kordjamshidi, Marie-Francine Moens
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description.
no code implementations • LREC 2014 • Goran Glava{\v{s}}, Jan {\v{S}}najder, Marie-Francine Moens, Parisa Kordjamshidi
In this work, we present HiEve, a corpus for recognizing relations of spatiotemporal containment between events.