Search Results for author: Filip Ilievski

Found 60 papers, 26 papers with code

Numeracy enhances the Literacy of Language Models

no code implementations EMNLP 2021 Avijit Thawani, Jay Pujara, Filip Ilievski

This paper studies the effect of using six different number encoders on the task of masked word prediction (MWP), as a proxy for evaluating literacy.


ReferenceNet: a semantic-pragmatic network for capturing reference relations.

no code implementations GWC 2018 Piek Vossen, Filip Ilievski, Marten Postrma

In this paper, we present ReferenceNet: a semantic-pragmatic network of reference relations between synsets.

Word Embeddings

The Predicate Matrix and the Event and Implied Situation Ontology: Making More of Events

no code implementations GWC 2016 Roxane Segers, Egoitz Laparra, Marco Rospocher, Piek Vossen, German Rigau, Filip Ilievski

This paper presents the Event and Implied Situation Ontology (ESO), a resource which formalizes the pre and post situations of events and the roles of the entities affected by an event.

SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

no code implementations22 Apr 2024 Yifan Jiang, Filip Ilievski, Kaixin Ma

In this paper, we split the original benchmark to also support fine-tuning setting and present SemEval Task 9: BRAIN-TEASER(S), the first task at this competition designed to test the system's reasoning and lateral thinking ability.

Common Sense Reasoning

MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning

1 code implementation21 Apr 2024 Yifan Jiang, Jiarui Zhang, Kexuan Sun, Zhivar Sourati, Kian Ahrabian, Kaixin Ma, Filip Ilievski, Jay Pujara

Further analysis of perception questions reveals that MLLMs struggle to comprehend the visual features (near-random performance) and even count the panels in the puzzle ( <45%), hindering their ability for abstract reasoning.

Visual Reasoning

Standardizing Knowledge Engineering Practices with a Reference Architecture

no code implementations4 Apr 2024 Bradley P. Allen, Filip Ilievski

The recent abstraction of systemic patterns into a boxology provides an opening for aligning the requirements and use cases of knowledge engineering with the systems, components, and software that can satisfy them best.

Language Modelling

Knowledge-Powered Recommendation for an Improved Diet Water Footprint

no code implementations26 Mar 2024 Saurav Joshi, Filip Ilievski, Jay Pujara

According to WWF, 1. 1 billion people lack access to water, and 2. 7 billion experience water scarcity at least one month a year.

graph construction Knowledge Graphs

Exploring Perceptual Limitation of Multimodal Large Language Models

1 code implementation12 Feb 2024 Jiarui Zhang, Jinyi Hu, Mahyar Khayatkhoei, Filip Ilievski, Maosong Sun

Multimodal Large Language Models (MLLMs) have recently shown remarkable perceptual capability in answering visual questions, however, little is known about the limits of their perception.

Object Question Answering

Contextualizing Internet Memes Across Social Media Platforms

no code implementations18 Nov 2023 Saurav Joshi, Filip Ilievski, Luca Luceri

Internet memes have emerged as a novel format for communication and expressing ideas on the web.

Hate Speech Detection

Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs

2 code implementations24 Oct 2023 Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip Ilievski

In particular, we show that their zero-shot accuracy in answering visual questions is very sensitive to the size of the visual subject of the question, declining up to 46% with size.

Question Answering Visual Question Answering

BRAINTEASER: Lateral Thinking Puzzles for Large Language Models

2 code implementations8 Oct 2023 Yifan Jiang, Filip Ilievski, Kaixin Ma, Zhivar Sourati

The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms.

Distractor Generation Language Modelling +3

ARN: Analogical Reasoning on Narratives

no code implementations2 Oct 2023 Zhivar Sourati, Filip Ilievski, Pia Sommerauer, Yifan Jiang

However, while cognitive theories of analogy often focus on narratives and study the distinction between surface, relational, and system similarities, existing work in natural language processing has a narrower focus as far as relational analogies between word pairs.

FIRE: Food Image to REcipe generation

1 code implementation28 Aug 2023 Prateek Chhikara, Dhiraj Chaurasia, Yifan Jiang, Omkar Masur, Filip Ilievski

Food computing has emerged as a prominent multidisciplinary field of research in recent years.

Decoder Language Modelling +3

Identifying and Consolidating Knowledge Engineering Requirements

no code implementations27 Jun 2023 Bradley P. Allen, Filip Ilievski, Saurav Joshi

Knowledge engineering is the process of creating and maintaining knowledge-producing systems.

A Study of Situational Reasoning for Traffic Understanding

1 code implementation5 Jun 2023 Jiarui Zhang, Filip Ilievski, Kaixin Ma, Aravinda Kollaa, Jonathan Francis, Alessandro Oltramari

Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure.

Decision Making Knowledge Graphs +2

Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models

no code implementations31 May 2023 Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip Ilievski

As our initial analysis of BLIP-family models revealed difficulty with answering fine-detail questions, we investigate the following question: Can visual cropping be employed to improve the performance of state-of-the-art visual question answering models on fine-detail questions?

Question Answering Visual Question Answering

Contextualizing Argument Quality Assessment with Relevant Knowledge

1 code implementation20 May 2023 Darshan Deshpande, Zhivar Sourati, Filip Ilievski, Fred Morstatter

Automatic assessment of the quality of arguments has been recognized as a challenging task with significant implications for misinformation and targeted speech.


Knowledge-enhanced Agents for Interactive Text Games

no code implementations8 May 2023 Prateek Chhikara, Jiarui Zhang, Filip Ilievski, Jonathan Francis, Kaixin Ma

We experiment with four models on the 10 tasks in the ScienceWorld text-based game environment, to illustrate the impact of knowledge injection on various model configurations and challenging task settings.

Instruction Following Knowledge Graphs +5

Transferring Procedural Knowledge across Commonsense Tasks

1 code implementation26 Apr 2023 Yifan Jiang, Filip Ilievski, Kaixin Ma

Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories.

Story Completion

Case-Based Reasoning with Language Models for Classification of Logical Fallacies

1 code implementation27 Jan 2023 Zhivar Sourati, Filip Ilievski, Hông-Ân Sandlin, Alain Mermoud

The ease and speed of spreading misinformation and propaganda on the Web motivate the need to develop trustworthy technology for detecting fallacies in natural language arguments.

Language Modelling Logical Fallacies +2

Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments

1 code implementation12 Dec 2022 Zhivar Sourati, Vishnu Priya Prasanna Venkatesh, Darshan Deshpande, Himanshu Rawlani, Filip Ilievski, Hông-Ân Sandlin, Alain Mermoud

Our three-stage framework natively consolidates prior datasets and methods from existing tasks, like propaganda detection, serving as an overarching evaluation testbed.

Data Augmentation Logical Fallacies +2

A Study of Slang Representation Methods

1 code implementation11 Dec 2022 Aravinda Kolla, Filip Ilievski, Hông-Ân Sandlin, Alain Mermoud

Considering the large amount of content created online by the minute, slang-aware automatic tools are critically needed to promote social good, and assist policymakers and moderators in restricting the spread of offensive language, abuse, and hate speech.

Representation Learning Word Embeddings

Multimodal and Explainable Internet Meme Classification

no code implementations11 Dec 2022 Abhinav Kumar Thakur, Filip Ilievski, Hông-Ân Sandlin, Zhivar Sourati, Luca Luceri, Riccardo Tommasini, Alain Mermoud

In the current context where online platforms have been effectively weaponized in a variety of geo-political events and social issues, Internet memes make fair content moderation at scale even more difficult.

Classification Explainable Models +2

Utilizing Background Knowledge for Robust Reasoning over Traffic Situations

1 code implementation4 Dec 2022 Jiarui Zhang, Filip Ilievski, Aravinda Kollaa, Jonathan Francis, Kaixin Ma, Alessandro Oltramari

Understanding novel situations in the traffic domain requires an intricate combination of domain-specific and causal commonsense knowledge.

Knowledge Graphs Multiple-choice +2

PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales

1 code implementation3 Nov 2022 Peifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, Xiang Ren

Neural language models (LMs) have achieved impressive results on various language-based reasoning tasks by utilizing latent knowledge encoded in their own pretrained parameters.

counterfactual Decision Making

Does Wikidata Support Analogical Reasoning?

no code implementations2 Oct 2022 Filip Ilievski, Jay Pujara, Kartik Shenoy

Analogical reasoning methods have been built over various resources, including commonsense knowledge bases, lexical resources, language models, or their combination.

Coalescing Global and Local Information for Procedural Text Understanding

1 code implementation COLING 2022 Kaixin Ma, Filip Ilievski, Jonathan Francis, Eric Nyberg, Alessandro Oltramari

In this paper, we propose Coalescing Global and Local Information (CGLI), a new model that builds entity- and timestep-aware input representations (local input) considering the whole context (global input), and we jointly model the entity states with a structured prediction objective (global output).

Procedural Text Understanding Structured Prediction

Enriching Wikidata with Linked Open Data

no code implementations1 Jul 2022 Bohui Zhang, Filip Ilievski, Pedro Szekely

We present a novel workflow that includes gap detection, source selection, schema alignment, and semantic validation.

Entity Alignment Knowledge Graphs

Understanding Narratives through Dimensions of Analogy

1 code implementation14 Jun 2022 Thiloshon Nagarajah, Filip Ilievski, Jay Pujara

Experiments with language models and neuro-symbolic AI reasoners on these tasks reveal that state-of-the-art methods can be applied to reason by analogy with a limited success, motivating the need for further research towards comprehensive and scalable analogical reasoning by AI.

An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs

no code implementations21 May 2022 Jiarui Zhang, Filip Ilievski, Kaixin Ma, Jonathan Francis, Alessandro Oltramari

In this paper, we study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.

Decoder Knowledge Graphs

Augmenting Knowledge Graphs for Better Link Prediction

1 code implementation26 Mar 2022 Jiang Wang, Filip Ilievski, Pedro Szekely, Ke-Thia Yao

Experiments on legacy benchmarks and a new large benchmark, DWD, show that augmenting the knowledge graph with quantities and years is beneficial for predicting both entities and numbers, as KGA outperforms the vanilla models and other relevant baselines.

Knowledge Graph Embedding Knowledge Graphs +1

Generalizable Neuro-symbolic Systems for Commonsense Question Answering

no code implementations17 Jan 2022 Alessandro Oltramari, Jonathan Francis, Filip Ilievski, Kaixin Ma, Roshanak Mirzaee

This chapter illustrates how suitable neuro-symbolic models for language understanding can enable domain generalizability and robustness in downstream tasks.

Knowledge Graphs Question Answering

Contextualized Scene Imagination for Generative Commonsense Reasoning

1 code implementation ICLR 2022 Peifeng Wang, Jonathan Zamora, Junfeng Liu, Filip Ilievski, Muhao Chen, Xiang Ren

In this paper, we propose an Imagine-and-Verbalize (I&V) method, which learns to imagine a relational scene knowledge graph (SKG) with relations between the input concepts, and leverage the SKG as a constraint when generating a plausible scene description.

Common Sense Reasoning Descriptive +2

Story Generation with Commonsense Knowledge Graphs and Axioms

no code implementations AKBC Workshop CSKB 2021 Filip Ilievski, Jay Pujara, Hanzhi Zhang

Our method aligns story types with commonsense axioms, and queries to a commonsense knowledge graph, enabling the generation of hundreds of thousands of stories.

Common Sense Reasoning Knowledge Graphs +1

User-friendly Comparison of Similarity Algorithms on Wikidata

1 code implementation11 Aug 2021 Filip Ilievski, Pedro Szekely, Gleb Satyukov, Amandeep Singh

While the similarity between two concept words has been evaluated and studied for decades, much less attention has been devoted to algorithms that can compute the similarity of nodes in very large knowledge graphs, like Wikidata.

Entity Linking Knowledge Graphs

Analyzing Race and Country of Citizenship Bias in Wikidata

no code implementations11 Aug 2021 Zaina Shaik, Filip Ilievski, Fred Morstatter

Through this analysis, we discovered that there is an overrepresentation of white individuals and those with citizenship in Europe and North America; the rest of the groups are generally underrepresented.

Creating and Querying Personalized Versions of Wikidata on a Laptop

no code implementations6 Aug 2021 Hans Chalupsky, Pedro Szekely, Filip Ilievski, Daniel Garijo, Kartik Shenoy

Application developers today have three choices for exploiting the knowledge present in Wikidata: they can download the Wikidata dumps in JSON or RDF format, they can use the Wikidata API to get data about individual entities, or they can use the Wikidata SPARQL endpoint.


A Study of the Quality of Wikidata

1 code implementation1 Jul 2021 Kartik Shenoy, Filip Ilievski, Daniel Garijo, Daniel Schwabe, Pedro Szekely

Wikidata has been increasingly adopted by many communities for a wide variety of applications, which demand high-quality knowledge to deliver successful results.

Do Language Models Perform Generalizable Commonsense Inference?

1 code implementation Findings (ACL) 2021 Peifeng Wang, Filip Ilievski, Muhao Chen, Xiang Ren

Inspired by evidence that pretrained language models (LMs) encode commonsense knowledge, recent work has applied LMs to automatically populate commonsense knowledge graphs (CKGs).

Knowledge Graphs

PaCo: Preconditions Attributed to Commonsense Knowledge

1 code implementation18 Apr 2021 Ehsan Qasemi, Filip Ilievski, Muhao Chen, Pedro Szekely

To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions.

Common Sense Reasoning

Dimensions of Commonsense Knowledge

no code implementations12 Jan 2021 Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L. McGuinness, Pedro Szekely

Recently, the focus has been on large text-based sources, which facilitate easier integration with neural (language) models and application to textual tasks, typically at the expense of the semantics of the sources and their harmonization.

CSKG: The CommonSense Knowledge Graph

1 code implementation21 Dec 2020 Filip Ilievski, Pedro Szekely, Bin Zhang

Sources of commonsense knowledge support applications in natural language understanding, computer vision, and knowledge graphs.

Knowledge Graphs Natural Language Understanding

Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering

1 code implementation7 Nov 2020 Kaixin Ma, Filip Ilievski, Jonathan Francis, Yonatan Bisk, Eric Nyberg, Alessandro Oltramari

Guided by a set of hypotheses, the framework studies how to transform various pre-existing knowledge resources into a form that is most effective for pre-training models.

Language Modelling Question Answering

Commonsense Knowledge in Wikidata

no code implementations18 Aug 2020 Filip Ilievski, Pedro Szekely, Daniel Schwabe

Our experiments reveal that: 1) albeit Wikidata-CS represents a small portion of Wikidata, it is an indicator that Wikidata contains relevant commonsense knowledge, which can be mapped to 15 ConceptNet relations; 2) the overlap between Wikidata-CS and other commonsense sources is low, motivating the value of knowledge integration; 3) Wikidata-CS has been evolving over time at a slightly slower rate compared to the overall Wikidata, indicating a possible lack of focus on commonsense knowledge.

Common Sense Reasoning Question Answering

Consolidating Commonsense Knowledge

no code implementations10 Jun 2020 Filip Ilievski, Pedro Szekely, Jingwei Cheng, Fu Zhang, Ehsan Qasemi

Commonsense reasoning is an important aspect of building robust AI systems and is receiving significant attention in the natural language understanding, computer vision, and knowledge graphs communities.

Common Sense Reasoning Knowledge Graphs +1

KGTK: A Toolkit for Large Knowledge Graph Manipulation and Analysis

1 code implementation29 May 2020 Filip Ilievski, Daniel Garijo, Hans Chalupsky, Naren Teja Divvala, Yixiang Yao, Craig Rogers, Rongpeng Li, Jun Liu, Amandeep Singh, Daniel Schwabe, Pedro Szekely

Knowledge graphs (KGs) have become the preferred technology for representing, sharing and adding knowledge to modern AI applications.

Knowledge Graphs

Combining Conceptual and Referential Annotation to Study Variation in Framing

no code implementations LREC 2020 Marten Postma, Levi Remijnse, Filip Ilievski, Antske Fokkens, Sam Titarsolej, Piek Vossen

The user can apply two types of annotations: 1) mappings from expressions to frames and frame elements, 2) reference relations from mentions to events and participants of the structured data.

Large-scale Cross-lingual Language Resources for Referencing and Framing

no code implementations LREC 2020 Piek Vossen, Filip Ilievski, Marten Postma, Antske Fokkens, Gosse Minnema, Levi Remijnse

In this article, we lay out the basic ideas and principles of the project Framing Situations in the Dutch Language.

The Profiling Machine: Active Generalization over Knowledge

no code implementations1 Oct 2018 Filip Ilievski, Eduard Hovy, Qizhe Xie, Piek Vossen

The human mind is a powerful multifunctional knowledge storage and management system that performs generalization, type inference, anomaly detection, stereotyping, and other tasks.

Anomaly Detection Management

Systematic Study of Long Tail Phenomena in Entity Linking

no code implementations COLING 2018 Filip Ilievski, Piek Vossen, Stefan Schlobach

In this paper we report on a series of hypotheses regarding the long tail phenomena in entity linking datasets, their interaction, and their impact on system performance.

Entity Linking

SemEval-2018 Task 5: Counting Events and Participants in the Long Tail

no code implementations SEMEVAL 2018 Marten Postma, Filip Ilievski, Piek Vossen

This paper discusses SemEval-2018 Task 5: a referential quantification task of counting events and participants in local, long-tail news documents with high ambiguity.

Word Sense Disambiguation

Semantic overfitting: what `world' do we consider when evaluating disambiguation of text?

no code implementations COLING 2016 Filip Ilievski, Marten Postma, Piek Vossen

Semantic text processing faces the challenge of defining the relation between lexical expressions and the world to which they make reference within a period of time.


Cannot find the paper you are looking for? You can Submit a new open access paper.