Search Results for author: Ali Modarressi

Found 16 papers, 11 papers with code

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge

1 code implementation17 Jun 2025 Zeinab Sadat Taghavi, Ali Modarressi, Yunpu Ma, Hinrich Schütze

But even with a short context of only ten documents, including the positive document, GPT-4. 1 scores only 35. 06%, showing that document-side reasoning remains a challenge.

Benchmarking Retrieval +3

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing

no code implementations27 May 2025 Raoyuan Zhao, Abdullatif Köksal, Ali Modarressi, Michael A. Hedderich, Hinrich Schütze

To evaluate these probing methods, in this paper, we propose a new process based on using input variations and quantitative metrics.

Knowledge Probing

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence

no code implementations6 Mar 2025 Mohsen Fayyaz, Ali Modarressi, Hinrich Schuetze, Nanyun Peng

Dense retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG).

Information Retrieval RAG +3

NoLiMa: Long-Context Evaluation Beyond Literal Matching

1 code implementation7 Feb 2025 Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A. Rossi, Seunghyun Yoon, Hinrich Schütze

A popular method for evaluating these capabilities is the needle-in-a-haystack (NIAH) test, which involves retrieving a "needle" (relevant information) from a "haystack" (long irrelevant context).

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

1 code implementation8 Oct 2024 Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schütze

This suggests that MEXA is a reliable method for estimating the multilingual capabilities of English-centric LLMs, providing a clearer understanding of their multilingual potential and the inner workings of LLMs.

ARC Belebele +1

Consistent Document-Level Relation Extraction via Counterfactuals

1 code implementation9 Jul 2024 Ali Modarressi, Abdullatif Köksal, Hinrich Schütze

We first demonstrate that models trained on factual data exhibit inconsistent behavior: while they accurately extract triples from factual data, they fail to extract the same triples after counterfactual modification.

counterfactual Document-level Relation Extraction +1

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory

1 code implementation17 Apr 2024 Ali Modarressi, Abdullatif Köksal, Ayyoob Imani, Mohsen Fayyaz, Hinrich Schütze

While current large language models (LLMs) perform well on many knowledge-related tasks, they are limited by relying on their parameters as an implicit storage mechanism.

Hallucination Language Modeling +4

DecompX: Explaining Transformers Decisions by Propagating Token Decomposition

1 code implementation5 Jun 2023 Ali Modarressi, Mohsen Fayyaz, Ehsan Aghazadeh, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar

An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed.

RET-LLM: Towards a General Read-Write Memory for Large Language Models

1 code implementation23 May 2023 Ali Modarressi, Ayyoob Imani, Mohsen Fayyaz, Hinrich Schütze

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization.

Question Answering

AdapLeR: Speeding up Inference by Adaptive Length Reduction

1 code implementation ACL 2022 Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar

To determine the importance of each token representation, we train a Contribution Predictor for each layer using a gradient-based saliency method.

Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations

no code implementations13 Sep 2021 Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar

Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models.

All

Cannot find the paper you are looking for? You can Submit a new open access paper.