Search Results for author:
Found 16 papers, 11 papers with code
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids’ Representations
Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models.
ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge
But even with a short context of only ten documents, including the positive document, GPT-4. 1 scores only 35. 06%, showing that document-side reasoning remains a challenge.
Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing
To evaluate these probing methods, in this paper, we propose a new process based on using input variations and quantitative metrics.
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence
Dense retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG).
NoLiMa: Long-Context Evaluation Beyond Literal Matching
A popular method for evaluating these capabilities is the needle-in-a-haystack (NIAH) test, which involves retrieving a "needle" (relevant information) from a "haystack" (long irrelevant context).
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
This suggests that MEXA is a reliable method for estimating the multilingual capabilities of English-centric LLMs, providing a clearer understanding of their multilingual potential and the inner workings of LLMs.
Consistent Document-Level Relation Extraction via Counterfactuals
We first demonstrate that models trained on factual data exhibit inconsistent behavior: while they accurately extract triples from factual data, they fail to extract the same triples after counterfactual modification.
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory
While current large language models (LLMs) perform well on many knowledge-related tasks, they are limited by relying on their parameters as an implicit storage mechanism.
DecompX: Explaining Transformers Decisions by Propagating Token Decomposition
An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed.
RET-LLM: Towards a General Read-Write Memory for Large Language Models
Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization.
Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities
A popular workaround is to train a robust model by re-weighting training examples based on a secondary biased model.
BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning
In this work, we employ these two metrics for the first time in NLP.
GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers
There has been a growing interest in interpreting the underlying dynamics of Transformers.
AdapLeR: Speeding up Inference by Adaptive Length Reduction
To determine the importance of each token representation, we train a Contribution Predictor for each layer using a gradient-based saliency method.
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations
Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models.
Exploring the Role of BERT Token Representations to Explain Sentence Probing Results
Several studies have been carried out on revealing linguistic features captured by BERT.
