Search Results for author:
Found 98 papers, 34 papers with code
Automatic Evaluation vs. User Preference in Neural Textual QuestionAnswering over COVID-19 Scientific Literature
In this paper we present a quantitative and qualitative analysis of the system.
Label Verbalization and Entailment for Effective Zero and Few-Shot Relation Extraction
In our experiments on TACRED we attain 63% F1 zero-shot, 69% with 16 examples per relation (17% points better than the best supervised system on the same conditions), and only 4 points short to the state-of-the-art (which uses 20 times more training data).
WiCkeD: A Simple Method to Make Multiple Choice Benchmarks More Challenging
The performance of the models drops 12. 1 points on average with respect to the original versions of the datasets.
Data Contamination Report from the 2024 CONDA Shared Task
The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models.
BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval
The novelty of BiVLC is to add a synthetic hard negative image generated from the synthetic text, resulting in two image-to-text retrieval examples (one for each image) and, more importantly, two text-to-image retrieval examples (one for each text).
Event Extraction in Basque: Typologically motivated Cross-Lingual Transfer-Learning Analysis
To perform the experiments we introduce EusIE, an event extraction dataset for Basque, which follows the Multilingual Event Extraction dataset (MEE).
Latxa: An Open Language Model and Evaluation Suite for Basque
We introduce Latxa, a family of large language models for Basque ranging from 7 to 70 billion parameters.
Grounding Spatial Relations in Text-Only Language Models
This paper shows that text-only Language Models (LM) can learn to ground spatial relations like "left of" or "below" if they are provided with explicit location information of objects and they are properly trained to leverage those locations.
Improving Explicit Spatial Relationships in Text-to-Image Generation through an Automatically Derived Dataset
We hypothesize that this is because explicit spatial relations rarely appear in the image captions used to train these models.
PixT3: Pixel-based Table-To-Text Generation
Table-to-text generation involves generating appropriate textual descriptions given structured tabular data.
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
In this position paper, we argue that the classical evaluation on Natural Language Processing (NLP) tasks using annotated benchmarks is in trouble.
Automatic Logical Forms improve fidelity in Table-to-Text generation
Table-to-text systems generate natural language statements from structured data like tables.
Unsupervised Domain Adaption for Neural Information Retrieval
We focus on the BEIR benchmark, which includes test datasets from several domains with no training data, and explore two scenarios: zero-shot, where the supervised system is trained in a large out-of-domain dataset (MS-MARCO); and unsupervised domain adaptation, where, in addition to MS-MARCO, the system is fine-tuned in synthetic data from the target domain.
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction
In this paper, we propose GoLLIE (Guideline-following Large Language Model for IE), a model able to improve zero-shot results on unseen IE tasks by virtue of being fine-tuned to comply with annotation guidelines.
Ranked #1 on
Zero-shot Named Entity Recognition (NER)
on HarveyNER
(using extra training data)
CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models
Methods for adapting language models (LMs) to new tasks and domains have traditionally assumed white-box access to the model, and work by modifying its parameters.
What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories
Language Models are the core for almost any Natural Language Processing system nowadays.
Lessons learned from the evaluation of Spanish Language Models
Given the impact of language models on the field of Natural Language Processing, a number of Spanish encoder-only masked language models (aka BERTs) have been trained and released.
Principled Paraphrase Generation with Parallel Corpora
Round-trip Machine Translation (MT) is a popular choice for paraphrase generation, which leverages readily available parallel corpora for supervision.
PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generation
During inference, we build control codes for the desired meter and rhyme scheme, and condition our language model on them to generate formal verse poetry.
Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning
In this work we show that entailment is also effective in Event Argument Extraction (EAE), reducing the need of manual annotation to 50% and 20% in ACE and WikiEvents respectively, while achieving the same performance as with full training.
Ranked #1 on
Event Argument Extraction
on WikiEvents
ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations
The current workflow for Information Extraction (IE) analysts involves the definition of the entities/relations of interest and a training corpus with annotated examples.
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field.
Image Captioning for Effective Use of Language Models in Knowledge-Based Visual Question Answering
Our results on a visual question answering task which requires external knowledge (OK-VQA) show that our text-only model outperforms pretrained multimodal (image-text) models of comparable number of parameters.
Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction
In our experiments on TACRED we attain 63% F1 zero-shot, 69% with 16 examples per relation (17% points better than the best supervised system on the same conditions), and only 4 points short to the state-of-the-art (which uses 20 times more training data).
Ranked #11 on
Relation Extraction
on TACRED
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring
Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embeddings.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+2
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining
Existing models of multilingual sentence embeddings require large parallel data resources which are not available for low-resource languages.
Inferring spatial relations from textual descriptions of images
Previous work did not use the caption text information, but a manually provided relation holding between the subject and the object.
Beyond Offline Mapping: Learning Cross Lingual Word Embeddings through Context Anchoring
Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embeddings.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+2
Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning
The interaction of conversational systems with users poses an exciting opportunity for improving them after deployment, but little evidence has been provided of its feasibility.
Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems
In this work, we introduce \emph{Spot The Bot}, a cost-efficient and robust evaluation framework that replaces human-bot conversations with conversations between bots.
DoQA - Accessing Domain-Specific FAQs via Conversational QA
We present DoQA, a dataset with 2, 437 dialogues and 10, 917 QA pairs.
DoQA -- Accessing Domain-Specific FAQs via Conversational QA
We present DoQA, a dataset with 2, 437 dialogues and 10, 917 QA pairs.
Conversational Question Answering in Low Resource Scenarios: A Dataset and Case Study for Basque
Conversational Question Answering (CQA) systems meet user information needs by having conversations with them, where answers to the questions are retrieved from text.
A Call for More Rigor in Unsupervised Cross-lingual Learning
We review motivations, definition, approaches, and methodology for unsupervised cross-lingual learning and call for a more rigorous position in each of them.
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation
For this, we introduce an intermediate representation that is based on the logical query plan in a database called Operation Trees (OT).
Translation Artifacts in Cross-lingual Transfer Learning
Both human and machine translation play a central role in cross-lingual transfer learning: many multilingual datasets have been created through professional translation services, and using machine translation to translate either the test set or the training set is a widely used transfer technique.
Evaluating Multimodal Representations on Visual Semantic Textual Similarity
In the case of textual representations, inference tasks such as Textual Entailment and Semantic Textual Similarity have been often used to benchmark the quality of textual representations.
Give your Text Representation Models some Love: the Case for Basque
This is suboptimal as, for many languages, the models have been trained on smaller (or lower quality) corpora.
Bilingual Lexicon Induction through Unsupervised Machine Translation
A recent research line has obtained strong results on bilingual lexicon induction by aligning independently trained word embeddings in two languages and using the resulting cross-lingual embeddings to induce word translation pairs through nearest neighbor or related retrieval methods.
Analyzing the Limitations of Cross-lingual Word Embedding Mappings
Recent research in cross-lingual word embeddings has almost exclusively focused on offline methods, which independently train word embeddings in different languages and map them to a shared space through linear transformations.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+1
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings
Word embeddings typically represent different meanings of a word in a single conflated vector.
Survey on Evaluation Methods for Dialogue Systems
We cover each class by introducing the main technologies developed for the dialogue systems and then by presenting the evaluation methods regarding this class.
An Effective Approach to Unsupervised Machine Translation
While machine translation has traditionally relied on large amounts of parallel corpora, a recent research line has managed to train both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) systems using monolingual corpora only.
Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation
Named Entity Disambiguation algorithms typically learn a single model for all target entities.
Evaluating Multimodal Representations on Sentence Similarity: vSTS, Visual Semantic Textual Similarity Dataset
In this paper we introduce vSTS, a new dataset for measuring textual similarity of sentences using multimodal information.
Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation
Following the recent success of word embeddings, it has been argued that there is no such thing as an ideal representation for words, as different models tend to capture divergent and often mutually incompatible aspects like semantics/syntax and similarity/relatedness.
Unsupervised Statistical Machine Translation
While modern machine translation has relied on large parallel corpora, a recent line of work has managed to train Neural Machine Translation (NMT) systems from monolingual corpora only (Artetxe et al., 2018c; Lample et al., 2018).
Ranked #3 on
Machine Translation
on WMT2014 French-English
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
Recent work has managed to learn cross-lingual word embeddings without parallel data by mapping monolingual embeddings to a shared space through adversarial training.
The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD
UKB is an open source collection of programs for performing, among other tasks, knowledge-based Word Sense Disambiguation (WSD).
Unsupervised Neural Machine Translation
In spite of the recent success of neural machine translation (NMT) in standard benchmarks, the lack of large parallel corpora poses a major practical problem for many language pairs.
Ranked #6 on
Machine Translation
on WMT2015 English-German
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
Semantic Textual Similarity (STS) measures the meaning similarity of sentences.
SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation
Semantic Textual Similarity (STS) measures the meaning similarity of sentences.
Learning bilingual word embeddings with (almost) no bilingual data
Most methods to learn bilingual word embeddings rely on large parallel corpora, which is difficult to obtain for most language pairs.
Improving Translation Selection with Supersenses
Selecting appropriate translations for source words with multiple meanings still remains a challenge for statistical machine translation (SMT).
QTLeap WSD/NED Corpora: Semantic Annotation of Parallel Corpora in Six Languages
This work presents parallel corpora automatically annotated with several NLP tools, including lemma and part-of-speech tagging, named-entity recognition and classification, named-entity disambiguation, word-sense disambiguation, and coreference.
Addressing the MFS Bias in WSD systems
Word Sense Disambiguation (WSD) systems tend to have a strong bias towards assigning the Most Frequent Sense (MFS), which results in high performance on the MFS but in a very low performance on the less frequent senses.
A comparison of Named-Entity Disambiguation and Word Sense Disambiguation
Named Entity Disambiguation (NED) is the task of linking a named-entity mention to an instance in a knowledge-base, typically Wikipedia-derived resources like DBpedia.
Evaluating Translation Quality and CLIR Performance of Query Sessions
Translation quality improved in all three types (generalization, specification, and drifting), and CLIR improved for generalization and specification sessions, preserving the performance in drifting sessions.
Word Sense-Aware Machine Translation: Including Senses as Contextual Features for Improved Translation Models
Although it is commonly assumed that word sense disambiguation (WSD) should help to improve lexical choice and improve the quality of machine translation systems, how to successfully integrate word senses into such systems remains an unanswered question.
Evaluating the word-expert approach for Named-Entity Disambiguation
Named Entity Disambiguation (NED) is the task of linking a named-entity mention to an instance in a knowledge-base, typically Wikipedia.
Improving distant supervision using inference learning
Distant supervision is a widely applied approach to automatic training of relation extraction systems and has the advantage that it can generate large amounts of labelled data with minimal effort.
Studying the Wikipedia Hyperlink Graph for Relatedness and Disambiguation
Hyperlinks and other relations in Wikipedia are a extraordinary resource which is still not fully understood.
Matching Cultural Heritage items to Wikipedia
Digitised Cultural Heritage (CH) items usually have short descriptions and lack rich contextual information.
