Search Results for author:
Found 17 papers, 6 papers with code
Sequence-to-Sequence Spanish Pre-trained Language Models
In recent years, substantial advancements in pre-trained language models have paved the way for the development of numerous non-English language versions, with a particular focus on encoder-only and decoder-only architectures.
Generative Question Answering
Natural Language Understanding
A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information
Existing question answering methods often assume that the input content (e. g., documents or videos) is always accessible to solve the task.
How Relevant is Selective Memory Population in Lifelong Language Learning?
Lifelong language learning seeks to have models continuously learn multiple tasks in a sequential order without suffering from catastrophic forgetting.
Populating Memory in Continual Learning with Consistency Aware Sampling
Based on these insights, we propose CAWS (Consistency AWare Sampling), an original storage policy that leverages a learning consistency score (C-Score) to populate the memory with elements that are easy to learn and representative of previous tasks.
ALBETO and DistilBETO: Lightweight Spanish Language Models
In this paper we present ALBETO and DistilBETO, which are versions of ALBERT and DistilBERT pre-trained exclusively on Spanish corpora.
Entropy-based Stability-Plasticity for Lifelong Learning
The ability to continuously learn remains elusive for deep learning models.
Evaluation Benchmarks for Spanish Sentence Representations
Due to the success of pre-trained language models, versions of languages other than English have been released in recent years.
DACT-BERT: Differentiable Adaptive Computation Time for an Efficient BERT Inference
Large-scale pre-trained language models have shown remarkable results in diverse NLP applications.
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations
Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level.
Stress Test Evaluation of Biomedical Word Embeddings
The success of pretrained word embeddings has motivated their use in the biomedical domain, with contextualized embeddings yielding remarkable results in several biomedical NLP tasks.
TNT: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification
Recently, few-shot learning has received increasing interest.
DACT-BERT: Increasing the efficiency and interpretability of BERT by using adaptive computation time.
DACT-BERT adds an adaptive computation mechanism to the regular processing pipeline of BERT.
Interpretable Contextual Team-aware Item Recommendation: Application in Multiplayer Online Battle Arena Games
Our evaluation indicates that both the Transformer architecture and the contextual information are essential to get the best results for this item recommendation task.
Adversarial Evaluation of BERT for Biomedical Named Entity Recognition
The success of pre-trained word embeddings of the BERT model has motivated its use in tasks in the biomedical domain.
Translating Natural Language Instructions for Behavioral Robot Navigation with a Multi-Head Attention Mechanism
We propose a multi-head attention mechanism as a blending layer in a neural network model that translates natural language to a high level behavioral language for indoor robot navigation.
On Adversarial Examples for Biomedical NLP Tasks
We also show that we can significantly improve the robustness of the models by training them with adversarial examples.
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
There has been significant progress in recent years in the field of Natural Language Processing thanks to the introduction of the Transformer architecture.
Natural Language Inference
Natural Language Understanding
+1
