Search Results for author:
Found 62 papers, 27 papers with code
Prompt-free and Efficient Few-shot Learning with Language Models
Current methods for few-shot fine-tuning of pretrained masked language models (PLMs) require carefully engineered prompts and verbalizers for each new task to convert examples into a cloze-format that the PLM can score.
Sentence-level Planning for Especially Abstractive Summarization
Abstractive summarization models heavily rely on copy mechanisms, such as the pointer network or attention, to achieve good performance, measured by textual overlap with reference summaries.
Fast-and-Frugal Text-Graph Transformers are Effective Link Predictors
Link prediction models can benefit from incorporating textual descriptions of entities and relations, enabling fully inductive learning and flexibility in dynamic graphs.
Enhancing Biomedical Knowledge Discovery for Diseases: An Open-Source Framework Applied on Rett Syndrome and Alzheimer's Disease
The ever-growing volume of biomedical publications creates a critical need for efficient knowledge discovery.
Contrastive learning of T cell receptor representations
Here we introduce a TCR language model called SCEPTR (Simple Contrastive Embedding of the Primary sequence of T cell Receptors), capable of data-efficient transfer learning.
Limits on Inferring T-cell Specificity from Partial Information
To perform this mapping requires the identification of sequence features most informative about function.
Nonparametric Variational Regularisation of Pretrained Transformers
We extend the NVIB framework to replace all types of attention functions in Transformers, and show that existing pretrained Transformers can be reinterpreted as Nonparametric Variational (NV) models using a proposed identity initialisation.
Transformers as Graph-to-Graph Models
We argue that Transformers are essentially graph-to-graph models, with sequences just being a special case.
Learning to Abstract with Nonparametric Variational Information Bottleneck
Finally, we show that NVIB compression results in a model which is more robust to adversarial perturbations.
GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction
Document-level relation extraction typically relies on text-based encoders and hand-coded pooling heuristics to aggregate information learned by the encoder.
TESS: Text-to-Text Self-Conditioned Simplex Diffusion
Diffusion models have emerged as a powerful paradigm for generation, obtaining strong performance in various continuous domains.
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question
We demonstrate that RQUGE has a higher correlation with human judgment without relying on the reference question.
SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages
In recent years, multilingual machine translation models have achieved promising performance on low-resource language pairs by sharing information between similar languages, thus enabling zero-shot translation.
A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck
We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings.
Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers
Recognizing and categorizing lexical collocations in context is useful for language learning, dictionary compilation and downstream NLP.
What Do Compressed Multilingual Machine Translation Models Forget?
In this work, we assess the impact of compression methods on Multilingual Neural Machine Translation models (MNMT) for various language groups, gender, and semantic biases by extensive analysis of compressed models on different machine translation benchmarks, i. e. FLORES-101, MT-Gender, and DiBiMT.
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models
Current methods for few-shot fine-tuning of pretrained masked language models (PLMs) require carefully engineered prompts and verbalizers for each new task to convert examples into a cloze-format that the PLM can score.
Graph Refinement for Coreference Resolution
The state-of-the-art models for coreference resolution are based on independent mention pair-wise decisions.
Ranked #9 on
Coreference Resolution
on OntoNotes
HyperMixer: An MLP-based Low Cost Alternative to Transformers
We find that existing architectures such as MLPMixer, which achieves token mixing through a static MLP applied to each feature independently, are too detached from the inductive biases required for natural language understanding.
Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation
We address this issue by extending their method to Bag-of-Vectors Autoencoders (BoV-AEs), which encode the text into a variable-size bag of vectors that grows with the size of the text, as in attention-based models.
Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning
Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is limited.
Ranked #10 on
Relation Extraction
on Adverse Drug Events (ADE) Corpus
The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task
The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure.
Variational Information Bottleneck for Effective Low-Resource Fine-Tuning
Moreover, we show that our VIB model finds sentence representations that are more robust to biases in natural language inference datasets, and thereby obtains better generalization to out-of-domain datasets.
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
In this work, we propose Compacter, a method for fine-tuning large-scale language models with a better trade-off between task performance and the number of trainable parameters than prior work.
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
State-of-the-art parameter-efficient fine-tuning methods rely on introducing adapter modules between the layers of a pretrained language model.
Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling
Recent models have shown that incorporating syntactic knowledge into the semantic role labelling (SRL) task leads to a significant improvement.
Ranked #6 on
Semantic Role Labeling
on CoNLL 2005
(using extra training data)
Inducing Meaningful Units from Character Sequences with Dynamic Capacity Slot Attention
Characters do not convey meaning, but sequences of characters do.
Multi-Adversarial Learning for Cross-Lingual Word Embeddings
Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings -- maps of matching words across languages -- without supervision.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+1
Plug and Play Autoencoders for Conditional Text Generation
Text autoencoders are commonly used for conditional generation tasks such as style transfer.
The Unstoppable Rise of Computational Linguistics in Deep Learning
In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures.
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing.
Ranked #8 on
Dependency Parsing
on Penn Treebank
Graph-to-Graph Transformer for Transition-based Dependency Parsing
We propose the Graph2Graph Transformer architecture for conditioning on and predicting arbitrary graphs, and apply it to the challenging task of transition-based dependency parsing.
Learning Entailment-Based Sentence Embeddings from Natural Language Inference
Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences.
End-to-End Bias Mitigation by Modelling Biases in Corpora
We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data.
Partially-supervised Mention Detection
Learning to detect entity mentions without using syntactic information can be useful for integration and joint optimization with other tasks.
Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains
In this study, we propose a multilingual neural language model architecture, trained jointly on the domain-specific data of several low-resource languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Deep Residual Output Layers for Neural Language Generation
Many tasks, including language generation, benefit from learning the structure of the output space, particularly when the space of output labels is large and the data is sparse.
Ranked #12 on
Language Modelling
on Penn Treebank (Word Level)
Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
Distributed representations of words which map each word to a continuous vector have proven useful in capturing important linguistic information not only in a single language but also across different languages.
Measuring Societal Biases from Text Corpora with Smoothed First-Order Co-occurrence
The common approach to measuring such biases using a corpus is by calculating the similarities between the embedding vector of a word (like nurse) and the vectors of the representative words of the concepts of interest (such as genders).
Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation
We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones.
Document-Level Neural Machine Translation with Hierarchical Attention Networks
Neural Machine Translation (NMT) can be improved by including document-level contextual information.
Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
The model is a generalized form of weight tying which shares parameters but allows learning a more flexible relationship with input word embeddings and allows the effective capacity of the output layer to be controlled.
GILE: A Generalized Input-Label Embedding for Text Classification
This forces their parametrization to be dependent on the label set size, and, hence, they are unable to scale to large label sets and generalize to unseen ones.
Unsupervised Learning of Entailment-Vector Word Embeddings
Entailment vectors are a principled way to encode in a vector what information is known and what is unknown.
Learning Word Embeddings for Hyponymy with Entailment-Based Distributional Semantics
Lexical entailment, such as hyponymy, is a fundamental issue in the semantics of natural language.
Bag-of-Vector Embeddings of Dependency Graphs for Semantic Induction
We demonstrate the usefulness of this representation by training bag-of-vector embeddings of dependency graphs and evaluating them on unsupervised semantic induction for the Semantic Textual Similarity and Natural Language Inference tasks.
CLCL (Geneva) DINN Parser: a Neural Network Dependency Parser Ten Years Later
This paper describes the University of Geneva{'}s submission to the CoNLL 2017 shared task Multilingual Parsing from Raw Text to Universal Dependencies (listed as the CLCL (Geneva) entry).
A Vector Space for Distributional Semantics for Entailment
Distributional semantics creates vector-space representations that capture many forms of semantic similarity, but their relation to semantic entailment has been less clear.
A Bayesian Model of Multilingual Unsupervised Semantic Role Induction
We propose a Bayesian model of unsupervised semantic role induction in multiple languages, and use it to explore the usefulness of parallel corpora for this task.
Efficient Computation of Mean Truncated Hitting Times on Very Large Graphs
Previous work has shown the effectiveness of random walk hitting times as a measure of dissimilarity in a variety of graph-based learning problems such as collaborative filtering, query suggestion or finding paraphrases.
