Search Results for author:
Found 80 papers, 24 papers with code
SemEval-2022 Task 4: Patronizing and Condescending Language Detection
This paper presents an overview of Task 4 at SemEval-2022, which was focused on detecting Patronizing and Condescending Language (PCL) towards vulnerable communities.
Self-Supervised Intermediate Fine-Tuning of Biomedical Language Models for Interpreting Patient Case Descriptions
Interpreting patient case descriptions has emerged as a challenging problem for biomedical NLP, where the aim is typically to predict diagnoses, to recommended treatments, or to answer questions about cases more generally.
Distilling Relation Embeddings from Pretrained Language Models
Among others, this makes it possible to distill high-quality word vectors from pre-trained language models.
Sentence Selection Strategies for Distilling Word Embeddings from BERT
Many applications crucially rely on the availability of high-quality word vectors.
Pre-Training Language Models for Identifying Patronizing and Condescending Language: An Analysis
Patronizing and Condescending Language (PCL) is a subtle but harmful type of discourse, yet the task of recognizing PCL remains under-studied by the NLP community.
Combining BERT with Static Word Embeddings for Categorizing Social Media
A particularly striking example is the performance of AraBERT, an LM for the Arabic language, which is successful in categorizing social media posts in Arabic dialects, despite only having been trained on Modern Standard Arabic.
Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis
One line of work treats this task as a Natural Language Inference (NLI) problem, thus relying on the knowledge captured by language models to identify the missing knowledge.
Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings
We show that this leads to embeddings which capture a more diverse range of commonsense properties, and consistently improves results in downstream tasks such as ultra-fine entity typing and ontology completion.
Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies
We focus in particular on the task of ranking entities according to a given conceptual space dimension.
Capturing Knowledge Graphs and Rules with Octagon Embeddings
Region based knowledge graph embeddings represent relations as geometric regions.
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction
Relation extraction is essentially a text classification problem, which can be tackled by fine-tuning a pre-trained language model (LM).
What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies
We show that by augmenting the label set with shared properties, we can improve the performance of the state-of-the-art models for this task.
Solving Hard Analogy Questions with Relation Embedding Chains
A common strategy is to rely on knowledge graphs (KGs) such as ConceptNet, and to model the relation between two concepts as a set of paths.
Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces
The theory of Conceptual Spaces is an influential cognitive-linguistic framework for representing the meaning of concepts.
RelBERT: Embedding Relations with Language Models
In particular, we show that masked language models such as RoBERTa can be straightforwardly fine-tuned for this purpose, using only a small amount of training data.
RAGAS: Automated Evaluation of Retrieval Augmented Generation
We introduce RAGAs (Retrieval Augmented Generation Assessment), a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines.
Inductive Knowledge Graph Completion with GNNs and Rules: An Analysis
The task of inductive knowledge graph completion requires models to learn inference patterns from a training graph, which can then be used to make predictions on a disjoint test graph.
A RelEntLess Benchmark for Modelling Graded Relations between Named Entities
Relations such as "is influenced by", "is known for" or "is a competitor of" are inherently graded: we can rank entity pairs based on how well they satisfy these relations, but it is hard to draw a line between those pairs that satisfy them and those that do not.
EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains
In this paper, we propose to improve on this process by pre-training an entity encoder such that embeddings of coreferring entities are more similar to each other than to the embeddings of other entities.
Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategy
In this paper, we show that the performance of existing methods can be improved using a simple technique: we use pre-trained label embeddings to cluster the labels into semantic domains and then treat these domains as additional types.
Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Models
Second, concept embeddings should capture the semantic properties of concepts, whereas contextualised word vectors are also affected by other factors.
What's the Meaning of Superhuman Performance in Today's NLU?
In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks such as SuperGLUE and SQuAD to measure their abilities in language understanding, reasoning, and reading comprehension.
Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge
In particular, we find that when the epistemic pooling principle is satisfied, in most cases it is impossible to verify the satisfaction of propositional formulas using linear scoring functions, with two exceptions: (i) max-pooling with embeddings that are upper-bounded and (ii) Hadamard pooling with non-negative embeddings.
Modelling Commonsense Properties using Pre-Trained Bi-Encoders
Grasping the commonsense properties of everyday concepts is an important prerequisite to language understanding.
Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention
A key feature of the multi-label setting is that images often have multiple labels, which typically refer to different regions of the image.
Distilling Relation Embeddings from Pre-trained Language Models
Among others, this makes it possible to distill high-quality word vectors from pre-trained language models.
Modelling Monotonic and Non-Monotonic Attribute Dependencies with Embeddings: A Theoretical Analysis
However, little is known about what kinds of semantic dependencies can be modelled in this way.
Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection
Second, rather than learning a word vector directly, we use a topic model to partition the contexts in which words appear, and then learn different topic-specific vectors for each word.
Probing Pre-Trained Language Models for Disease Knowledge
Pre-trained language models such as ClinicalBERT have achieved impressive results on tasks such as medical Natural Language Inference.
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning
While the use of class names has already been explored in previous work, our approach differs in two key aspects.
BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
Analogies play a central role in human commonsense reasoning.
A Description Logic for Analogical Reasoning
Ontologies formalise how the concepts from a given domain are interrelated.
Few-shot Image Classification with Multi-Facet Prototypes
The aim of few-shot learning (FSL) is to learn how to recognize image categories from a small number of training examples.
Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings
While the success of pre-trained language models has largely eliminated the need for high-quality static word vectors in many NLP applications, such vectors continue to play an important role in tasks where words need to be modelled in the absence of linguistic context.
Cardiff University at SemEval-2020 Task 6: Fine-tuning BERT for Domain-Specific Definition Classification
We describe the system submitted to SemEval-2020 Task 6, Subtask 1.
A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings
This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i. e. to encourage each of the individual embeddings to capture a meaningful facet.
Don't Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communities
In this paper, we introduce a new annotated dataset which is aimed at supporting the development of NLP models to identify and categorize language that is patronizing or condescending towards vulnerable communities (e. g. refugees, homeless people, poor families).
Plausible Reasoning about EL-Ontologies using Concept Interpolation
In this paper, we instead propose an inductive inference mechanism which is based on a clear model-theoretic semantics, and can thus be tightly integrated with standard deductive reasoning.
From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
Some common concerns are identified and discussed such as the types of used representation, the roles of knowledge and data, the lack or the excess of information, or the need for explanations and causal understanding.
Modelling Semantic Categories using Conceptual Neighborhood
Unfortunately, meaningful regions can be difficult to estimate, especially since we often have few examples of individuals that belong to a given category.
Inducing Relational Knowledge from BERT
Starting from a few seed instances of a given relation, we first use a large text corpus to find sentences that are likely to express this relation.
Learning Conceptual Spaces with Disentangled Facets
To address this gap, we analyze how, and to what extent, a given vector space embedding can be decomposed into meaningful facets in an unsupervised fashion.
Meemi: A Simple Method for Post-processing and Integrating Cross-lingual Word Embeddings
While monolingual word embeddings encode information about words in the context of a particular language, cross-lingual embeddings define a multilingual space where word embeddings from two or more languages are integrated together.
Cross-Lingual Natural Language Inference
Cross-Lingual Word Embeddings
+3
Learning Household Task Knowledge from WikiHow Descriptions
Commonsense procedural knowledge is important for AI agents and robots that operate in a human environment.
On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning
Cross-lingual word embeddings are vector representations of words in different languages where words with similar meaning are represented by similar vectors, regardless of the language.
Collocation Classification with Unsupervised Relation Vectors
Lexical relation classification is the task of predicting whether a certain relation holds between a given pair of words.
Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors
To this end, our model relies on the assumption that context word vectors are drawn from a mixture of von Mises-Fisher (vMF) distributions, where the parameters of this mixture distribution are jointly optimized with the word vectors.
Relational Word Embeddings
While word embeddings have been shown to implicitly encode various forms of attributional knowledge, the extent to which they capture relational information is far more limited.
Cardiff University at SemEval-2019 Task 4: Linguistic Features for Hyperpartisan News Detection
This paper summarizes our contribution to the Hyperpartisan News Detection task in SemEval 2019.
Learning Cross-lingual Embeddings from Twitter via Distant Supervision
Cross-lingual embeddings represent the meaning of words from different languages in the same vector space.
Predicting ConceptNet Path Quality Using Crowdsourced Assessments of Naturalness
In this paper we instead propose to learn to predict path quality from crowdsourced human assessments.
Interpretable Emoji Prediction via Label-Wise Attention LSTMs
Human language has evolved towards newer forms of communication such as social media, where emojis (i. e., ideograms bearing a visual meaning) play a key role.
Modelling Salient Features as Directions in Fine-Tuned Semantic Spaces
In this paper, we argue that there is an inherent trade-off between capturing similarity and faithfully modelling features as directions.
Improving Cross-Lingual Word Embeddings by Meeting in the Middle
Cross-lingual word embeddings are becoming increasingly important in multilingual NLP.
SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors
For example, by examining clusters of relation vectors, we observe that relational similarities can be identified at a more abstract level than with traditional word vector differences.
Relation Induction in Word Embeddings Revisited
Given a set of instances of some relation, the relation induction task is to predict which other word pairs are likely to be related in the same way.
Knowledge Representation with Conceptual Spaces
Conceptual spaces, as proposed by Grdenfors, are similar to entity embeddings, but provide more structure.
Unsupervised Learning of Distributional Relation Vectors
Word embedding models such as GloVe rely on co-occurrence statistics to learn vector representations of word meaning.
Syntactically Aware Neural Architectures for Definition Extraction
Automatically identifying definitional knowledge in text corpora (Definition Extraction or DE) is an important task with direct applications in, among others, Automatic Glossary Generation, Taxonomy Learning, Question Answering and Semantic Search.
From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules
To address this shortcoming, in this paper we introduce a general framework based on a view of relations as regions, which allows us to study the compatibility between ontological knowledge and different types of vector space embeddings.
Learning Conceptual Space Representations of Interrelated Concepts
Several recently proposed methods aim to learn conceptual space representations from large text collections.
VC-Dimension Based Generalization Bounds for Relational Learning
In many applications of relational learning, the available data can be seen as a sample from a larger relational structure (e. g. we may be given a small fragment from some social network).
PAC-Reasoning in Relational Domains
We consider the problem of predicting plausible missing facts in relational data, given a set of imperfect logical rules.
Modeling Semantic Relatedness using Global Relation Vectors
Word embedding models such as GloVe rely on co-occurrence statistics from a large corpus to learn vector representations of word meaning.
Stacked Structure Learning for Lifted Relational Neural Networks
Lifted Relational Neural Networks (LRNNs) describe relational domains using weighted first-order rules which act as templates for constructing feed-forward neural networks.
Relational Marginal Problems: Theory and Estimation
In the propositional setting, the marginal problem is to find a (maximum-entropy) distribution that has some given marginals.
Probabilistic Relation Induction in Vector Space Embeddings
Word embeddings have been found to capture a surprisingly rich amount of syntactic and semantic knowledge.
Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding
Although region representations of word meaning offer a natural alternative to word vectors, only few methods have been proposed that can effectively learn word regions.
Jointly Learning Word Embeddings and Latent Topics
Word embedding models such as Skip-gram learn a vector-space representation for each word, based on the local word collocation patterns that are observed in a text corpus.
Induction of Interpretable Possibilistic Logic Theories from Relational Data
Compared to Markov Logic Networks (MLNs), our method is faster and produces considerably more interpretable models.
D-GloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities
We propose a new word embedding model, inspired by GloVe, which is formulated as a feasible least squares optimization problem.
Stratified Knowledge Bases as Interpretable Probabilistic Models (Extended Abstract)
In this paper, we advocate the use of stratified logical theories for representing probabilistic models.
Learning Possibilistic Logic Theories from Default Rules
We introduce a setting for learning possibilistic logic theories from defaults of the form "if alpha then typically beta".
Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoning
Conceptual spaces are geometric representations of conceptual knowledge, in which entities correspond to points, natural properties correspond to convex regions, and the dimensions of the space correspond to salient features.
Solving stable matching problems using answer set programming
Since the introduction of the stable marriage problem (SMP) by Gale and Shapley (1962), several variants and extensions have been investigated.
Encoding Markov Logic Networks in Possibilistic Logic
Markov logic uses weighted formulas to compactly encode a probability distribution over possible worlds.
Realizing RCC8 networks using convex regions
First, we identify all ways in which the set of RCC8 base relations can be restricted to guarantee that consistent networks can be convexly realized in respectively 1D, 2D, 3D, and 4D.
Characterizing and Extending Answer Set Semantics using Possibility Theory
In particular, while the complexity of most reasoning tasks coincides with standard disjunctive ASP, we find that brave reasoning for programs with weak disjunctions is easier.
Modeling Stable Matching Problems with Answer Set Programming
Our encoding can easily be extended and adapted to the needs of specific applications.
