Search Results for author:
Found 98 papers, 32 papers with code
I Wish I Would Have Loved This One, But I Didn’t – A Multilingual Dataset for Counterfactual Detection in Product Review
We consider the problem of counterfactual detection (CFD) in product reviews.
Learning to Borrow– Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion
In this paper, we propose and evaluate several methods to address this problem, where we borrow LDPs from the entity pairs that co-occur in sentences in the corpus (i. e. with mentions entity pairs) to represent entity pairs that do not co-occur in any sentence in the corpus (i. e. without mention entity pairs).
Sense Embeddings are also Biased – Evaluating Social Biases in Static and Contextualised Sense Embeddings
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
Position-based Prompting for Health Outcome Generation
Probing factual knowledge in Pre-trained Language Models (PLMs) using prompts has indirectly implied that language models (LMs) can be treated as knowledge bases.
Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs, and find that there exists only a weak correlation between these two types of evaluation measures.
Query Obfuscation by Semantic Decomposition
Our experimental results show that the proposed method can accurately reconstruct the search results for user queries, without compromising the privacy of the search engine users.
Detect and Classify – Joint Span Detection and Classification for Health Outcomes
To address this, we propose a method that uses both word-level and sentence-level information to simultaneously perform outcome span detection and outcome type classification.
Constituents Correspond to Word Sequence Patterns among Sentences with Equivalent Predicate-Argument Structures: Unsupervised Constituency Parsing by Span Matching
In this study, we empirically verify that \textbf{constituents correspond to word sequence patterns in the PAS-equivalent sentence set}.
Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NER
To address insensitive fine-tuning, we propose Mask Specific Language Modeling (MSLM), an approach that efficiently acquires target domain knowledge by appropriately weighting the importance of domain-specific terms (DS-terms) during fine-tuning.
Evaluating Unsupervised Dimensionality Reduction Methods for Pretrained Sentence Embeddings
Sentence embeddings produced by Pretrained Language Models (PLMs) have received wide attention from the NLP community due to their superior performance when representing texts in numerous downstream applications.
A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection
Detecting temporal semantic changes of words is an important task for various NLP applications that must make time-sensitive predictions.
Eagle: Ethical Dataset Given from Real Interactions
The existing evaluation metrics and methods to address these ethical challenges use datasets intentionally created by instructing humans to create instances including ethical problems.
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting
In this study, we examine the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks.
The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasing
Moreover, the performance degradation due to debiasing is also lower in the ICL case compared to that in the FT case.
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models
Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work.
$\textit{Swap and Predict}$ -- Predicting the Semantic Changes in Words across Corpora by Context Swapping
Intuitively, if the meaning of $w$ does not change between $\mathcal{C}_1$ and $\mathcal{C}_2$, we would expect the distributions of contextualised word embeddings of $w$ to remain the same before and after this random swapping process.
Can Word Sense Distribution Detect Semantic Changes of Words?
Given this relationship between WSD and SCD, we explore the possibility of predicting whether a target word has its meaning changed between two corpora collected at different time steps, by comparing the distributions of senses of that word in each corpora.
A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings
We propose a Neighbourhood-Aware Differential Privacy (NADP) mechanism considering the neighbourhood of a word in a pretrained static word embedding space to determine the minimal amount of noise required to guarantee a specified privacy level.
The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated
In this study, we compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets that containing female, male, and stereotypical words.
In-Contextual Gender Bias Suppression for Large Language Models
We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2.
Learning to Predict Concept Ordering for Common Sense Generation
Prior work has shown that the ordering in which concepts are shown to a commonsense generator plays an important role, affecting the quality of the generated sentence.
Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddings
Our proposed method can combine source sense embeddings that cover different sets of word senses.
Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context
Using fractional anisotropy, a common method used in medical imaging for comparison, we then expand these measures to examine the average isotropy of a set of clusters.
Solving Cosine Similarity Underestimation between High Frequency Words by L2 Norm Discounting
This similarity underestimation problem is particularly severe for highly frequent words.
Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings
However, some of the previously associated meanings of a target word can become obsolete over time (e. g. meaning of gay as happy), while novel usages of existing words are observed (e. g. meaning of cell as a mobile phone).
Evaluating the Robustness of Discrete Prompts
To address this question, we conduct a systematic study of the robustness of discrete prompts by applying carefully designed perturbations into an application using AutoPrompt and then measure their performance in two Natural Language Inference (NLI) datasets.
Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples
Prior works have relied on human annotated examples to compare existing intrinsic bias evaluation measures.
On the Curious Case of $\ell_2$ norm of Sense Embeddings
We show that the $\ell_2$ norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings.
Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures.
Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptation
We then generate prompts by filling manually compiled templates using the extracted pivot and anchor terms.
Random projections and Kernelised Leave One Cluster Out Cross-Validation: Universal baselines and evaluation tools for supervised machine learning for materials properties
We also find that the radial basis function improves the linear separability of chemical datasets in all 10 datasets tested and provide a framework for the application of this function in the LOCO-CV process to improve the outcome of LOCO-CV measurements regardless of machine learning algorithm, choice of metric, and choice of compound representation.
Gender Bias in Meta-Embeddings
Different methods have been proposed to develop meta-embeddings from a given set of source embeddings.
Gender Bias in Masked Language Models for Multiple Languages
Unfortunately, it was reported that MLMs also learn discriminative biases regarding attributes such as gender and race.
Learning to Borrow -- Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion
In this paper, we propose and evaluate several methods to address this problem, where we borrow LDPs from the entity pairs that co-occur in sentences in the corpus (i. e. with mention entity pairs) to represent entity pairs that do not co-occur in any sentence in the corpus (i. e. without mention entity pairs).
Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings
Given multiple source word embeddings learnt using diverse algorithms and lexical resources, meta word embedding learning methods attempt to learn more accurate and wide-coverage word embeddings.
A Survey on Word Meta-Embedding Learning
Meta-embedding (ME) learning is an emerging approach that attempts to learn more accurate word embeddings given existing (source) word embeddings as the sole input.
Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Models
A variety of contextualised language models have been proposed in the NLP community, which are trained on diverse corpora to produce numerous Neural Language Models (NLMs).
Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
Assessment of contextualised representations in detecting outcome phrases in clinical trials
We reach a consensus on which contextualized representations are best suited for detecting outcomes from clinical-trial abstracts.
Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy
Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context.
Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance
This paper presents a novel unsupervised abstractive summarization method for opinionated texts.
Backretrieval: An Image-Pivoted Evaluation Metric for Cross-Lingual Text Representations Without Parallel Corpora
Cross-lingual text representations have gained popularity lately and act as the backbone of many tasks such as unsupervised machine translation and cross-lingual information retrieval, to name a few.
Detect and Classify -- Joint Span Detection and Classification for Health Outcomes
To address this, we propose a method that uses both word-level and sentence-level information to simultaneously perform outcome span detection and outcome type classification.
Unmasking the Mask -- Evaluating Social Biases in Masked Language Models
To overcome the above-mentioned disfluencies, we propose All Unmasked Likelihood (AUL), a bias evaluation measure that predicts all tokens in a test case given the MLM embedding of the unmasked input.
I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews
We consider the problem of counterfactual detection (CFD) in product reviews.
RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding
Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities.
Semantically-Conditioned Negative Samples for Efficient Contrastive Learning
In the knowledge distillation setting, (1) the performance of student networks increase by 4. 56\% percentage points on Tiny-ImageNet-200 and 3. 29\% on CIFAR-100 over student networks trained with no teacher and (2) 1. 23\% and 1. 72\% respectively over a \textit{hard-to-beat} baseline (Hinton et al., 2015).
RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding
Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities.
Dictionary-based Debiasing of Pre-trained Word Embeddings
Word embeddings trained on large corpora have shown to encode high levels of unfair discriminatory gender, racial, religious and ethnic biases.
Debiasing Pre-trained Contextualised Embeddings
In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention.
$k$-Neighbor Based Curriculum Sampling for Sequence Prediction
At test time, a sequence predictor is required to make predictions given past predictions as the input, instead of the past targets that are provided during training.
Graph Convolution over Multiple Dependency Sub-graphs for Relation Extraction
We propose a contextualised graph convolution network over multiple dependency-based sub-graphs for relation extraction.
Autoencoding Improves Pre-trained Word Embeddings
Prior work investigating the geometry of pre-trained word embeddings have shown that word embeddings to be distributed in a narrow cone and by centering and projecting using principal component vectors one can increase the accuracy of a given set of pre-trained word embeddings.
Spatio-temporal Attention Model for Tactile Texture Recognition
Recently, tactile sensing has attracted great interest in robotics, especially for facilitating exploration of unstructured environments and effective manipulation.
Tree-Structured Neural Topic Model
This paper presents a tree-structured neural topic model, which has a topic distribution over a tree with an infinite number of branches.
Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answering
The Conversational Question Answering (CoQA) task involves answering a sequence of inter-related conversational questions about a contextual paragraph.
Weakly-Supervised Neural Response Selection from an Ensemble of Task-Specialised Dialogue Agents
Dialogue engines that incorporate different types of agents to converse with humans are popular.
Do not let the history haunt you: Mitigating Compounding Errors in Conversational Question Answering
The Conversational Question Answering (CoQA) task involves answering a sequence of inter-related conversational questions about a contextual paragraph.
Contextualised Graph Attention for Improved Relation Extraction
This paper presents a contextualized graph attention network that combines edge features and multiple sub-graphs for improving relation extraction.
Multi-source Attention for Unsupervised Domain Adaptation
We model source-selection as an attention-learning problem, where we learn attention over sources for a given target instance.
General Classification
Multi-Source Unsupervised Domain Adaptation
+3
Language-Independent Tokenisation Rivals Language-Specific Tokenisation for Word Similarity Prediction
Moreover, they compactly represent a language using a fixed size vocabulary and can efficiently handle unseen or rare words.
Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA
We also use our method for comparing image and text encoders trained using different modern approaches, thus addressing the issues hindering the development of novel methods for cross-modal recipe retrieval.
Query Obfuscation Semantic Decomposition
Our experimental results show that the proposed method can accurately reconstruct the search results for user queries, without compromising the privacy of the search engine users.
Transfer Reward Learning for Policy Gradient-Based Text Generation
However, we argue that current n-gram overlap based measures that are used as rewards can be improved by using model-based rewards transferred from tasks that directly compare the similarity of sentence pairs.
Self-Adaptation for Unsupervised Domain Adaptation
Using the labelled data from the source domain, we first learn a projection that maximises the distance among the nearest neighbours with opposite labels in the source domain.
Gender-preserving Debiasing for Pre-trained Word Embeddings
Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings.
RelWalk -- A Latent Variable Model Approach to Knowledge Graph Embedding
Existing methods for learning KGEs can be seen as a two-stage process where (a) entities and relations in the knowledge graph are represented using some linear algebraic structures (embeddings), and (b) a scoring function is defined that evaluates the strength of a relation that holds between two entities using the corresponding relation and entity embeddings.
Error-Correcting Neural Sequence Prediction
We propose a novel neural sequence prediction method based on \textit{error-correcting output codes} that avoids exact softmax normalization and allows for a tradeoff between speed and performance.
Learning Relation Representations from Word Representations
We model relation representation as a supervised learning problem and learn parametrised operators that map pre-trained word embeddings to relation representations.
Joint Learning of Hierarchical Word Embeddings from a Corpus and a Taxonomy
To learn the word embeddings, the proposed method considers not only the hypernym relations that exists between words on a taxonomy, but also their contextual information in a large text corpus.
Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction
We propose in this paper a combined model of Long Short Term Memory and Convolutional Neural Networks (LSTM-CNN) that exploits word embeddings and positional embeddings for cross-sentence n-ary relation extraction.
Analysing Dropout and Compounding Errors in Neural Language Models
Moreover, we propose an extension of variational dropout to concrete dropout and curriculum dropout with varying schedules.
Meta-Embedding as Auxiliary Task Regularization
For intrinsic task evaluation, supervision comes from various labeled word similarity datasets.
Curriculum-Based Neighborhood Sampling For Sequence Prediction
At test time, a language model is required to make predictions given past predictions as input, instead of the past targets that are provided during training.
Angular-Based Word Meta-Embedding Learning
This work compares meta-embeddings trained for different losses, namely loss functions that account for angular distance between the reconstructed embedding and the target and those that account normalized distances based on the vector length.
Learning Word Meta-Embeddings by Autoencoding
Distributed word embeddings have shown superior performances in numerous Natural Language Processing (NLP) tasks.
Why does PairDiff work? - A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection
We show that, if the word embed- dings are standardised and uncorrelated, such an operator will be independent of bilinear terms, and can be simplified to a linear form, where PairDiff is a special case.
An Empirical Study on Fine-Grained Named Entity Recognition
However, there is little research on fine-grained NER (FG-NER), in which hundreds of named entity categories must be recognized, especially for non-English languages.
Is Something Better than Nothing? Automatically Predicting Stance-based Arguments Using Deep Learning and Small Labelled Dataset
In the work described here, we automatically annotate stance as implicit or explicit and our results show that the datasets we generate, although noisy, can be used to learn better models for implicit/explicit opinion classification.
Solving Feature Sparseness in Text Classification using Core-Periphery Decomposition
We observe that prioritising features that are common to both training and test instances as cores during the CP decomposition to further improve the accuracy of text classification.
Dropping Networks for Transfer Learning
We also compare against models that are fully trained on the target task in the standard supervised learning setup.
Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings
Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately.
ClassiNet -- Predicting Missing Features for Short-Text Classification
We propose \emph{ClassiNet} -- a network of classifiers trained for predicting missing features in a given instance, to overcome the feature sparseness problem.
Why PairDiff works? -- A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection
We show that, if the word embeddings are standardised and uncorrelated, such an operator will be independent of bilinear terms, and can be simplified to a linear form, where \PairDiff is a special case.
Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words
Distributed word embeddings have shown superior performances in numerous Natural Language Processing (NLP) tasks.
Using $k$-way Co-occurrences for Learning Word Embeddings
Co-occurrences between two words provide useful insights into the semantics of those words.
Learning Neural Word Salience Scores
Measuring the salience of a word is an essential step in numerous NLP tasks.
Compositional Approaches for Representing Relations Between Words: A Comparative Study
In contrast, a compositional approach for representing relations between words overcomes these issues by using the attributes of each individual word to indirectly compose a representation for the common relations that hold between the two words.
Joint Word Representation Learning using a Corpus and a Semantic Lexicon
For this purpose, we propose a joint word representation learning method that simultaneously predicts the co-occurrences of two words in a sentence subject to the relational constrains given by the semantic lexicon.
Unsupervised Cross-Domain Word Representation Learning
Given a pair of \emph{source}-\emph{target} domains, we propose an unsupervised method for learning domain-specific word representations that accurately capture the domain-specific aspects of word semantics.
Embedding Semantic Relations into Word Representations
We propose an unsupervised method for learning vector representations for words such that the learnt representations are sensitive to the semantic relations that exist between two words.
Learning Word Representations from Relational Graphs
To evaluate the accuracy of the word representations learnt using the proposed method, we use the learnt word representations to solve semantic word analogy problems.
Dynamic Feature Scaling for Online Learning of Binary Classifiers
Scaling feature values is an important step in numerous machine learning tasks.
