1 code implementation • EMNLP 2021 • James O’Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, Danushka Bollegala
We consider the problem of counterfactual detection (CFD) in product reviews.
1 code implementation • NAACL 2022 • Huda Hakami, Mona Hakami, Angrosh Mandya, Danushka Bollegala
In this paper, we propose and evaluate several methods to address this problem, where we borrow LDPs from the entity pairs that co-occur in sentences in the corpus (i. e. with mentions entity pairs) to represent entity pairs that do not co-occur in any sentence in the corpus (i. e. without mention entity pairs).
no code implementations • ACL 2022 • Yi Zhou, Masahiro Kaneko, Danushka Bollegala
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
no code implementations • BioNLP (ACL) 2022 • Micheal Abaho, Danushka Bollegala, Paula Williamson, Susanna Dodd
Probing factual knowledge in Pre-trained Language Models (PLMs) using prompts has indirectly implied that language models (LMs) can be treated as knowledge bases.
1 code implementation • EMNLP 2021 • Micheal Abaho, Danushka Bollegala, Paula Williamson, Susanna Dodd
To address this, we propose a method that uses both word-level and sentence-level information to simultaneously perform outcome span detection and outcome type classification.
no code implementations • LREC 2022 • Danushka Bollegala, Tomoya Machide, Ken-ichi Kawarabayashi
Our experimental results show that the proposed method can accurately reconstruct the search results for user queries, without compromising the privacy of the search engine users.
no code implementations • COLING 2022 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs, and find that there exists only a weak correlation between these two types of evaluation measures.
1 code implementation • 3 Oct 2024 • Junjie Chen, Xiangheng He, Yusuke Miyao, Danushka Bollegala
In this paper, we introduce a novel objective for training unsupervised parsers: maximizing the information between constituent structures and sentence semantics (SemInfo).
1 code implementation • 3 Jul 2024 • Taichi Aida, Danushka Bollegala
Our experimental results reveal (a) although there exist a smaller number of axes that are specific to semantic changes of words in the pre-trained CWE space, this information gets distributed across all dimensions when fine-tuned, and (b) in contrast to prior work studying the geometry of CWEs, we find that PCA to better represent semantic changes than ICA within the top 10% of axes.
no code implementations • 19 Jun 2024 • Yi Zhou, Danushka Bollegala, Jose Camacho-Collados
Given that MLMs are continuously trained with increasing amounts of additional data collected over time, an important yet unanswered question is how the social biases encoded with MLMs vary over time.
1 code implementation • 25 Apr 2024 • Tianhui Zhang, Bei Peng, Danushka Bollegala
Generative Commonsense Reasoning (GCR) requires a model to reason about a situation using commonsense knowledge, while generating coherent sentences.
no code implementations • 18 Apr 2024 • Junjie Chen, Xiangheng He, Danushka Bollegala, Yusuke Miyao
Linguists identify the constituent by evaluating a set of Predicate-Argument Structure (PAS) equivalent sentences where we find the constituent appears more frequently than non-constituents (i. e., the constituent corresponds to a frequent word sequence within the sentence set).
1 code implementation • 26 Mar 2024 • Micheal Abaho, Danushka Bollegala, Gary Leeming, Dan Joyce, Iain E Buchan
To address insensitive fine-tuning, we propose Mask Specific Language Modeling (MSLM), an approach that efficiently acquires target domain knowledge by appropriately weighting the importance of domain-specific terms (DS-terms) during fine-tuning.
no code implementations • 20 Mar 2024 • Gaifan Zhang, Yi Zhou, Danushka Bollegala
Sentence embeddings produced by Pretrained Language Models (PLMs) have received wide attention from the NLP community due to their superior performance when representing texts in numerous downstream applications.
2 code implementations • 1 Mar 2024 • Taichi Aida, Danushka Bollegala
Experimental results on multiple benchmark datasets for SCD show that our proposed method achieves strong performance in multiple languages.
no code implementations • 22 Feb 2024 • Masahiro Kaneko, Danushka Bollegala, Timothy Baldwin
The existing evaluation metrics and methods to address these ethical challenges use datasets intentionally created by instructing humans to create instances including ethical problems.
no code implementations • 28 Jan 2024 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki, Timothy Baldwin
In this study, we examine the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks.
no code implementations • 16 Jan 2024 • Masahiro Kaneko, Danushka Bollegala, Timothy Baldwin
Moreover, the performance degradation due to debiasing is also lower in the ICL case compared to that in the FT case.
no code implementations • 19 Oct 2023 • Yi Zhou, Jose Camacho-Collados, Danushka Bollegala
Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work.
1 code implementation • 16 Oct 2023 • Xiaohang Tang, Yi Zhou, Taichi Aida, Procheta Sen, Danushka Bollegala
Given this relationship between WSD and SCD, we explore the possibility of predicting whether a target word has its meaning changed between two corpora collected at different time steps, by comparing the distributions of senses of that word in each corpora.
1 code implementation • 16 Oct 2023 • Taichi Aida, Danushka Bollegala
Intuitively, if the meaning of $w$ does not change between $\mathcal{C}_1$ and $\mathcal{C}_2$, we would expect the distributions of contextualised word embeddings of $w$ to remain the same before and after this random swapping process.
1 code implementation • 19 Sep 2023 • Danushka Bollegala, Shuichi Otake, Tomoya Machide, Ken-ichi Kawarabayashi
We propose a Neighbourhood-Aware Differential Privacy (NADP) mechanism considering the neighbourhood of a word in a pretrained static word embedding space to determine the minimal amount of noise required to guarantee a specified privacy level.
no code implementations • 16 Sep 2023 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
In this study, we compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets that containing female, male, and stereotypical words.
1 code implementation • 13 Sep 2023 • Daisuke Oba, Masahiro Kaneko, Danushka Bollegala
We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2.
1 code implementation • 12 Sep 2023 • Tianhui Zhang, Danushka Bollegala, Bei Peng
Prior work has shown that the ordering in which concepts are shown to a commonsense generator plays an important role, affecting the quality of the generated sentence.
1 code implementation • 30 May 2023 • Haochen Luo, Yi Zhou, Danushka Bollegala
Our proposed method can combine source sense embeddings that cover different sets of word senses.
no code implementations • 25 May 2023 • Samantha Durdy, Michael W. Gaultois, Vladimir Gusev, Danushka Bollegala, Matthew J. Rosseinsky
Using fractional anisotropy, a common method used in medical imaging for comparison, we then expand these measures to examine the average isotropy of a set of clusters.
1 code implementation • 17 May 2023 • Saeth Wannasuphoprasit, Yi Zhou, Danushka Bollegala
This similarity underestimation problem is particularly severe for highly frequent words.
1 code implementation • 15 May 2023 • Taichi Aida, Danushka Bollegala
However, some of the previously associated meanings of a target word can become obsolete over time (e. g. meaning of gay as happy), while novel usages of existing words are observed (e. g. meaning of cell as a mobile phone).
1 code implementation • 11 Feb 2023 • Yoichi Ishibashi, Danushka Bollegala, Katsuhito Sudoh, Satoshi Nakamura
To address this question, we conduct a systematic study of the robustness of discrete prompts by applying carefully designed perturbations into an application using AutoPrompt and then measure their performance in two Natural Language Inference (NLI) datasets.
no code implementations • 28 Jan 2023 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Prior works have relied on human annotated examples to compare existing intrinsic bias evaluation measures.
no code implementations • 26 Oct 2022 • Yi Zhou, Danushka Bollegala
We show that the $\ell_2$ norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings.
no code implementations • 6 Oct 2022 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures.
1 code implementation • 23 Aug 2022 • Xiaohang Tang, Yi Zhou, Danushka Bollegala
We then generate prompts by filling manually compiled templates using the extracted pivot and anchor terms.
1 code implementation • 17 Jun 2022 • Samantha Durdy, Michael Gaultois, Vladimir Gusev, Danushka Bollegala, Matthew J. Rosseinsky
We also find that the radial basis function improves the linear separability of chemical datasets in all 10 datasets tested and provide a framework for the application of this function in the LOCO-CV process to improve the outcome of LOCO-CV measurements regardless of machine learning algorithm, choice of metric, and choice of compound representation.
no code implementations • 19 May 2022 • Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Different methods have been proposed to develop meta-embeddings from a given set of source embeddings.
1 code implementation • NAACL 2022 • Masahiro Kaneko, Aizhan Imankulova, Danushka Bollegala, Naoaki Okazaki
Unfortunately, it was reported that MLMs also learn discriminative biases regarding attributes such as gender and race.
1 code implementation • 27 Apr 2022 • Huda Hakami, Mona Hakami, Angrosh Mandya, Danushka Bollegala
In this paper, we propose and evaluate several methods to address this problem, where we borrow LDPs from the entity pairs that co-occur in sentences in the corpus (i. e. with mention entity pairs) to represent entity pairs that do not co-occur in any sentence in the corpus (i. e. without mention entity pairs).
no code implementations • 26 Apr 2022 • Danushka Bollegala
Given multiple source word embeddings learnt using diverse algorithms and lexical resources, meta word embedding learning methods attempt to learn more accurate and wide-coverage word embeddings.
no code implementations • 25 Apr 2022 • Danushka Bollegala, James O'Neill
Meta-embedding (ME) learning is an emerging approach that attempts to learn more accurate word embeddings given existing (source) word embeddings as the sole input.
no code implementations • LREC 2022 • Keigo Takahashi, Danushka Bollegala
A variety of contextualised language models have been proposed in the NLP community, which are trained on diverse corpora to produce numerous Neural Language Models (NLMs).
1 code implementation • 14 Mar 2022 • Yi Zhou, Masahiro Kaneko, Danushka Bollegala
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
no code implementations • 13 Feb 2022 • Micheal Abaho, Danushka Bollegala, Paula R Williamson, Susanna Dodd
We reach a consensus on which contextualized representations are best suited for detecting outcomes from clinical-trial abstracts.
no code implementations • PACLIC 2021 • Yi Zhou, Danushka Bollegala
Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context.
1 code implementation • 15 Jun 2021 • Masaru Isonuma, Junichiro Mori, Danushka Bollegala, Ichiro Sakata
This paper presents a novel unsupervised abstractive summarization method for opinionated texts.
no code implementations • 11 May 2021 • Mikhail Fain, Niall Twomey, Danushka Bollegala
Cross-lingual text representations have gained popularity lately and act as the backbone of many tasks such as unsupervised machine translation and cross-lingual information retrieval, to name a few.
1 code implementation • 15 Apr 2021 • Michael Abaho, Danushka Bollegala, Paula Williamson, Susanna Dodd
To address this, we propose a method that uses both word-level and sentence-level information to simultaneously perform outcome span detection and outcome type classification.
2 code implementations • 15 Apr 2021 • Masahiro Kaneko, Danushka Bollegala
To overcome the above-mentioned disfluencies, we propose All Unmasked Likelihood (AUL), a bias evaluation measure that predicts all tokens in a test case given the MLM embedding of the unmasked input.
1 code implementation • 14 Apr 2021 • James O'Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, Danushka Bollegala
We consider the problem of counterfactual detection (CFD) in product reviews.
no code implementations • EACL 2021 • Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi
Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities.
no code implementations • 12 Feb 2021 • James O' Neill, Danushka Bollegala
In the knowledge distillation setting, (1) the performance of student networks increase by 4. 56\% percentage points on Tiny-ImageNet-200 and 3. 29\% on CIFAR-100 over student networks trained with no teacher and (2) 1. 23\% and 1. 72\% respectively over a \textit{hard-to-beat} baseline (Hinton et al., 2015).
1 code implementation • 25 Jan 2021 • Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi
Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities.
1 code implementation • EACL 2021 • Masahiro Kaneko, Danushka Bollegala
Word embeddings trained on large corpora have shown to encode high levels of unfair discriminatory gender, racial, religious and ethnic biases.
2 code implementations • EACL 2021 • Masahiro Kaneko, Danushka Bollegala
In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention.
no code implementations • 22 Jan 2021 • James O' Neill, Danushka Bollegala
At test time, a sequence predictor is required to make predictions given past predictions as the input, instead of the past targets that are provided during training.
no code implementations • COLING 2020 • Angrosh Mandya, Danushka Bollegala, Frans Coenen
We propose a contextualised graph convolution network over multiple dependency-based sub-graphs for relation extraction.
no code implementations • COLING 2020 • Masahiro Kaneko, Danushka Bollegala
Prior work investigating the geometry of pre-trained word embeddings have shown that word embeddings to be distributed in a narrow cone and by centering and projecting using principal component vectors one can increase the accuracy of a given set of pre-trained word embeddings.
no code implementations • 10 Aug 2020 • Guanqun Cao, Yi Zhou, Danushka Bollegala, Shan Luo
Recently, tactile sensing has attracted great interest in robotics, especially for facilitating exploration of unstructured environments and effective manipulation.
no code implementations • ACL 2020 • Masaru Isonuma, Junichiro Mori, Danushka Bollegala, Ichiro Sakata
This paper presents a tree-structured neural topic model, which has a topic distribution over a tree with an infinite number of branches.
no code implementations • 12 May 2020 • Angrosh Mandya, James O'Neill, Danushka Bollegala, Frans Coenen
The Conversational Question Answering (CoQA) task involves answering a sequence of inter-related conversational questions about a contextual paragraph.
no code implementations • 6 May 2020 • Asir Saeed, Khai Mai, Pham Minh, Nguyen Tuan Duc, Danushka Bollegala
Dialogue engines that incorporate different types of agents to converse with humans are popular.
no code implementations • LREC 2020 • M, Angrosh ya, James O{'} Neill, Danushka Bollegala, Frans Coenen
The Conversational Question Answering (CoQA) task involves answering a sequence of inter-related conversational questions about a contextual paragraph.
1 code implementation • 22 Apr 2020 • Angrosh Mandya, Danushka Bollegala, Frans Coenen
This paper presents a contextualized graph attention network that combines edge features and multiple sub-graphs for improving relation extraction.
1 code implementation • Asian Chapter of the Association for Computational Linguistics 2020 • Xia Cui, Danushka Bollegala
We model source-selection as an attention-learning problem, where we learn attention over sources for a given target instance.
General Classification
Multi-Source Unsupervised Domain Adaptation
+3
no code implementations • LREC 2020 • Danushka Bollegala, Ryuichi Kiryo, Kosuke Tsujino, Haruki Yukawa
Moreover, they compactly represent a language using a fixed size vocabulary and can efficiently handle unseen or rare words.
no code implementations • 28 Nov 2019 • Mikhail Fain, Niall Twomey, Andrey Ponikar, Ryan Fox, Danushka Bollegala
We also use our method for comparing image and text encoders trained using different modern approaches, thus addressing the issues hindering the development of novel methods for cross-modal recipe retrieval.
no code implementations • 12 Sep 2019 • Danushka Bollegala, Tomoya Machide, Ken-ichi Kawarabayashi
Our experimental results show that the proposed method can accurately reconstruct the search results for user queries, without compromising the privacy of the search engine users.
no code implementations • 9 Sep 2019 • James O' Neill, Danushka Bollegala
However, we argue that current n-gram overlap based measures that are used as rewards can be improved by using model-based rewards transferred from tasks that directly compare the similarity of sentence pairs.
no code implementations • RANLP 2019 • Xia Cui, Danushka Bollegala
Using the labelled data from the source domain, we first learn a projection that maximises the distance among the nearest neighbours with opposite labels in the source domain.
1 code implementation • ACL 2019 • Masahiro Kaneko, Danushka Bollegala
Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings.
no code implementations • ICLR 2019 • Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi
Existing methods for learning KGEs can be seen as a two-stage process where (a) entities and relations in the knowledge graph are represented using some linear algebraic structures (embeddings), and (b) a scoring function is defined that evaluates the strength of a relation that holds between two entities using the corresponding relation and entity embeddings.
no code implementations • 21 Jan 2019 • James O' Neill, Danushka Bollegala
We propose a novel neural sequence prediction method based on \textit{error-correcting output codes} that avoids exact softmax normalization and allows for a tradeoff between speed and performance.
no code implementations • AKBC 2019 • Mohammed Alsuhaibani, Takanori Maehara, Danushka Bollegala
To learn the word embeddings, the proposed method considers not only the hypernym relations that exists between words on a taxonomy, but also their contextual information in a large text corpus.
no code implementations • AKBC 2019 • Huda Hakami, Danushka Bollegala
We model relation representation as a supervised learning problem and learn parametrised operators that map pre-trained word embeddings to relation representations.
no code implementations • 2 Nov 2018 • James O' Neill, Danushka Bollegala
Moreover, we propose an extension of variational dropout to concrete dropout and curriculum dropout with varying schedules.
no code implementations • AKBC 2019 • Angrosh Mandya, Danushka Bollegala, Frans Coenen, Katie Atkinson
We propose in this paper a combined model of Long Short Term Memory and Convolutional Neural Networks (LSTM-CNN) that exploits word embeddings and positional embeddings for cross-sentence n-ary relation extraction.
no code implementations • 16 Sep 2018 • James O' Neill, Danushka Bollegala
At test time, a language model is required to make predictions given past predictions as input, instead of the past targets that are provided during training.
no code implementations • 16 Sep 2018 • James O' Neill, Danushka Bollegala
For intrinsic task evaluation, supervision comes from various labeled word similarity datasets.
no code implementations • 13 Aug 2018 • James O' Neill, Danushka Bollegala
This work compares meta-embeddings trained for different losses, namely loss functions that account for angular distance between the reconstructed embedding and the target and those that account normalized distances based on the vector length.
no code implementations • COLING 2018 • Huda Hakami, Kohei Hayashi, Danushka Bollegala
We show that, if the word embed- dings are standardised and uncorrelated, such an operator will be independent of bilinear terms, and can be simplified to a linear form, where PairDiff is a special case.
no code implementations • COLING 2018 • Khai Mai, Thai-Hoang Pham, Minh Trung Nguyen, Tuan Duc Nguyen, Danushka Bollegala, Ryohei Sasano, Satoshi Sekine
However, there is little research on fine-grained NER (FG-NER), in which hundreds of named entity categories must be recognized, especially for non-English languages.
1 code implementation • COLING 2018 • Danushka Bollegala, Cong Bao
Distributed word embeddings have shown superior performances in numerous Natural Language Processing (NLP) tasks.
no code implementations • NAACL 2018 • Pavithra Rajendran, Danushka Bollegala, Simon Parsons
In the work described here, we automatically annotate stance as implicit or explicit and our results show that the datasets we generate, although noisy, can be used to learn better models for implicit/explicit opinion classification.
no code implementations • SEMEVAL 2018 • Xia Cui, Sadamori Kojaku, Naoki Masuda, Danushka Bollegala
We observe that prioritising features that are common to both training and test instances as cores during the CP decomposition to further improve the accuracy of text classification.
no code implementations • 23 Apr 2018 • James O' Neill, Danushka Bollegala
We also compare against models that are fully trained on the target task in the standard supervised learning setup.
1 code implementation • NAACL 2018 • Joshua Coates, Danushka Bollegala
Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately.
no code implementations • 14 Apr 2018 • Danushka Bollegala, Vincent Atanasov, Takanori Maehara, Ken-ichi Kawarabayashi
We propose \emph{ClassiNet} -- a network of classifiers trained for predicting missing features in a given instance, to overcome the feature sparseness problem.
no code implementations • 19 Sep 2017 • Huda Hakami, Danushka Bollegala, Hayashi Kohei
We show that, if the word embeddings are standardised and uncorrelated, such an operator will be independent of bilinear terms, and can be simplified to a linear form, where \PairDiff is a special case.
1 code implementation • 19 Sep 2017 • Danushka Bollegala, Kohei Hayashi, Ken-ichi Kawarabayashi
Distributed word embeddings have shown superior performances in numerous Natural Language Processing (NLP) tasks.
no code implementations • 5 Sep 2017 • Danushka Bollegala, Yuichi Yoshida, Ken-ichi Kawarabayashi
Co-occurrences between two words provide useful insights into the semantics of those words.
1 code implementation • SEMEVAL 2018 • Krasen Samardzhiev, Andrew Gargett, Danushka Bollegala
Measuring the salience of a word is an essential step in numerous NLP tasks.
no code implementations • 4 Sep 2017 • Huda Hakami, Danushka Bollegala
In contrast, a compositional approach for representing relations between words overcomes these issues by using the attributes of each individual word to indirectly compose a representation for the common relations that hold between the two words.
1 code implementation • 19 Nov 2015 • Danushka Bollegala, Alsuhaibani Mohammed, Takanori Maehara, Ken-ichi Kawarabayashi
For this purpose, we propose a joint word representation learning method that simultaneously predicts the co-occurrences of two words in a sentence subject to the relational constrains given by the semantic lexicon.
no code implementations • IJCNLP 2015 • Danushka Bollegala, Takanori Maehara, Ken-ichi Kawarabayashi
Given a pair of \emph{source}-\emph{target} domains, we propose an unsupervised method for learning domain-specific word representations that accurately capture the domain-specific aspects of word semantics.
no code implementations • 1 May 2015 • Danushka Bollegala, Takanori Maehara, Ken-ichi Kawarabayashi
We propose an unsupervised method for learning vector representations for words such that the learnt representations are sensitive to the semantic relations that exist between two words.
no code implementations • 7 Dec 2014 • Danushka Bollegala, Takanori Maehara, Yuichi Yoshida, Ken-ichi Kawarabayashi
To evaluate the accuracy of the word representations learnt using the proposed method, we use the learnt word representations to solve semantic word analogy problems.
no code implementations • 28 Jul 2014 • Danushka Bollegala
Scaling feature values is an important step in numerous machine learning tasks.