Search Results for author:
Found 53 papers, 14 papers with code
CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns
The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet).
Leveraging Contextual Embeddings and Idiom Principle for Detecting Idiomaticity in Potentially Idiomatic Expressions
In this work, we leverage the Idiom Principle (Sinclair et al., 1991) and contextualized word embeddings (CWEs), focusing on Context2Vec (Melamud et al., 2016) and BERT (Devlin et al., 2019) to distinguish between literal and idiomatic senses of such expressions in context.
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Generative LLM Inference
We also show that adapting LLMs that have been pre-trained on more balanced multilingual data results in downstream performance comparable to the original models.
Word Boundary Information Isn't Useful for Encoder Language Models
As such, removing these symbols has been shown to have a beneficial effect on the processing of morphologically complex words for transformer encoders in the pretrain-finetune paradigm.
Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information
The long-standing one-to-many issue of the open-domain dialogues poses significant challenges for automatic evaluation methods, i. e., there may be multiple suitable responses which differ in semantics for a given conversational context.
Assessing Linguistic Generalisation in Language Models: A Dataset for Brazilian Portuguese
These tasks are designed to evaluate how different language models generalise information related to grammatical structures and multiword expressions (MWEs), thus allowing for an assessment of whether the model has learned different linguistic phenomena.
Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5
We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two tasks, one of which depends on the other.
Sample Efficient Approaches for Idiomaticity Detection
In particular we study the impact of Pattern Exploit Training (PET), a few-shot method of classification, and BERTRAM, an efficient method of creating contextual embeddings, on the task of idiomaticity detection.
SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding
This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence Embedding, which consists of two subtasks: (a) a binary classification task aimed at identifying whether a sentence contains an idiomatic expression, and (b) a task based on semantic text similarity which requires the model to adequately represent potentially idiomatic expressions in context.
Improving Tokenisation by Alternative Treatment of Spaces
We find that our modified algorithms lead to improved performance on downstream NLP tasks that involve handling complex words, whilst having no detrimental effect on performance in general natural language understanding tasks.
AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models
Despite their success in a variety of NLP tasks, pre-trained language models, due to their heavy reliance on compositionality, fail in effectively capturing the meanings of multiword expressions (MWEs), especially idioms.
Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels
This paper presents the Noun Compound Type and Token Idiomaticity (NCTTI) dataset, with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token level.
Unsupervised Word Segmentation from Discrete Speech Units in Low-Resource Settings
Our results suggest that neural models for speech discretization are difficult to exploit in our setting, and that it might be necessary to adapt them to limit sequence length.
Probing for idiomaticity in vector space models
Contextualised word representation models have been successfully used for capturing different word usages and they may be an attractive alternative for representing idiomaticity in language.
Token Level Identification of Multiword Expressions Using Contextual Information
Studies on detecting idiomatic expressions mostly focus on discovering potentially idiomatic expressions disregarding the context.
Investigating Language Impact in Bilingual Approaches for Computational Language Documentation
For answering this question, we use the MaSS multilingual speech corpus (Boito et al., 2020) for creating 56 bilingual pairs that we apply to the task of low-resource unsupervised word segmentation and alignment.
How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
For language documentation initiatives, transcription is an expensive resource: one minute of audio is estimated to take one hour and a half on average of a linguist's work (Austin and Sallabank, 2013).
Why So Down? The Role of Negative (and Positive) Pointwise Mutual Information in Distributional Semantics
In distributional semantics, the pointwise mutual information ($\mathit{PMI}$) weighting of the cooccurrence matrix performs far better than raw counts.
When the whole is greater than the sum of its parts: Multiword expressions and idiomaticity
Multiword expressions (MWEs) feature prominently in the mental lexicon of native speakers (Jackendoff, 1997) in all languages and domains, from informal to technical contexts (Biber et al., 1999) with about four MWEs being produced per minute of discourse (Glucksberg, 1989).
Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settings
This task consists in aligning word sequences in a source language with phoneme sequences in a target language, inferring from it word segmentation on the target side [5].
Unsupervised Compositionality Prediction of Nominal Compounds
General crosslingual analyses reveal the impact of morphological variation and corpus size in the ability of the model to predict compositionality, and of a uniform combination of the components for best results.
A small Griko-Italian speech translation corpus
This paper presents an extension to a very low-resource parallel corpus collected in an endangered language, Griko, making it useful for computational research.
Unsupervised Word Segmentation from Speech with Attention
We present a first attempt to perform attentional word segmentation directly from the speech signal, with the final goal to automatically identify lexical units in a low-resource, unwritten language (UL).
Similarity Measures for the Detection of Clinical Conditions with Verbal Fluency Tasks
Semantic Verbal Fluency tests have been used in the detection of certain clinical conditions, like Dementia.
Incorporating Subword Information into Matrix Factorization Word Embeddings
The positive effect of adding subword information to word embeddings has been demonstrated for predictive models.
Unwritten Languages Demand Attention Too! Word Discovery with Encoder-Decoder Models
Word discovery is the task of extracting words from unsegmented text.
Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality
Increasing the capacity of recurrent neural networks (RNN) usually involves augmenting the size of the hidden layer, with significant increase of computational cost.
Automatic Construction of Large Readability Corpora
In a comparison between shallow and deeper features, the former already produce F-measures of over 0. 75 for Portuguese texts, but the use of additional features results in even better results, in most cases.
Enhancing the LexVec Distributed Word Representation Model Using Positional Contexts and External Memory
The effectiveness of both modifications is shown using word similarity and analogy tasks.
Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations
In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Information matrix via stochastic gradient descent, employing a weighting scheme that assigns heavier penalties for errors on frequent co-occurrences while still accounting for negative co-occurrence.
B2SG: a TOEFL-like Task for Portuguese
They can be used as the basis for evaluating the accuracy of the similarity relations on distributional thesauri by comparing the proximity of the target word with the related and unrelated options and observing if the related word has the highest similarity value among them.
Multiword Expressions in Child Language
Focusing on compound nouns (CN), we then verify in a longitudinal study if there are differences in the distribution and compositionality of CNs in child-directed and child-produced sentences across ages.
mwetoolkit+sem: Integrating Word Embeddings in the mwetoolkit for Semantic MWE Processing
This paper presents mwetoolkit+sem: an extension of the mwetoolkit that estimates semantic compositionality scores for multiword expressions (MWEs) based on word embeddings.
VerbLexPor: a lexical resource with semantic roles for Portuguese
The sentences from both corpora were annotated separately, so that it is possible to access sentences either from the Cardiology or from the newspaper corpus.
Identification of Multiword Expressions in the brWaC
To indirectly assess the quality of the resulting corpus we examined the impact of corpus origin in a specific task, the identification of Multiword Expressions with association measures, against a standard corpus.
Comparing the Quality of Focused Crawlers and of the Translation Resources Obtained from them
Comparable corpora have been used as an alternative for parallel corpora as resources for computational tasks that involve domain-specific natural language processing.
Comparing Similarity Measures for Distributional Thesauri
Distributional thesauri have been applied for a variety of tasks involving semantic relatedness.
A large scale annotated child language construction database
This paper describes such an initiative for combining information from various sources to extend the annotation of the English CHILDES corpora with linguistic, psycholinguistic and distributional information, along with an example illustrating an application of this approach to the extraction of verb alternation information.
