no code implementations • CMCL (ACL) 2022 • Inga Lang, Lonneke Plas, Malvina Nissim, Albert Gatt
We find that adding visual vectors increases classification performance on our dataset in many cases.
1 code implementation • ACL 2022 • Wietse de Vries, Martijn Wieling, Malvina Nissim
Cross-lingual transfer learning with large multilingual pre-trained models can be an effective approach for low-resource languages with no labeled training data.
no code implementations • IWCS (ACL) 2021 • Gosse Minnema, Malvina Nissim
Frame-semantic parsers traditionally predict predicates, frames, and semantic roles in a fixed order.
1 code implementation • GeBNLP (COLING) 2020 • Marion Bartl, Malvina Nissim, Albert Gatt
Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems.
no code implementations • LREC 2022 • Evelien de Graaf, Silvia Stopponi, Jasper K. Bos, Saskia Peels-Matthey, Malvina Nissim
To facilitate corpus searches by classicists as well as to reduce data sparsity when training models, we focus on the automatic lemmatization of ancient Greek inscriptions, which have not received as much attention in this sense as literary text data has.
no code implementations • ACL (GEM) 2021 • Lorenzo De Mattei, Huiyuan Lai, Felice Dell’Orletta, Malvina Nissim
We ask subjects whether they perceive as human-produced a bunch of texts, some of which are actually human-written, while others are automatically generated.
1 code implementation • ACL (WOAH) 2021 • Tommaso Caselli, Arjan Schelhaas, Marieke Weultjes, Folkert Leistra, Hylke van der Veen, Gerben Timmerman, Malvina Nissim
As socially unacceptable language become pervasive in social media platforms, the need for automatic content moderation become more pressing.
1 code implementation • 4 Mar 2025 • Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, Arianna Bisazza
Word-level quality estimation (QE) detects erroneous spans in machine translations, which can direct and facilitate human post-editing.
no code implementations • 16 Dec 2024 • Leonidas Zotos, Hedderik van Rijn, Malvina Nissim
In an educational setting, an estimate of the difficulty of multiple-choice questions (MCQs), a commonly used strategy to assess learning progress, constitutes very useful information for both teachers and students.
no code implementations • 27 Nov 2024 • Daniel Scalena, Elisabetta Fersini, Malvina Nissim
Adapting models to a language that was only partially present in the pre-training data requires fine-tuning, which is expensive in terms of both data and computational resources.
no code implementations • 29 Oct 2024 • Shaozhen Shi, Yevgen Matusevych, Malvina Nissim
We further experiment with having a single teacher (instead of an ensemble of two teachers) and implement additional optimization strategies to improve the distillation process.
1 code implementation • 7 Jul 2024 • Leonidas Zotos, Hedderik van Rijn, Malvina Nissim
Estimating the difficulty of multiple-choice questions would be great help for educators who must spend substantial time creating and piloting stimuli for their tests, and for learners who want to practice.
1 code implementation • 25 Jun 2024 • Daniel Scalena, Gabriele Sarti, Malvina Nissim
Activation steering methods were shown to be effective in conditioning language model generation by additively intervening over models' intermediate representations.
no code implementations • 11 Jun 2024 • Daniela Occhipinti, Michele Marchi, Irene Mondella, Huiyuan Lai, Felice Dell'Orletta, Malvina Nissim, Marco Guerini
Results from both human and automatic evaluation show that the different quality of training data is clearly perceived and it has an impact also on the models trained on such data.
1 code implementation • 4 Jun 2024 • Huiyuan Lai, Malvina Nissim
Large language models (LLMs) with Chain-of-thought (CoT) have recently emerged as a powerful technique for eliciting reasoning to improve various downstream tasks.
1 code implementation • 1 Feb 2024 • Elizaveta Sivak, Paulina Pankowska, Adrienne Mendrik, Tom Emery, Javier Garcia-Bernardo, Seyit Hocuk, Kasia Karpinska, Angelica Maineri, Joris Mulder, Malvina Nissim, Gert Stulp
We outline the ways in which measuring the predictability of fertility outcomes using these datasets and combining their strengths in the data challenge can advance our understanding of fertility behaviour and computational social science.
4 code implementations • 2 Oct 2023 • Gabriele Sarti, Grzegorz Chrupała, Malvina Nissim, Arianna Bisazza
Establishing whether language models can use contextual information in a human-plausible way is important to ensure their trustworthiness in real-world settings.
1 code implementation • 1 Sep 2023 • Daniel Scalena, Gabriele Sarti, Malvina Nissim, Elisabetta Fersini
Due to language models' propensity to generate toxic or hateful responses, several techniques were developed to align model generations with users' preferences.
1 code implementation • 1 Jun 2023 • Gosse Minnema, Huiyuan Lai, Benedetta Muscato, Malvina Nissim
Different ways of linguistically expressing the same real-world event can lead to different perceptions of what happened.
1 code implementation • 31 May 2023 • Chunliu Wang, Huiyuan Lai, Malvina Nissim, Johan Bos
Pre-trained language models (PLMs) have achieved great success in NLP and have recently been used for tasks in computational semantics.
1 code implementation • 31 May 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Figures of speech help people express abstract concepts and evoke stronger emotions than literal expressions, thereby making texts more creative and engaging.
2 code implementations • 22 May 2023 • Wietse de Vries, Martijn Wieling, Malvina Nissim
The benchmark includes a diverse set of datasets for low-, medium- and high-resource tasks.
no code implementations • 2 May 2023 • Anya Belz, Craig Thomson, Ehud Reiter, Gavin Abercrombie, Jose M. Alonso-Moral, Mohammad Arvan, Anouck Braggaar, Mark Cieliebak, Elizabeth Clark, Kees Van Deemter, Tanvi Dinkar, Ondřej Dušek, Steffen Eger, Qixiang Fang, Mingqi Gao, Albert Gatt, Dimitra Gkatzia, Javier González-Corbelle, Dirk Hovy, Manuela Hürlimann, Takumi Ito, John D. Kelleher, Filip Klubicka, Emiel Krahmer, Huiyuan Lai, Chris van der Lee, Yiru Li, Saad Mahamood, Margot Mieskes, Emiel van Miltenburg, Pablo Mosteiro, Malvina Nissim, Natalie Parde, Ondřej Plátek, Verena Rieser, Jie Ruan, Joel Tetreault, Antonio Toral, Xiaojun Wan, Leo Wanner, Lewis Watson, Diyi Yang
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
1 code implementation • 26 Apr 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We investigate the potential of ChatGPT as a multidimensional evaluator for the task of \emph{Text Style Transfer}, alongside, and in comparison to, existing automatic metrics as well as human judgements.
2 code implementations • 27 Feb 2023 • Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, Arianna Bisazza
Past work in natural language processing interpretability focused mainly on popular classification tasks while largely overlooking generation settings, partly due to a lack of dedicated tools.
1 code implementation • 24 Sep 2022 • Gosse Minnema, Sara Gemelli, Chiara Zanchi, Tommaso Caselli, Malvina Nissim
We then train regression models that predict the salience of GBV participants with respect to different dimensions of perceived responsibility.
1 code implementation • COLING 2022 • Huiyuan Lai, Malvina Nissim
Figurative language generation is the task of reformulating a given text in the desired figure of speech while still being faithful to the original context.
1 code implementation • HumEval (ACL) 2022 • Huiyuan Lai, Jiali Mao, Antonio Toral, Malvina Nissim
Although text style transfer has witnessed rapid development in recent years, there is as yet no established standard for evaluation, which is performed using several automatic metrics, lacking the possibility of always resorting to human judgement.
1 code implementation • ACL 2022 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We exploit the pre-trained seq2seq model mBART for multilingual text style transfer.
3 code implementations • 7 Mar 2022 • Gabriele Sarti, Malvina Nissim
We introduce IT5, the first family of encoder-decoder transformer models pretrained specifically on Italian.
no code implementations • ACL 2022 • Gosse Minnema, Sara Gemelli, Chiara Zanchi, Tommaso Caselli, Malvina Nissim
SOCIOFILLMORE is a multilingual tool which helps to bring to the fore the focus or the perspective that a text expresses in depicting an event.
1 code implementation • EMNLP 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Style transfer aims to rewrite a source text in a different target style while preserving its content.
1 code implementation • ACL 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content.
1 code implementation • Findings (ACL) 2021 • Wietse de Vries, Martijn Bartelds, Malvina Nissim, Martijn Wieling
For many (minority) languages, the resources needed to train large models are not available.
1 code implementation • NAACL (TeachingNLP) 2021 • Ludovica Pannitto, Lucia Busso, Claudia Roberta Combei, Lucio Messina, Alessio Miaschi, Gabriele Sarti, Malvina Nissim
To raise awareness, curiosity, and longer-term interest in young people, we have developed an interactive workshop designed to illustrate the basic principles of NLP and computational linguistics to high school Italian students aged between 13 and 18 years.
1 code implementation • NAACL (TeachingNLP) 2021 • Lucio Messina, Lucia Busso, Claudia Roberta Combei, Ludovica Pannitto, Alessio Miaschi, Gabriele Sarti, Malvina Nissim
We describe and make available the game-based material developed for a laboratory run at several Italian science festivals to popularize NLP among young students.
1 code implementation • ACL (EvalNLGEval, INLG) 2020 • Lorenzo De Mattei, Michele Cafagna, Huiyuan Lai, Felice Dell'Orletta, Malvina Nissim, Albert Gatt
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics.
1 code implementation • Findings (ACL) 2021 • Wietse de Vries, Malvina Nissim
Specifically, we describe the adaptation of English GPT-2 to Italian and Dutch by retraining lexical embeddings without tuning the Transformer layers.
1 code implementation • 16 Nov 2020 • Gaetana Ruggiero, Albert Gatt, Malvina Nissim
Existing research on Authorship Attribution (AA) focuses on texts for which a lot of data is available (e. g novels), mainly in English.
1 code implementation • 11 Nov 2020 • Elisa Bassignana, Malvina Nissim, Viviana Patti
We present a novel corpus for personality prediction in Italian, containing a larger number of authors and a different genre compared to previously available resources.
1 code implementation • COLING (PEOPLES) 2020 • Elisa Bassignana, Malvina Nissim, Viviana Patti
As a contribution to personality detection in languages other than English, we rely on distant supervision to create Personal-ITY, a novel corpus of YouTube comments in Italian, where authors are labelled with personality traits.
1 code implementation • 27 Oct 2020 • Marion Bartl, Malvina Nissim, Albert Gatt
Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems.
no code implementations • CL 2020 • Malvina Nissim, Rik van Noord, Rob van der Goot
Analogies such as man is to king as woman is to X are often used to illustrate the amazing power of word embeddings.
no code implementations • LREC 2020 • Juliet van Rosendaal, Tommaso Caselli, Malvina Nissim
Strategies used until now to increase density of abusive language and obtain more meaningful data overall, include data filtering on the basis of pre-selected keywords and hate-rich sources of data.
no code implementations • LREC 2020 • Hessel Haagsma, Johan Bos, Malvina Nissim
Given the limited size of existing idiom corpora, we aim to enable progress in automatic idiom processing and linguistic analysis by creating the largest-to-date corpus of idioms for English.
no code implementations • LREC 2020 • Lorenzo De Mattei, Michele Cafagna, Felice Dell{'}Orletta, Malvina Nissim
We automatically generate headlines that are expected to comply with the specific styles of two different Italian newspapers.
1 code implementation • 29 Apr 2020 • Lorenzo De Mattei, Michele Cafagna, Felice Dell'Orletta, Malvina Nissim, Marco Guerini
We provide a thorough analysis of GePpeTto's quality by means of both an automatic and a human-based evaluation.
2 code implementations • Findings of the Association for Computational Linguistics 2020 • Wietse de Vries, Andreas van Cranenburgh, Malvina Nissim
Peeking into the inner workings of BERT has shown that its layers resemble the classical NLP pipeline, with progressively more complex tasks being concentrated in later layers.
2 code implementations • 19 Dec 2019 • Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, Tommaso Caselli, Gertjan van Noord, Malvina Nissim
The transformer-based pre-trained language model BERT has helped to improve state-of-the-art performance on many natural language processing (NLP) tasks.
Ranked #3 on
Sentiment Analysis
on DBRD
no code implementations • 20 Nov 2019 • Hessel Haagsma, Malvina Nissim, Johan Bos
To further progress on the extraction and disambiguation of potentially idiomatic expressions, larger corpora of PIEs are required.
no code implementations • ACL 2019 • Angelo Basile, Albert Gatt, Malvina Nissim
Inspired by Labov's seminal work on stylistic variation as a function of social stratification, we develop and compare neural models that predict a person's presumed socio-economic status, obtained through distant supervision, from their writing style on social media.
1 code implementation • 23 May 2019 • Malvina Nissim, Rik van Noord, Rob van der Goot
However, beside the intrinsic problems with the analogy task as a bias detection tool, in this paper we show that a series of issues related to how analogies have been implemented and used might have yielded a distorted picture of bias in word embeddings.
no code implementations • COLING 2018 • Hessel Haagsma, Malvina Nissim, Johan Bos
Disambiguation of potentially idiomatic expressions involves determining the sense of a potentially idiomatic expression in a given context, e. g. determining that make hay in {`}Investment banks made hay while takeovers shone.
no code implementations • SEMEVAL 2018 • Artur Kulmizev, Mostafa Abdou, Vinit Ravishankar, Malvina Nissim
We participated to the SemEval-2018 shared task on capturing discriminative attributes (Task 10) with a simple system that ranked 8th amongst the 26 teams that took part in the evaluation.
1 code implementation • ACL 2018 • Rob van der Goot, Nikola Ljubešić, Ian Matroos, Malvina Nissim, Barbara Plank
Gender prediction has typically focused on lexical and social network features, yielding good performance, but making systems highly language-, topic-, and platform-dependent.
no code implementations • WS 2017 • Artur Kulmizev, Bo Blankers, Johannes Bjerva, Malvina Nissim, Gertjan van Noord, Barbara Plank, Martijn Wieling
In this paper, we explore the performance of a linear SVM trained on language independent character features for the NLI Shared Task 2017.
1 code implementation • WS 2017 • Rob van der Goot, Barbara Plank, Malvina Nissim
Does normalization help Part-of-Speech (POS) tagging accuracy on noisy, non-canonical data?
no code implementations • 12 Jul 2017 • Angelo Basile, Gareth Dwyer, Maria Medvedeva, Josine Rawee, Hessel Haagsma, Malvina Nissim
We describe our participation in the PAN 2017 shared task on Author Profiling, identifying authors' gender and language variety for English, Spanish, Arabic and Portuguese.
no code implementations • 10 Nov 2016 • Marco Del Tredici, Malvina Nissim, Andrea Zaninello
From a diachronic corpus of Italian, we build consecutive vector spaces in time and use them to compare a term's cosine similarity to itself in different time spans.
no code implementations • 9 Nov 2016 • Barbara Plank, Malvina Nissim
We bootstrap a state-of-the-art part-of-speech tagger to tag Italian Twitter data, in the context of the Evalita 2016 PoSTWITA shared task.
no code implementations • WS 2016 • Chris Pool, Malvina Nissim
We exploit the Facebook reaction feature in a distant supervised fashion to train a support vector machine classifier for emotion detection, using several feature combinations and combining different Facebook pages.
no code implementations • LREC 2016 • Lennart Kloppenburg, Malvina Nissim
The first is a binary model for detecting whether a preposition should be used at all in a given position or not.
no code implementations • LREC 2014 • Marco Del Tredici, Malvina Nissim
We introduce a modular rule-based approach to text categorisation which is more flexible and less time consuming to build than a standard rule-based system because it works with a hierarchical structure and allows for re-usability of rules.