Search Results for author: Carolina Scarton

Found 77 papers, 26 papers with code

Revisiting Rumour Stance Classification: Dealing with Imbalanced Data

no code implementations RDSM (COLING) 2020 Yue Li, Carolina Scarton

Correctly classifying stances of replies can be significantly helpful for the automatic detection and classification of online rumours.

Classification Rumour Detection +1

Controlling Extra-Textual Attributes about Dialogue Participants: A Case Study of English-to-Polish Neural Machine Translation

no code implementations EAMT 2022 Sebastian T. Vincent, Loïc Barrault, Carolina Scarton

We focus on the underresearched problem of utilising external metadata in automatic translation of TV dialogue, proposing a case study where a wide range of approaches for controlling attributes in translation is employed in a multi-attribute scenario.

Attribute Machine Translation +2

The (Un)Suitability of Automatic Evaluation Metrics for Text Simplification

1 code implementation CL (ACL) 2021 Fernando Alva-Manchego, Carolina Scarton, Lucia Specia

Second, we conduct the first meta-evaluation of automatic metrics in Text Simplification, using our new data set (and other existing data) to analyze the variation of the correlation between metrics’ scores and human judgments across three dimensions: the perceived simplicity level, the system type, and the set of references used for computation.

Sentence Text Simplification

GateNLP at SemEval-2025 Task 10: Hierarchical Three-Step Prompting for Multilingual Narrative Classification

1 code implementation28 May 2025 Iknoor Singh, Carolina Scarton, Kalina Bontcheva

The proliferation of online news and the increasing spread of misinformation necessitate robust methods for automatic data analysis.

Articles Language Modeling +3

SCRum-9: Multilingual Stance Classification over Rumours on Social Media

no code implementations25 May 2025 Yue Li, Jake Vasilakes, Zhixue Zhao, Carolina Scarton

We introduce SCRum-9, a multilingual dataset for Rumour Stance Classification, containing 7, 516 tweet-reply pairs from X. SCRum-9 goes beyond existing stance classification datasets by covering more languages (9), linking examples to more fact-checked claims (2. 1k), and including complex annotations from multiple annotators to account for intra- and inter-annotator variability.

Rumour Detection Stance Classification

UKElectionNarratives: A Dataset of Misleading Narratives Surrounding Recent UK General Elections

no code implementations8 May 2025 Fatima Haouari, Carolina Scarton, Nicolò Faggiani, Nikolaos Nikolaidis, Bonka Kotseva, Ibrahim Abu Farha, Jens Linge, Kalina Bontcheva

Misleading narratives play a crucial role in shaping public opinion during elections, as they can influence how voters perceive candidates and political parties.

Exploring Vision Language Models for Multimodal and Multilingual Stance Detection

no code implementations29 Jan 2025 Jake Vasilakes, Carolina Scarton, Zhixue Zhao

Our results show that VLMs generally rely more on text than images for stance detection and this trend persists across languages.

Stance Detection

Leveraging Large Language Models for Zero-shot Lay Summarisation in Biomedicine and Beyond

no code implementations9 Jan 2025 Tomas Goldsack, Carolina Scarton, Chenghua Lin

In this work, we explore the application of Large Language Models to zero-shot Lay Summarisation.

Articles

A Cross-Domain Study of the Use of Persuasion Techniques in Online Disinformation

no code implementations19 Dec 2024 João A. Leite, Olesya Razuvayevskaya, Carolina Scarton, Kalina Bontcheva

Disinformation, irrespective of domain or language, aims to deceive or manipulate public opinion, typically through employing advanced persuasion techniques.

Persuasion Strategies

Investigating Idiomaticity in Word Representations

1 code implementation4 Nov 2024 wei he, Tiago Kramer Vieira, Marcos Garcia, Carolina Scarton, Marco Idiart, Aline Villavicencio

Idiomatic expressions are an integral part of human languages, often used to express complex ideas in compressed or conventional ways (e. g. eager beaver as a keen and enthusiastic person).

Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Models

no code implementations24 Oct 2024 Yue Li, Zhixue Zhao, Carolina Scarton

In-context learning (ICL) performance is known to be sensitive to the prompt design, yet the impact of class label options in zero-shot classification has been largely overlooked.

Classification In-Context Learning +3

Overview of the BioLaySumm 2024 Shared Task on the Lay Summarization of Biomedical Research Articles

no code implementations16 Aug 2024 Tomas Goldsack, Carolina Scarton, Matthew Shardlow, Chenghua Lin

This paper presents the setup and results of the second edition of the BioLaySumm shared task on the Lay Summarisation of Biomedical Research Articles, hosted at the BioNLP Workshop at ACL 2024.

Articles Lay Summarization

A Case Study on Contextual Machine Translation in a Professional Scenario of Subtitling

no code implementations27 Jun 2024 Sebastian Vincent, Charlotte Prescott, Chris Bayliss, Chris Oakley, Carolina Scarton

Incorporating extra-textual context such as film metadata into the machine translation (MT) pipeline can enhance translation quality, as indicated by automatic evaluation in recent work.

Machine Translation Translation

Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss

no code implementations21 Jun 2024 wei he, Marco Idiart, Carolina Scarton, Aline Villavicencio

Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP).

Contrastive Learning Machine Translation +1

EUvsDisinfo: A Dataset for Multilingual Detection of Pro-Kremlin Disinformation in News Articles

1 code implementation18 Jun 2024 João A. Leite, Olesya Razuvayevskaya, Kalina Bontcheva, Carolina Scarton

This work introduces EUvsDisinfo, a multilingual dataset of disinformation articles originating from pro-Kremlin outlets, along with trustworthy articles from credible / less biased sources.

Articles

ATLAS: Improving Lay Summarisation with Attribute-based Control

no code implementations9 Jun 2024 Zhihao Zhang, Tomas Goldsack, Carolina Scarton, Chenghua Lin

Lay summarisation aims to produce summaries of scientific articles that are comprehensible to non-expert audiences.

Articles Attribute

ExU: AI Models for Examining Multilingual Disinformation Narratives and Understanding their Spread

no code implementations30 May 2024 Jake Vasilakes, Zhixue Zhao, Ivan Vykopal, Michal Gregor, Martin Hyben, Carolina Scarton

We describe the ExU project proposal and summarise the results of a user requirements survey regarding the design of tools to support fact-checking.

Fact Checking Retrieval +2

Word Boundary Information Isn't Useful for Encoder Language Models

no code implementations15 Jan 2024 Edward Gow-Smith, Dylan Phelps, Harish Tayyar Madabushi, Carolina Scarton, Aline Villavicencio

As such, removing these symbols has been shown to have a beneficial effect on the processing of morphologically complex words for transformer encoders in the pretrain-finetune paradigm.

NER Sentence

Don't Waste a Single Annotation: Improving Single-Label Classifiers Through Soft Labels

no code implementations9 Nov 2023 Ben Wu, Yue Li, Yida Mu, Carolina Scarton, Kalina Bontcheva, Xingyi Song

In this paper, we address the limitations of the common data annotation and training methods for objective single-label classification tasks.

Enhancing Biomedical Lay Summarisation with External Knowledge Graphs

1 code implementation24 Oct 2023 Tomas Goldsack, Zhihao Zhang, Chen Tang, Carolina Scarton, Chenghua Lin

Previous approaches for automatic lay summarisation are exclusively reliant on the source article that, given it is written for a technical audience (e. g., researchers), is unlikely to explicitly define all technical concepts or state all of the background information that is relevant for a lay audience.

Decoder Knowledge Graphs

Analysing State-Backed Propaganda Websites: a New Dataset and Linguistic Study

1 code implementation21 Oct 2023 Freddy Heppell, Kalina Bontcheva, Carolina Scarton

This paper analyses two hitherto unstudied sites sharing state-backed disinformation, Reliable Recent News (rrn. world) and WarOnFakes (waronfakes. com), which publish content in Arabic, Chinese, English, French, German, and Spanish.

Articles

Overview of the BioLaySumm 2023 Shared Task on Lay Summarization of Biomedical Research Articles

no code implementations29 Sep 2023 Tomas Goldsack, Zheheng Luo, Qianqian Xie, Carolina Scarton, Matthew Shardlow, Sophia Ananiadou, Chenghua Lin

This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm), hosted at the BioNLP Workshop at ACL 2023.

Articles Lay Summarization

Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals

no code implementations14 Sep 2023 João A. Leite, Olesya Razuvayevskaya, Kalina Bontcheva, Carolina Scarton

This paper introduces Pastel (Prompted weAk Supervision wiTh crEdibility signaLs), a weakly supervised approach that leverages large language models (LLMs) to extract credibility signals from web content, and subsequently combines them to predict the veracity of content without relying on human supervision.

Misinformation Veracity Classification

Noisy Self-Training with Data Augmentations for Offensive and Hate Speech Detection Tasks

1 code implementation31 Jul 2023 João A. Leite, Carolina Scarton, Diego F. Silva

Online social media is rife with offensive and hateful comments, prompting the need for their automatic detection given the sheer amount of posts created every second.

 Ranked #1 on Hate Speech Detection on OLID (using extra training data)

Data Augmentation Hate Speech Detection

MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation

1 code implementation25 May 2023 Sebastian Vincent, Robert Flynn, Carolina Scarton

This work introduces MTCue, a novel neural machine translation (NMT) framework that interprets all context (including discrete variables) as text.

Machine Translation NMT +1

A Large-Scale Comparative Study of Accurate COVID-19 Information versus Misinformation

no code implementations10 Apr 2023 Yida Mu, Ye Jiang, Freddy Heppell, Iknoor Singh, Carolina Scarton, Kalina Bontcheva, Xingyi Song

This motivated us to carry out a comparative study of the characteristics of COVID-19 misinformation versus those of accurate COVID-19 information through a large-scale computational analysis of over 242 million tweets.

Misinformation

Reference-less Analysis of Context Specificity in Translation with Personalised Language Models

1 code implementation29 Mar 2023 Sebastian Vincent, Alice Dowek, Rowanne Sumner, Charlotte Blundell, Emily Preston, Chris Bayliss, Chris Oakley, Carolina Scarton

Our results suggest that the degree to which professional translations in our domain are context-specific can be preserved to a better extent by a contextual machine translation model than a non-contextual model, which is also reflected in the contextual model's superior reference-based scores.

Language Modelling Machine Translation +2

Can We Identify Stance Without Target Arguments? A Study for Rumour Stance Classification

no code implementations22 Mar 2023 Yue Li, Carolina Scarton

Considering a conversation thread, rumour stance classification aims to identify the opinion (e. g. agree or disagree) of replies towards a target (rumour story).

Classification Sentiment Analysis +1

SheffieldVeraAI at SemEval-2023 Task 3: Mono and multilingual approaches for news genre, topic and persuasion technique classification

1 code implementation16 Mar 2023 Ben Wu, Olesya Razuvayevskaya, Freddy Heppell, João A. Leite, Carolina Scarton, Kalina Bontcheva, Xingyi Song

For Subtask 2 (Framing), we achieved first place in 3 languages, and the best average rank across all the languages, by using two separate ensembles: a monolingual RoBERTa-MUPPETLARGE and an ensemble of XLM-RoBERTaLARGE with adapters and task adaptive pretraining.

VaxxHesitancy: A Dataset for Studying Hesitancy towards COVID-19 Vaccination on Twitter

1 code implementation17 Jan 2023 Yida Mu, Mali Jin, Charlie Grimshaw, Carolina Scarton, Kalina Bontcheva, Xingyi Song

Annotated data is also necessary for training data-driven models for more nuanced analysis of attitudes towards vaccination.

Language Modelling

Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature

1 code implementation18 Oct 2022 Tomas Goldsack, Zhihao Zhang, Chenghua Lin, Carolina Scarton

Lay summarisation aims to jointly summarise and simplify a given text, thus making its content more comprehensible to non-experts.

Articles Lay Summarization

Classifying COVID-19 vaccine narratives

no code implementations18 Jul 2022 Yue Li, Carolina Scarton, Xingyi Song, Kalina Bontcheva

This paper addresses the need for monitoring and analysing vaccine narratives online by introducing a novel vaccine narrative classification task, which categorises COVID-19 vaccine claims into one of seven categories.

Data Augmentation

Sample Efficient Approaches for Idiomaticity Detection

no code implementations LREC (MWE) 2022 Dylan Phelps, Xuan-Rui Fan, Edward Gow-Smith, Harish Tayyar Madabushi, Carolina Scarton, Aline Villavicencio

In particular we study the impact of Pattern Exploit Training (PET), a few-shot method of classification, and BERTRAM, an efficient method of creating contextual embeddings, on the task of idiomaticity detection.

Controlling Formality in Low-Resource NMT with Domain Adaptation and Re-Ranking: SLT-CDT-UoS at IWSLT2022

no code implementations IWSLT (ACL) 2022 Sebastian T. Vincent, Loïc Barrault, Carolina Scarton

This paper describes the SLT-CDT-UoS group's submission to the first Special Task on Formality Control for Spoken Language Translation, part of the IWSLT 2022 Evaluation Campaign.

Domain Adaptation Low Resource NMT +4

Controlling Extra-Textual Attributes about Dialogue Participants -- A Case Study of English-to-Polish Neural Machine Translation

no code implementations10 May 2022 Sebastian T. Vincent, Loïc Barrault, Carolina Scarton

We focus on the underresearched problem of utilising external metadata in automatic translation of TV dialogue, proposing a case study where a wide range of approaches for controlling attributes in translation is employed in a multi-attribute scenario.

Attribute Machine Translation +2

SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding

1 code implementation SemEval (NAACL) 2022 Harish Tayyar Madabushi, Edward Gow-Smith, Marcos Garcia, Carolina Scarton, Marco Idiart, Aline Villavicencio

This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence Embedding, which consists of two subtasks: (a) a binary classification task aimed at identifying whether a sentence contains an idiomatic expression, and (b) a task based on semantic text similarity which requires the model to adequately represent potentially idiomatic expressions in context.

Binary Classification Sentence +4

Improving Tokenisation by Alternative Treatment of Spaces

1 code implementation8 Apr 2022 Edward Gow-Smith, Harish Tayyar Madabushi, Carolina Scarton, Aline Villavicencio

We find that our modified algorithms lead to improved performance on downstream NLP tasks that involve handling complex words, whilst having no detrimental effect on performance in general natural language understanding tasks.

Natural Language Understanding

AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models

1 code implementation Findings (EMNLP) 2021 Harish Tayyar Madabushi, Edward Gow-Smith, Carolina Scarton, Aline Villavicencio

Despite their success in a variety of NLP tasks, pre-trained language models, due to their heavy reliance on compositionality, fail in effectively capturing the meanings of multiword expressions (MWEs), especially idioms.

Language Modeling Language Modelling

Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels

1 code implementation ACL 2021 Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, Aline Villavicencio

This paper presents the Noun Compound Type and Token Idiomaticity (NCTTI) dataset, with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token level.

Vocal Bursts Type Prediction

Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of COVID-19 Infodemic

no code implementations22 Jun 2021 Ye Jiang, Xingyi Song, Carolina Scarton, Ahmet Aker, Kalina Bontcheva

In this paper, we introduce a fine-grained annotated misinformation tweets dataset including social behaviours annotation (e. g. comment or question to the misinformation).

Misinformation

Probing for idiomaticity in vector space models

1 code implementation EACL 2021 Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, Aline Villavicencio

Contextualised word representation models have been successfully used for capturing different word usages and they may be an attractive alternative for representing idiomaticity in language.

Multistage BiCross encoder for multilingual access to COVID-19 health information

1 code implementation8 Jan 2021 Iknoor Singh, Carolina Scarton, Kalina Bontcheva

The Coronavirus (COVID-19) pandemic has led to a rapidly growing 'infodemic' of health information online.

Retrieval

ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations

1 code implementation ACL 2020 Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, Lucia Specia

Furthermore, we motivate the need for developing better methods for automatic evaluation using ASSET, since we show that current popular metrics may not be suitable when multiple simplification transformations are performed.

Sentence

Data-Driven Sentence Simplification: Survey and Benchmark

no code implementations CL 2020 Fern Alva-Manchego, o, Carolina Scarton, Lucia Specia

Sentence Simplification (SS) aims to modify a sentence in order to make it easier to read and understand.

Sentence Survey

EASSE: Easier Automatic Sentence Simplification Evaluation

1 code implementation IJCNLP 2019 Fernando Alva-Manchego, Louis Martin, Carolina Scarton, Lucia Specia

We introduce EASSE, a Python package aiming to facilitate and standardise automatic evaluation and comparison of Sentence Simplification (SS) systems.

Sentence

Sheffield Submissions for WMT18 Multimodal Translation Shared Task

no code implementations WS 2018 Chiraag Lala, Pranava Swaroop Madhyastha, Carolina Scarton, Lucia Specia

For task 1b, we explore three approaches: (i) re-ranking based on cross-lingual word sense disambiguation (as for task 1), (ii) re-ranking based on consensus of NMT n-best lists from German-Czech, French-Czech and English-Czech systems, and (iii) data augmentation by generating English source data through machine translation from French to English and from German to English followed by hypothesis selection using a multimodal-reranker.

Data Augmentation Multimodal Machine Translation +4

Learning Simplifications for Specific Target Audiences

no code implementations ACL 2018 Carolina Scarton, Lucia Specia

Text simplification (TS) is a monolingual text-to-text transformation task where an original (complex) text is transformed into a target (simpler) text.

Lexical Simplification Machine Translation +4

MUSST: A Multilingual Syntactic Simplification Tool

no code implementations IJCNLP 2017 Carolina Scarton, Alessio Palmero Aprosio, Sara Tonelli, Tamara Mart{\'\i}n Wanton, Lucia Specia

Our implementation includes a set of general-purpose simplification rules, as well as a sentence selection module (to select sentences to be simplified) and a confidence model (to select only promising simplifications).

Lexical Simplification Sentence +1

Learning How to Simplify From Explicit Labeling of Complex-Simplified Text Pairs

1 code implementation IJCNLP 2017 Fern Alva-Manchego, o, Joachim Bingel, Gustavo Paetzold, Carolina Scarton, Lucia Specia

Current research in text simplification has been hampered by two central problems: (i) the small amount of high-quality parallel simplification data available, and (ii) the lack of explicit annotations of simplification operations, such as deletions or substitutions, on existing data.

Machine Translation Sentence +2

Improving Evaluation of Document-level Machine Translation Quality Estimation

no code implementations EACL 2017 Yvette Graham, Qingsong Ma, Timothy Baldwin, Qun Liu, Carla Parra, Carolina Scarton

Meaningful conclusions about the relative performance of NLP systems are only possible if the gold standard employed in a given evaluation is both valid and reliable.

Document Level Machine Translation Machine Translation +2

Cannot find the paper you are looking for? You can Submit a new open access paper.