Search Results for author: Suzan Verberne

Found 47 papers, 21 papers with code

Conversation-Aware Filtering of Online Patient Forum Messages

no code implementations SMM4H (COLING) 2020 Anne Dirkson, Suzan Verberne, Wessel Kraaij

We experiment with two approaches to add conversational context to a BERT model: a sequential CRF layer and manually engineered features.

Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study

2 code implementations5 Apr 2024 Myrthe Reuver, Suzan Verberne, Antske Fokkens

In this paper, we investigate the robustness of operationalization choices for few-shot stance detection, with special attention to modelling stance across different topics.

Few-Shot Stance Detection Natural Language Inference

CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems

no code implementations27 Mar 2024 Amin Abolghasemi, Zhaochun Ren, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke, Suzan Verberne

In this work, we leverage large language models (LLMs) and unlock their ability to generate satisfaction-aware counterfactual dialogues to augment the set of original dialogues of a test collection.

counterfactual Data Augmentation +1

Measuring Bias in a Ranked List using Term-based Representations

no code implementations9 Mar 2024 Amin Abolghasemi, Leif Azzopardi, Arian Askari, Maarten de Rijke, Suzan Verberne

With TExFAIR, we extend the AWRF framework to allow for the evaluation of fairness in settings with term-based representations of groups in documents in a ranked list.

Document Ranking Fairness +1

Learning to Use Tools via Cooperative and Interactive Agents

no code implementations5 Mar 2024 Zhengliang Shi, Shen Gao, Xiuyi Chen, Lingyong Yan, Haibo Shi, Dawei Yin, Zhumin Chen, Pengjie Ren, Suzan Verberne, Zhaochun Ren

Tool learning empowers large language models (LLMs) as agents to use external tools to extend their capability.

A Multi-Agent Conversational Recommender System

no code implementations2 Feb 2024 Jiabao Fang, Shen Gao, Pengjie Ren, Xiuying Chen, Suzan Verberne, Zhaochun Ren

First, we design a multi-agent act planning framework, which can control the dialogue flow based on four LLM-based agents.

Recommendation Systems

Answer Retrieval in Legal Community Question Answering

1 code implementation9 Jan 2024 Arian Askari, Zihui Yang, Zhaochun Ren, Suzan Verberne

Furthermore, we propose LegalQA: a real-world benchmark dataset for evaluating answer retrieval in the legal domain.

Community Question Answering Retrieval

ChatGPT as a commenter to the news: can LLMs generate human-like opinions?

1 code implementation21 Dec 2023 Rayden Tseng, Suzan Verberne, Peter van der Putten

ChatGPT, GPT-3. 5, and other large language models (LLMs) have drawn significant attention since their release, and the abilities of these models have been investigated for a wide variety of tasks.

ChiSCor: A Corpus of Freely Told Fantasy Stories by Dutch Children for Computational Linguistics and Cognitive Science

no code implementations31 Oct 2023 Bram M. A. van Dijk, Max J. van Duijn, Suzan Verberne, Marco R. Spruit

ChiSCor was compiled for studying how children render character perspectives, and unravelling language and cognition in development, with computational tools.

LEMMA valid

Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection

no code implementations12 Sep 2023 Sophia Althammer, Guido Zuccon, Sebastian Hofstätter, Suzan Verberne, Allan Hanbury

We further find that gains provided by AL strategies come at the expense of more assessments (thus higher annotation costs) and AL strategies underperform random selection when comparing effectiveness given a fixed annotation cost.

Active Learning Domain Adaptation

Fine-grained Affective Processing Capabilities Emerging from Large Language Models

no code implementations4 Sep 2023 Joost Broekens, Bernhard Hilpert, Suzan Verberne, Kim Baraka, Patrick Gebhard, Aske Plaat

Large language models, in particular generative pre-trained transformers (GPTs), show impressive results on a wide variety of language-related tasks.

Sentiment Analysis

Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts

1 code implementation3 May 2023 Arian Askari, Mohammad Aliannejadi, Evangelos Kanoulas, Suzan Verberne

We introduce a new dataset, ChatGPT-RetrievalQA, and compare the effectiveness of models fine-tuned on LLM-generated and human-generated data.

Re-Ranking Retrieval

Retrieval for Extremely Long Queries and Documents with RPRS: a Highly Efficient and Effective Transformer-based Re-Ranker

1 code implementation2 Mar 2023 Arian Askari, Suzan Verberne, Amin Abolghasemi, Wessel Kraaij, Gabriella Pasi

Furthermore, our method solves the problem of the low-resource training in QBD retrieval tasks as it does not need large amounts of training data, and has only three parameters with a limited range that can be optimized with a grid search even if a small amount of labeled data is available.

Information Retrieval Retrieval

Injecting the BM25 Score as Text Improves BERT-Based Re-rankers

1 code implementation23 Jan 2023 Arian Askari, Amin Abolghasemi, Gabriella Pasi, Wessel Kraaij, Suzan Verberne

In this paper we propose a novel approach for combining first-stage lexical retrieval models and Transformer-based re-rankers: we inject the relevance score of the lexical model as a token in the middle of the input of the cross-encoder re-ranker.


On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval

1 code implementation11 Oct 2022 Amin Abolghasemi, Arian Askari, Suzan Verberne

In this work, we examine the generalizability of two of these deep contextualized term-based models in the context of query-by-example (QBE) retrieval in which a seed document acts as the query to find relevant documents.


LeiBi@COLIEE 2022: Aggregating Tuned Lexical Models with a Cluster-driven BERT-based Model for Case Law Retrieval

no code implementations26 May 2022 Arian Askari, Georgios Peikos, Gabriella Pasi, Suzan Verberne

Our methodology consists of four steps; in detail, given a legal case as a query, we reformulate it by extracting various meaningful sentences or n-grams.

Re-Ranking Retrieval +1

Improving BERT-based Query-by-Document Retrieval with Multi-Task Optimization

no code implementations1 Feb 2022 Amin Abolghasemi, Suzan Verberne, Leif Azzopardi

Query-by-document (QBD) retrieval is an Information Retrieval task in which a seed document acts as the query and the goal is to retrieve related documents -- it is particular common in professional search tasks.

Information Retrieval Representation Learning +1

Expert Finding in Legal Community Question Answering

1 code implementation19 Jan 2022 Arian Askari, Suzan Verberne, Gabriella Pasi

In the legal domain, there is a large knowledge gap between the experts and the searchers, and the content on the legal QA websites consist of a combination formal and informal communication.

Community Question Answering

PARM: A Paragraph Aggregation Retrieval Model for Dense Document-to-Document Retrieval

1 code implementation5 Jan 2022 Sophia Althammer, Sebastian Hofstätter, Mete Sertkan, Suzan Verberne, Allan Hanbury

However in the web domain we are in a setting with large amounts of training data and a query-to-passage or a query-to-document retrieval task.

Passage Retrieval Retrieval

Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Study

no code implementations14 Oct 2021 Myrthe Reuver, Suzan Verberne, Roser Morante, Antske Fokkens

Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics.

Specificity Stance Detection

Small data problems in political research: a critical replication study

no code implementations27 Sep 2021 Hugo de Vos, Suzan Verberne

In an often-cited 2019 paper on the use of machine learning in political research, Anastasopoulos & Whitford (A&W) propose a text classification method for tweets related to organizational reputation.

text-classification Text Classification

Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack

1 code implementation23 Sep 2021 Anne Dirkson, Suzan Verberne, Wessel Kraaij

Experimental results show that BERT models are vulnerable to variation in the entity context with 20. 2 to 45. 0% of entities predicted completely wrong and another 29. 3 to 53. 3% of entities predicted wrong partially.

Adversarial Attack Decision Making +3

DoSSIER@COLIEE 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval

1 code implementation9 Aug 2021 Sophia Althammer, Arian Askari, Suzan Verberne, Allan Hanbury

We address this challenge by combining lexical and dense retrieval methods on the paragraph-level of the cases for the first stage retrieval.

Passage Retrieval Re-Ranking +1

Can BERT Dig It? -- Named Entity Recognition for Information Retrieval in the Archaeology Domain

no code implementations14 Jun 2021 Alex Brandsen, Suzan Verberne, Karsten Lambers, Milco Wansleeben

We also investigate ensemble methods for combining multiple BERT models, and combining the best BERT model with a domain thesaurus using Conditional Random Fields (CRF).

Information Retrieval named-entity-recognition +4

Helping results assessment by adding explainable elements to the deep relevance matching model

1 code implementation9 Jun 2021 Ioannis Chios, Suzan Verberne

This indicates that the explainability of the search engine result page leads to a better user experience.

Improving reference mining in patents with BERT

1 code implementation4 Jan 2021 Ken Voskuil, Suzan Verberne

In this paper we build on prior work using Conditional Random Fields (CRF) and Flair for reference extraction.

Named Entity Recognition for Chinese biomedical patents

1 code implementation COLING 2020 Yuting Hu, Suzan Verberne

There is a large body of work on Biomedical Entity Recognition (Bio-NER) for English but there have only been a few attempts addressing NER for Chinese biomedical texts.

named-entity-recognition Named Entity Recognition +1

Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering

no code implementations19 Oct 2020 Jeroen Offerijns, Suzan Verberne, Tessa Verhoef

In this work, we train a GPT-2 language model to generate three distractors for a given question and text context, using the RACE dataset.

Distractor Generation Language Modelling +4

Challenges of Applying Automatic Speech Recognition for Transcribing EU Parliament Committee Meetings: A Pilot Study

no code implementations LREC 2020 Hugo de Vos, Suzan Verberne

Challenges of Applying Automatic Speech Recognition for Transcribing EUParliament Committee Meetings: A Pilot StudyHugo de Vos and Suzan VerberneInstitute of Public Administration and Leiden Institute of Advanced Computer Science, Leiden Universityh. p. de. vos@fgga. leidenuniv. nl, s. verberne@liacs. leidenuniv. nlAbstractWe tested the feasibility of automatically transcribing committee meetings of the European Union parliament with the use of AutomaticSpeech Recognition techniques.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviews

2 code implementations2 Oct 2019 Benjamin van der Burgh, Suzan Verberne

We evaluated the effectiveness of using language models, that were pre-trained in one domain, as the basis for a classification model in another domain: Dutch book reviews.

Classification General Classification +3

Lexical Normalization of User-Generated Medical Text

no code implementations WS 2019 Anne Dirkson, Suzan Verberne, Wessel Kraaij

In the medical domain, user-generated social media text is increasingly used as a valuable complementary knowledge source to scientific medical literature.

Lexical Normalization Mistake Detection +1

Transfer Learning for Health-related Twitter Data

1 code implementation WS 2019 Anne Dirkson, Suzan Verberne

Transfer learning is promising for many NLP applications, especially in tasks with limited labeled data.

Transfer Learning

Information search in a professional context - exploring a collection of professional search tasks

no code implementations11 May 2019 Suzan Verberne, Jiyin He, Gineke Wiggers, Tony Russell-Rose, Udo Kruschwitz, Arjen P. de Vries

Search conducted in a work context is an everyday activity that has been around since long before the Web was invented, yet we still seem to understand little about its general characteristics.

Cannot find the paper you are looking for? You can Submit a new open access paper.