no code implementations • 25 Oct 2024 • Clemencia Siro, Yifei Yuan, Mohammad Aliannejadi, Maarten de Rijke
Human evaluation and CrowdLLM show that the AGENT-CQ - generation stage, consistently outperforms baselines in various aspects of question and answer quality.
no code implementations • 16 Oct 2024 • Amin Abolghasemi, Leif Azzopardi, Seyyed Hadi Hashemi, Maarten de Rijke, Suzan Verberne
Our results show that adding authorship information to source documents can significantly change the attribution quality of LLMs by 3% to 18%.
no code implementations • 10 Oct 2024 • Yougang Lyu, Lingyong Yan, Zihan Wang, Dawei Yin, Pengjie Ren, Maarten de Rijke, Zhaochun Ren
As large language models (LLMs) are rapidly advancing and achieving near-human capabilities, aligning them with human values is becoming more urgent.
no code implementations • 8 Oct 2024 • Zhaochun Ren, Xiangnan He, Dawei Yin, Maarten de Rijke
Methods for information discovery in e-commerce largely focus on improving the effectiveness of e-commerce search and recommender systems, on enriching and using knowledge graphs to support e-commerce, and on developing innovative question answering and bot-based solutions that help to connect people to goods and services.
1 code implementation • 4 Oct 2024 • Vaishali Pal, Evangelos Kanoulas, Andrew Yates, Maarten de Rijke
TableQA models trained on our large-scale datasets outperform state-of-the-art LLMs.
no code implementations • 3 Oct 2024 • Yougang Lyu, XiaoYu Zhang, Zhaochun Ren, Maarten de Rijke
Despite large language models (LLMs) increasingly becoming important components of news recommender systems, employing LLMs in such systems introduces new risks, such as the influence of cognitive biases in LLMs.
no code implementations • 27 Sep 2024 • Yubao Tang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Xueqi Cheng
GR$^2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training.
1 code implementation • 23 Sep 2024 • Weichao Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text.
no code implementations • 20 Sep 2024 • Mohanna Hoveyda, Arjen P. de Vries, Maarten de Rijke, Harrie Oosterhuis, Faegheh Hasibi
In question answering (QA), different questions can be effectively addressed with different answering strategies.
no code implementations • 15 Sep 2024 • Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
We propose a novel approach, proximal ranking policy optimization (PRPO), that provides safety in deployment without assumptions about user behavior.
1 code implementation • 2 Sep 2024 • Mohanna Hoveyda, Arjen P. de Vries, Maarten de Rijke, Faegheh Hasibi
Entity linking (EL) in conversations faces notable challenges in practical applications, primarily due to the scarcity of entity-annotated conversational datasets and sparse knowledge bases (KB) containing domain-specific, long-tail entities.
1 code implementation • 30 Aug 2024 • XiaoYu Zhang, Ruobing Xie, Yougang Lyu, Xin Xin, Pengjie Ren, Mingfei Liang, Bo Zhang, Zhanhui Kang, Maarten de Rijke, Zhaochun Ren
With empathy we refer to a system's ability to capture and express emotions.
no code implementations • 29 Jul 2024 • Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Our experiments show that both our novel safe doubly robust method and PRPO provide higher performance than the existing safe inverse propensity scoring approach.
no code implementations • 21 Jul 2024 • Mariya Hendriksen, Shuo Zhang, Ridho Reinanda, Mohamed Yahya, Edgar Meij, Maarten de Rijke
We examine the brittleness of the image-text retrieval (ITR) evaluation pipeline with a focus on concept granularity.
no code implementations • 16 Jul 2024 • Yubao Tang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
Generative retrieval uses differentiable search indexes to directly generate relevant document identifiers in response to a query.
1 code implementation • 9 Jul 2024 • Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
Recent advances in neural information retrieval (IR) models have significantly enhanced their effectiveness over various IR tasks.
1 code implementation • 28 Jun 2024 • Xinyi Chen, Baohao Liao, Jirui Qi, Panagiotis Eustratiadis, Christof Monz, Arianna Bisazza, Maarten de Rijke
Following multiple instructions is a crucial ability for large language models (LLMs).
no code implementations • 13 Jun 2024 • Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke
Beyond effectiveness, the robustness of an information retrieval (IR) system is increasingly attracting attention.
no code implementations • 24 May 2024 • Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, Maarten de Rijke
Through the integration of LLMs, ToolRec enables conventional recommender systems to become external tools with a natural language interface.
1 code implementation • 9 May 2024 • Shashank Gupta, Olivier Jeunen, Harrie Oosterhuis, Maarten de Rijke
The foundation of our framework is the derivation of an equivalent baseline correction for all of the existing control variates.
no code implementations • 8 May 2024 • Weijia Zhang, Vaishali Pal, Jia-Hong Huang, Evangelos Kanoulas, Maarten de Rijke
Our approach, which comprises a table serialization module, a summarization controller, and a large language model (LLM), utilizes textual queries and multiple tables to generate query-dependent table summaries tailored to users' information needs.
no code implementations • 2 May 2024 • Ming Li, Yuanna Liu, Sami Jullien, Mozhdeh Ariannezhad, Mohammad Aliannejadi, Andrew Yates, Maarten de Rijke
So far, most NBR studies have focused on optimizing the accuracy of the recommendation, whereas optimizing for beyond-accuracy metrics, e. g., item fairness and diversity remains largely unexplored.
1 code implementation • 1 May 2024 • Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Despite the fact that the extreme sparsity of preference elicitation interactions make them severely more prone to selection bias than natural interactions, the effect of selection bias in preference elicitation on the resulting recommendations has not been studied yet.
1 code implementation • 29 Apr 2024 • Jin Huang, Harrie Oosterhuis, Masoud Mansoury, Herke van Hoof, Maarten de Rijke
Debiasing methods aim to mitigate the effect of selection bias on the evaluation and optimization of RSs.
1 code implementation • 28 Apr 2024 • Chuan Meng, Negar Arabzadeh, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke
RLT is crucial for re-ranking as it can improve re-ranking efficiency by sending variable-length candidate lists to a re-ranker on a per-query basis.
1 code implementation • 26 Apr 2024 • WenHao Zhang, Mengqi Zhang, Shiguang Wu, Jiahuan Pei, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, Pengjie Ren
However, in information retrieval community, there is little research on exclusionary retrieval, where users express what they do not want in their queries.
1 code implementation • 19 Apr 2024 • Peibo Li, Maarten de Rijke, Hao Xue, Shuang Ao, Yang song, Flora D. Salim
Our results show that the proposed framework outperforms the state-of-the-art models in all three datasets.
1 code implementation • 19 Apr 2024 • Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
Workers are more susceptible to user feedback on usefulness and interestingness compared to LLMs on interestingness and relevance.
1 code implementation • 18 Apr 2024 • Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis
Stochastic learning to rank (LTR) is a recent branch in the LTR field that concerns the optimization of probabilistic ranking models.
1 code implementation • 15 Apr 2024 • Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
Using the first user utterance as context leads to consistent ratings, akin to those obtained using the entire dialogue, with significantly reduced annotation effort.
2 code implementations • 3 Apr 2024 • Philipp Hager, Romain Deffayet, Jean-Michel Renders, Onno Zoeter, Maarten de Rijke
Our experiments reveal that gains in click prediction do not necessarily translate to enhanced ranking performance on expert relevance annotations, implying that conclusions strongly depend on how success is measured in this benchmark.
no code implementations • 2 Apr 2024 • Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
However, limiting perturbations to a single level of granularity may reduce the flexibility of adversarial examples, thereby diminishing the potential threat of the attack.
1 code implementation • 1 Apr 2024 • Chuan Meng, Negar Arabzadeh, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke
To solve the challenges, we devise an approximation strategy to predict an IR measure considering recall and propose to fine-tune open-source LLMs using human-labeled relevance judgments.
1 code implementation • 31 Mar 2024 • Shiguang Wu, Wenda Wei, Mengqi Zhang, Zhumin Chen, Jun Ma, Zhaochun Ren, Maarten de Rijke, Pengjie Ren
Both methods compute relevance as a sum of products of query and document vectors and an alignment matrix.
1 code implementation • 28 Mar 2024 • Hengran Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
Retrieval-augmented generation (RAG) is considered to be a promising approach to alleviate the hallucination issue of large language models (LLMs), and it has received widespread attention from researchers recently.
no code implementations • 27 Mar 2024 • Amin Abolghasemi, Zhaochun Ren, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke, Suzan Verberne
In this work, we leverage large language models (LLMs) and unlock their ability to generate satisfaction-aware counterfactual dialogues to augment the set of original dialogues of a test collection.
1 code implementation • 24 Mar 2024 • Maria Heuss, Maarten de Rijke, Avishek Anand
We evaluate RankingSHAP for commonly used learning-to-rank datasets to showcase the more nuanced use of an attribution method while highlighting the limitations of selection-based explanations.
1 code implementation • 19 Mar 2024 • Yubao Tang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Xueqi Cheng
Specifically, we view the generation of a ranked docid list as a sequence learning process: at each step we learn a subset of parameters that maximizes the corresponding generation likelihood of the $i$-th docid given the (preceding) top $i-1$ docids.
no code implementations • 9 Mar 2024 • Amin Abolghasemi, Leif Azzopardi, Arian Askari, Maarten de Rijke, Suzan Verberne
With TExFAIR, we extend the AWRF framework to allow for the evaluation of fairness in settings with term-based representations of groups in documents in a ranked list.
1 code implementation • 27 Feb 2024 • Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, Jiahuan Pei
Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large language models (LLMs), especially as the models' scale and the diversity of tasks increase.
1 code implementation • 27 Feb 2024 • Maurits Bleeker, Mariya Hendriksen, Andrew Yates, Maarten de Rijke
Hence, contrastive losses are not sufficient to learn task-optimal representations, i. e., representations that contain all task-relevant information shared between the image and associated captions.
1 code implementation • 27 Feb 2024 • Thong Nguyen, Mariya Hendriksen, Andrew Yates, Maarten de Rijke
Our proposed approach efficiently transforms dense vectors from a frozen dense model into sparse lexical vectors.
1 code implementation • 26 Feb 2024 • Jiafeng Guo, Changjiang Zhou, Ruqing Zhang, Jiangui Chen, Maarten de Rijke, Yixing Fan, Xueqi Cheng
Very recently, a pre-trained generative retrieval model for KILTs, named CorpusBrain, was proposed and reached new state-of-the-art retrieval performance.
1 code implementation • 24 Feb 2024 • Ziyi Ye, Jingtao Zhan, Qingyao Ai, Yiqun Liu, Maarten de Rijke, Christina Lioma, Tuukka Ruotsalo
If the quality of the initially retrieved documents is low, then the effectiveness of query augmentation would be limited as well.
2 code implementations • 17 Feb 2024 • Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten de Rijke, Zhaochun Ren
To address these problems, we propose a knowledge-aware fine-tuning (KnowTuning) method to improve fine-grained and coarse-grained knowledge awareness of LLMs.
1 code implementation • 12 Feb 2024 • Yifei Yuan, Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke, Wai Lam
Therefore, we propose to add images to clarifying questions and formulate the novel task of asking multimodal clarifying questions in open-domain, mixed-initiative conversational search systems.
no code implementations • 16 Dec 2023 • Yu-An Liu, Ruqing Zhang, Mingkun Zhang, Wei Chen, Maarten de Rijke, Jiafeng Guo, Xueqi Cheng
We decompose the robust ranking error into two components, i. e., a natural ranking error for effectiveness evaluation and a boundary ranking error for assessing adversarial robustness.
1 code implementation • 28 Nov 2023 • Romain Deffayet, Thibaut Thonet, Dongyoon Hwang, Vassilissa Lehoux, Jean-Michel Renders, Maarten de Rijke
Simulators can provide valuable insights for researchers and practitioners who wish to improve recommender systems, because they allow one to easily tweak the experimental setup in which recommender systems operate, and as a result lower the cost of identifying general trends and uncovering novel findings about the candidate methods.
1 code implementation • 16 Nov 2023 • Ziyi Ye, Qingyao Ai, Yiqun Liu, Maarten de Rijke, Min Zhang, Christina Lioma, Tuukka Ruotsalo
Inspired by recent research that revealed associations between the brain and the large computational language models, we propose a generative language BCI that utilizes the capacity of a large language model (LLM) jointly with a semantic brain decoder to directly generate language from functional magnetic resonance imaging (fMRI) input.
1 code implementation • 4 Nov 2023 • Shiguang Wu, Xin Xin, Pengjie Ren, Zhumin Chen, Jun Ma, Maarten de Rijke, Zhaochun Ren
CSRec contains a teacher module that generates high-quality and confident soft labels and a student module that acts as the target recommender and is trained on the combination of dense, soft labels and sparse, one-hot labels.
1 code implementation • 19 Oct 2023 • Olivier Sprangers, Wander Wadman, Sebastian Schelter, Maarten de Rijke
We implement our sparse hierarchical loss function within an existing forecasting model at bol, a large European e-commerce platform, resulting in an improved forecasting performance of 2% at the product level.
1 code implementation • 18 Oct 2023 • Hengran Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence.
1 code implementation • 15 Oct 2023 • Zihan Wang, Ziqi Zhao, Zhumin Chen, Pengjie Ren, Maarten de Rijke, Zhaochun Ren
To address this limitation, recent studies enable generalization to an unseen target domain with only a few labeled examples using data augmentation techniques.
1 code implementation • 18 Sep 2023 • Maria Heuss, Daniel Cohen, Masoud Mansoury, Maarten de Rijke, Carsten Eickhoff
Prior work on bias mitigation often assumes that ranking scores, which correspond to the utility that a document holds for a user, can be accurately determined.
no code implementations • 29 Aug 2023 • Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Yixing Fan, Xueqi Cheng
We put forward a novel Continual-LEarner for generatiVE Retrieval (CLEVER) model and make two major contributions to continual learning for GR: (i) To encode new documents into docids with low computational cost, we present Incremental Product Quantization, which updates a partial quantization codebook according to two adaptive thresholds; and (ii) To memorize new documents for querying without forgetting previous knowledge, we propose a memory-augmented learning mechanism, to form meaningful connections between old and new documents.
no code implementations • 19 Aug 2023 • Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Yixing Fan, Xueqi Cheng
The AREA task is meant to trick DR models into retrieving a target document that is outside the initial set of candidate documents retrieved by the DR model in response to a query.
no code implementations • 5 Aug 2023 • Ali Vardasbi, Maarten de Rijke, Fernando Diaz, Mostafa Dehghani
With group membership bias, the utility of the sensitive groups is under-estimated, hence, without correcting for this bias, a supposedly fair ranking is not truly fair.
1 code implementation • 2 Aug 2023 • Ming Li, Mozhdeh Ariannezhad, Andrew Yates, Maarten de Rijke
In next basket recommendation (NBR), it is useful to distinguish between repeat items, i. e., items that a user has consumed before, and explore items, i. e., items that a user has not consumed before.
1 code implementation • 8 Jul 2023 • Weiwei Sun, Hengyi Cai, Hongshen Chen, Pengjie Ren, Zhumin Chen, Maarten de Rijke, Zhaochun Ren
To provide feasible answers to an ambiguous question, one approach is to directly predict all valid answers, but this can struggle with balancing relevance and diversity.
1 code implementation • 15 Jun 2023 • Gabriel Bénédict, Olivier Jeunen, Samuele Papa, Samarth Bhargav, Daan Odijk, Maarten de Rijke
In this paper we propose RecFusion, which comprise a set of diffusion models for recommendation.
no code implementations • 26 May 2023 • Sami Jullien, Romain Deffayet, Jean-Michel Renders, Paul Groth, Maarten de Rijke
Motivated by the efficiency of $L_2$-based learning, we propose to jointly learn expectiles and quantiles of the return distribution in a way that allows efficient learning while keeping an estimate of the full distribution of returns.
1 code implementation • 22 May 2023 • Vaishali Pal, Andrew Yates, Evangelos Kanoulas, Maarten de Rijke
Recent advances in tabular question answering (QA) with large language models are constrained in their coverage and only answer questions over a single table.
1 code implementation • 18 May 2023 • Chuan Meng, Negar Arabzadeh, Mohammad Aliannejadi, Maarten de Rijke
The QPP task is to predict the retrieval quality of a search system for a query without relevance judgments.
1 code implementation • 17 May 2023 • Zihan Wang, Kai Zhao, Yongquan He, Zhumin Chen, Pengjie Ren, Maarten de Rijke, Zhaochun Ren
Recent work on knowledge graph completion (KGC) focused on learning embeddings of entities and relations in knowledge graphs.
1 code implementation • 9 May 2023 • Xin Xin, Xiangyuan Liu, Hanbing Wang, Pengjie Ren, Zhumin Chen, Jiahuan Lei, Xinlei Shi, Hengliang Luo, Joemon Jose, Maarten de Rijke, Zhaochun Ren
Recommender systems that learn from implicit feedback often use large volumes of a single type of implicit user feedback, such as clicks, to enhance the prediction of sparse target behavior such as purchases.
1 code implementation • 1 May 2023 • Fatemeh Sarvi, Ali Vardasbi, Mohammad Aliannejadi, Sebastian Schelter, Maarten de Rijke
We therefore propose an outlier-aware click model that accounts for both outlier and position bias, called outlier-aware position-based model ( OPBM).
1 code implementation • 28 Apr 2023 • Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yiqun Liu, Yixing Fan, Xueqi Cheng
Learning task-specific retrievers that return relevant contexts at an appropriate level of semantic granularity, such as a document retriever, passage retriever, sentence retriever, and entity retriever, may help to achieve better performance on the end-to-end task.
1 code implementation • 28 Apr 2023 • Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Yixing Fan, Xueqi Cheng
In this paper, we focus on a more general type of perturbation and introduce the topic-oriented adversarial ranking attack task against NRMs, which aims to find an imperceptible perturbation that can promote a target document in ranking for a group of queries with the same topic.
1 code implementation • 26 Apr 2023 • Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
For the CLTR field, our novel exposure-based risk minimization method enables practitioners to adopt CLTR methods in a safer manner that mitigates many of the risks attached to previous methods.
2 code implementations • 19 Apr 2023 • Romain Deffayet, Philipp Hager, Jean-Michel Renders, Maarten de Rijke
We prove that debiasedness is a necessary condition for recovering unbiased and consistent relevance scores and for the invariance of click prediction under covariate shift.
1 code implementation • 4 Mar 2023 • Yujie Lin, Chenyang Wang, Zhumin Chen, Zhaochun Ren, Xin Xin, Qiang Yan, Maarten de Rijke, Xiuzhen Cheng, Pengjie Ren
STEAM first corrects an input item sequence by adjusting the misclicked and/or missed items.
no code implementations • 20 Jan 2023 • Romain Deffayet, Thibaut Thonet, Jean-Michel Renders, Maarten de Rijke
Our findings suggest that representation learning using generative models is a promising direction towards generalizable RL-based slate recommendation.
1 code implementation • 12 Jan 2023 • Mariya Hendriksen, Svitlana Vakulenko, Ernst Kuiper, Maarten de Rijke
Additionally, we select two scene-centric datasets, and three object-centric datasets, and determine the relative performance of the selected models on these datasets.
1 code implementation • 4 Jan 2023 • Yujie Lin, Zhumin Chen, Zhaochun Ren, Chenyang Wang, Qiang Yan, Maarten de Rijke, Xiuzhen Cheng, Pengjie Ren
To address the limitation of sequential recommenders with side information, we define a way to fuse side information and alleviate the problem of missing side information by proposing a unified task, namely the missing information imputation (MII), which randomly masks some feature fields in a given sequence of items, including item IDs, and then forces a predictive model to recover them.
no code implementations • 3 Jan 2023 • Romain Deffayet, Thibaut Thonet, Jean-Michel Renders, Maarten de Rijke
In this paper, we argue that the paradigm commonly adopted for offline evaluation of sequential recommender systems is unsuitable for evaluating reinforcement learning-based recommenders.
1 code implementation • 20 Dec 2022 • Weiwei Sun, Zhengliang Shi, Shen Gao, Pengjie Ren, Maarten de Rijke, Zhaochun Ren
MixCL effectively reduces the hallucination of LMs in conversations and achieves the highest performance among LM-based dialogue agents in terms of relevancy and factuality.
1 code implementation • 11 Dec 2022 • Yougang Lyu, Piji Li, Yechang Yang, Maarten de Rijke, Pengjie Ren, Yukun Zhao, Dawei Yin, Zhaochun Ren
We also propose a dynamic negative sampling strategy to capture the dynamic influence of biases by employing a bias-only model to dynamically select the most similar biased negative samples.
no code implementations • 17 Sep 2022 • Sanne Vrijenhoek, Gabriel Bénédict, Mateo Gutierrez Granada, Daan Odijk, Maarten de Rijke
In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models.
1 code implementation • 14 Sep 2022 • Chen Wu, Ruqing Zhang, Jiafeng Guo, Wei Chen, Yixing Fan, Maarten de Rijke, Xueqi Cheng
A ranking model is said to be Certified Top-$K$ Robust on a ranked list when it is guaranteed to keep documents that are out of the top $K$ away from the top $K$ under any attack.
1 code implementation • 19 Aug 2022 • Ali Vardasbi, Maarten de Rijke, Mostafa Dehghani
Using this result, we propose to train two parallel instances of a linear model, initialized with different random seeds, and use their intersection as a signal to detect overfitting.
no code implementations • 6 Jul 2022 • Ana Lucic, Sheeraz Ahmad, Amanda Furtado Brinhosa, Vera Liao, Himani Agrawal, Umang Bhatt, Krishnaram Kenthapadi, Alice Xiang, Maarten de Rijke, Nicholas Drabowski
In this paper, we report on ongoing work regarding (i) the development of an AI system for flagging and explaining low-quality medical images in real-time, (ii) an interview study to understand the explanation needs of stakeholders using the AI system at OurCompany, and, (iii) a longitudinal user study design to examine the effect of including explanations on the workflow of the technicians in our clinics.
1 code implementation • 24 Jun 2022 • Zihan Wang, Na Huang, Fei Sun, Pengjie Ren, Zhumin Chen, Hengliang Luo, Maarten de Rijke, Zhaochun Ren
To address the above limitations, we propose a Debiasing Learning for Membership Inference Attacks against recommender systems (DL-MIA) framework that has four main components: (1) a difference vector generator, (2) a disentangled encoder, (3) a weight estimator, and (4) an attack model.
no code implementations • 30 May 2022 • Sami Jullien, Mozhdeh Ariannezhad, Paul Groth, Maarten de Rijke
We frame inventory restocking as a new reinforcement learning task that exhibits stochastic behavior conditioned on the agent's actions, making the environment partially observable.
Distributional Reinforcement Learning reinforcement-learning +2
1 code implementation • 25 May 2022 • Maria Heuss, Fatemeh Sarvi, Maarten de Rijke
In this work, we discuss how to approach fairness of exposure in cases where the policy contains rankings of which, due to inter-item dependencies, we cannot reliably estimate the exposure distribution.
1 code implementation • 10 May 2022 • Jin Huang, Harrie Oosterhuis, Bunyamin Cetinkaya, Thijs Rood, Maarten de Rijke
In response to these shortcomings, we reproduce and expand on the existing comparison of attention-based state encoders (1) in the publicly available debiased RL4Rec SOFA simulator with (2) a different RL method, (3) more state encoders, and (4) a different dataset.
1 code implementation • 5 May 2022 • Shaojie Jiang, Ruqing Zhang, Svitlana Vakulenko, Maarten de Rijke
The cross-entropy objective has proved to be an all-purpose training objective for autoregressive language models (LMs).
1 code implementation • 28 Apr 2022 • Maurits Bleeker, Andrew Yates, Maarten de Rijke
We add an additional decoder to the contrastive ICR framework, to reconstruct the input caption in a latent space of a general-purpose sentence encoder, which prevents the image and caption encoder from suppressing predictive features.
1 code implementation • 28 Apr 2022 • Ali Vardasbi, Fatemeh Sarvi, Maarten de Rijke
Different from PL, where pointwise logits are used as the distribution parameters, in PPG pairwise inversion probabilities together with a reference permutation construct the distribution.
no code implementations • 26 Apr 2022 • Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
What is the influence of user experience on the user satisfaction rating of TDS as opposed to, or in addition to, utility?
Conversational Recommendation Task-Oriented Dialogue Systems
1 code implementation • 15 Apr 2022 • Maartje ter Hoeve, Julia Kiseleva, Maarten de Rijke
Motivated from these two angles, we propose a new task: summarization with graphical elements, and we verify that these summaries are helpful for a critical mass of people.
1 code implementation • dialdoc (ACL) 2022 • Vaishali Pal, Evangelos Kanoulas, Maarten de Rijke
In this work, we study parameter-efficient abstractive QA in encoder-decoder models over structured tabular data and unstructured textual data using only 1. 5% additional parameters for each modality.
no code implementations • 4 Apr 2022 • Chen Wu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
We focus on the decision-based black-box attack setting, where the attackers cannot directly get access to the model information, but can only query the target model to obtain the rank positions of the partial retrieved list.
2 code implementations • 2 Apr 2022 • Weiwei Sun, Shuyu Guo, Shuo Zhang, Pengjie Ren, Zhumin Chen, Maarten de Rijke, Zhaochun Ren
Employing existing user simulators to evaluate TDSs is challenging as user simulators are primarily designed to optimize dialogue policies for TDSs and have limited evaluation capabilities.
Conversational Recommendation Task-Oriented Dialogue Systems
1 code implementation • 14 Feb 2022 • Maurits Bleeker, Maarten de Rijke
Recent progress in metric learning has given rise to new loss functions that outperform the triplet loss on tasks such as image retrieval and representation learning.
1 code implementation • 21 Dec 2021 • Mariya Hendriksen, Maurits Bleeker, Svitlana Vakulenko, Nanne van Noord, Ernst Kuiper, Maarten de Rijke
One aspect of this data is a category tree that is being used in search and recommendation.
1 code implementation • 21 Dec 2021 • Fatemeh Sarvi, Maria Heuss, Mohammad Aliannejadi, Sebastian Schelter, Maarten de Rijke
We formalize outlierness in a ranking, show that outliers are present in realistic datasets, and present the results of an eye-tracking study, showing that users scanning order and the exposure of items are influenced by the presence of outliers.
1 code implementation • 6 Dec 2021 • Olivier Sprangers, Sebastian Schelter, Maarten de Rijke
However, these methods require a large number of parameters to be learned, which imposes high memory requirements on the computational resources for training such models.
1 code implementation • 24 Nov 2021 • Jin Huang, Harrie Oosterhuis, Maarten de Rijke
We theoretically show that in a dynamic scenario in which both the selection bias and user preferences are dynamic, existing debiasing methods are no longer unbiased.
no code implementations • 1 Nov 2021 • Ana Lucic, Maurits Bleeker, Sami Jullien, Samarth Bhargav, Maarten de Rijke
In this work, we explain the setup for a technical, graduate-level course on Fairness, Accountability, Confidentiality, and Transparency in Artificial Intelligence (FACT-AI) at the University of Amsterdam, which teaches FACT-AI concepts through the lens of reproducibility.
1 code implementation • 29 Sep 2021 • Ming Li, Sami Jullien, Mozhdeh Ariannezhad, Maarten de Rijke
We propose a set of metrics that measure the repeat/explore ratio and performance of NBR models.
1 code implementation • 1 Sep 2021 • Guojun Yan, Jiahuan Pei, Pengjie Ren, Zhaochun Ren, Xin Xin, Huasheng Liang, Maarten de Rijke, Zhumin Chen
(1) there is no dataset with large-scale medical dialogues that covers multiple medical services and contains fine-grained medical labels (i. e., intents, actions, slots, values), and (2) there is no set of established benchmarks for MDSs for multi-domain, multi-service medical dialogues.
1 code implementation • 24 Aug 2021 • Gabriel Bénédict, Vincent Koops, Daan Odijk, Maarten de Rijke
We propose a loss function, sigmoidF1, which is an approximation of the F1 score that (1) is smooth and tractable for stochastic gradient descent, (2) naturally approximates a multilabel metric, and (3) estimates label propensities and label counts.
1 code implementation • 19 Aug 2021 • Ali Vardasbi, Maarten de Rijke, Ilya Markov
Affine correction (AC) is a generalization of IPS that corrects for position bias and trust bias.
1 code implementation • ACL 2021 • Yangjun Zhang, Pengjie Ren, Maarten de Rijke
HMCEval casts dialogue evaluation as a sample assignment problem, where we need to decide to assign a sample to a human or a machine for evaluation.
1 code implementation • ACL 2021 • Zhongkun Liu, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Maarten de Rijke, Ming Zhou
Conversational Question Simplification (CQS) aims to simplify self-contained questions into conversational ones by incorporating some conversational characteristics, e. g., anaphora and ellipsis.
1 code implementation • In2Writing (ACL) 2022 • Nikos Voskarides, Edgar Meij, Sabrina Sauer, Maarten de Rijke
Given an incomplete narrative that specifies a main event and a context, we aim to retrieve news articles that discuss relevant events that would enable the continuation of the narrative.
1 code implementation • 29 Jun 2021 • Shanshan Wang, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Huasheng Liang, Qiang Yan, Evangelos Kanoulas, Maarten de Rijke
We seek to improve the performance for both frequent and rare ICD codes by using a contrastive graph-based EHR coding framework, CoGraph, which re-casts EHR coding as a few-shot learning task.
no code implementations • 23 Jun 2021 • Muyang Ma, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Huasheng Liang, Jun Ma, Maarten de Rijke
By doing so, the SR model is able to learn how to identify common and unique user preferences, and thereby do better user preference extraction and representation.
1 code implementation • 3 Jun 2021 • Olivier Sprangers, Sebastian Schelter, Maarten de Rijke
We propose Probabilistic Gradient Boosting Machines (PGBM), a method to create probabilistic predictions with a single ensemble of decision trees in a computationally efficient manner.
1 code implementation • 18 May 2021 • Pengjie Ren, Zhongkun Liu, Xiaomeng Song, Hongtao Tian, Zhumin Chen, Zhaochun Ren, Maarten de Rijke
(2) We release a benchmark dataset, called wizard of search engine (WISE), which allows for comprehensive and in-depth research on all aspects of CIS.
no code implementations • 13 May 2021 • Zihan Wang, Hongye Song, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Xiaozhong Liu, Hongsong Li, Maarten de Rijke
First, contract elements are far more fine-grained than named entities, which hinders the transfer of extractors.
Cross-Domain Named Entity Recognition Graph Neural Network +5
1 code implementation • 13 May 2021 • Dongdong Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Miao Fan, Jun Ma, Maarten de Rijke
We propose an end-to-end variational reasoning approach to medical dialogue generation.
1 code implementation • 8 May 2021 • Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Maarten de Rijke
The purpose of the task is to increase the evaluation power of user simulations and to make the simulation more human-like.
1 code implementation • 30 Apr 2021 • Ziming Li, Julia Kiseleva, Maarten de Rijke
The proposed backward reasoning step pushes the model to produce more informative and coherent content because the forward generation step's output is used to infer the dialogue context in the backward direction.
no code implementations • 14 Apr 2021 • Svitlana Vakulenko, Evangelos Kanoulas, Maarten de Rijke
These datasets were collected to inform different dialogue-based tasks including conversational search.
1 code implementation • 16 Feb 2021 • Jiahuan Pei, Pengjie Ren, Maarten de Rijke
We find that CoMemNN is able to enrich user profiles effectively, which results in an improvement of 3. 06% in terms of response selection accuracy compared to state-of-the-art methods.
1 code implementation • 11 Feb 2021 • Harrie Oosterhuis, Maarten de Rijke
We introduce the Generalization and Specialization (GENSPEC) algorithm, a robust feature-based counterfactual LTR method that pursues per-query memorization when it is safe to do so.
1 code implementation • 5 Feb 2021 • Ana Lucic, Maartje ter Hoeve, Gabriele Tolomei, Maarten de Rijke, Fabrizio Silvestri
In this work, we propose a method for generating CF explanations for GNNs: the minimal perturbation to the input (graph) data such that the prediction changes.
no code implementations • 23 Jan 2021 • Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, Tat-Seng Chua
In this paper, we provide a systematic review of the techniques used in current CRSs.
1 code implementation • 18 Jan 2021 • Qintong Li, Piji Li, Xinyi Li, Zhaochun Ren, Zhumin Chen, Maarten de Rijke
In this paper, we propose the abstractive opinion tagging task, where systems have to automatically generate a ranked list of opinion tags that are based on, but need not occur in, a given set of user-generated reviews.
no code implementations • 16 Dec 2020 • Mariya Hendriksen, Ernst Kuiper, Pim Nauts, Sebastian Schelter, Maarten de Rijke
In this paper, we focus on purchase prediction for both anonymous and identified sessions on an e-commerce platform.
no code implementations • NAACL 2022 • Maartje ter Hoeve, Julia Kiseleva, Maarten de Rijke
Motivated by our findings, we present ways to mitigate this mismatch in future research on automatic summarization: we propose research directions that impact the design, the development and the evaluation of automatically generated summaries.
1 code implementation • 8 Dec 2020 • Harrie Oosterhuis, Maarten de Rijke
With the introduction of the intervention-aware estimator, we aim to bridge the online/counterfactual LTR division as it is shown to be highly effective in both online and counterfactual scenarios.
1 code implementation • 7 Dec 2020 • Svitlana Vakulenko, Vadim Savenkov, Maarten de Rijke
How can we better understand the mechanisms behind multi-turn information seeking dialogues?
1 code implementation • 1 Dec 2020 • Muyang Ma, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Lifan Zhao, Jun Ma, Maarten de Rijke
One of the key challenges in cross-domain sequential recommendation is to grasp and transfer the flow of information from multiple domains so as to promote recommendations in all domains.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Ziming Li, Sungjin Lee, Baolin Peng, Jinchao Li, Julia Kiseleva, Maarten de Rijke, Shahin Shayandeh, Jianfeng Gao
Reinforcement learning methods have emerged as a popular choice for training an efficient and effective dialogue policy.
no code implementations • CIKM 2020 • Zhiqiang Pan, Fei Cai, Wanyu Chen, Honghui Chen, Maarten de Rijke
The proposed SGNN-HN applies a star graph neural network (SGNN) to model the complex transition relationship between items in an ongoing session.
Ranked #1 on Session-Based Recommendations on yoochoose1/64
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Ziming Li, Julia Kiseleva, Maarten de Rijke
Then, the traditional multi-label classification solution for dialogue policy learning is extended by adding dense layers to improve the dialogue agent performance.
1 code implementation • 24 Aug 2020 • Ali Vardasbi, Harrie Oosterhuis, Maarten de Rijke
Our main contribution is a new estimator based on affine corrections: it both reweights clicks and penalizes items displayed on ranks with high trust bias.
no code implementations • 21 Aug 2020 • Yangjun Zhang, Pengjie Ren, Maarten de Rijke
In this paper, we define the task of Malevolent Dialogue Response Detection and Classification (MDRDC).
1 code implementation • 7 Aug 2020 • Phillip Lippe, Pengjie Ren, Hinda Haned, Bart Voorn, Maarten de Rijke
Instead of generating a response from scratch, P2-Net generates system responses by paraphrasing template-based responses.
1 code implementation • 3 Aug 2020 • Anton Steenvoorden, Emanuele Di Gloria, Wanyu Chen, Pengjie Ren, Maarten de Rijke
Users prefer diverse recommendations over homogeneous ones.
1 code implementation • 24 Jul 2020 • Harrie Oosterhuis, Maarten de Rijke
LogOpt turns the counterfactual approach - which is indifferent to the logging policy - into an online approach, where the algorithm decides what rankings to display.
1 code implementation • 20 Jul 2020 • Fatemeh Sarvi, Nikos Voskarides, Lois Mooiman, Sebastian Schelter, Maarten de Rijke
As recent learning to match methods have made important advances in bridging the vocabulary gap for these traditional IR areas, we investigate their potential in the context of product search.
no code implementations • 7 Jul 2020 • Yang Fang, Xiang Zhao, Yifan Chen, Weidong Xiao, Maarten de Rijke
We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network.
no code implementations • 19 Jun 2020 • Ziming Li, Julia Kiseleva, Alekh Agarwal, Maarten de Rijke, Ryen W. White
Effective optimization is essential for real-world interactive systems to provide a satisfactory user experience in response to changing user behavior.
no code implementations • 25 May 2020 • Ali Vardasbi, Maarten de Rijke, Ilya Markov
Unbiased CLTR requires click propensities to compensate for the difference between user clicks and true relevance of search results via IPS.
no code implementations • 25 May 2020 • Svitlana Vakulenko, Evangelos Kanoulas, Maarten de Rijke
The ability to engage in mixed-initiative interaction is one of the core requirements for a conversational search system.
1 code implementation • 24 May 2020 • Nikos Voskarides, Dan Li, Pengjie Ren, Evangelos Kanoulas, Maarten de Rijke
Context from the conversational history can be used to arrive at a better expression of the current turn query, defined as the task of query resolution.
1 code implementation • 21 May 2020 • Rolf Jagerman, Maarten de Rijke
Counterfactual Learning to Rank (LTR) algorithms learn a ranking model from logged user interactions, often collected using a production system.
1 code implementation • 18 May 2020 • Harrie Oosterhuis, Maarten de Rijke
We prove that the policy-aware estimator is unbiased if every relevant item has a non-zero probability to appear in the top-k ranking.
no code implementations • 9 May 2020 • Zhiqiang Pan, Fei Cai, Yanxiang Ling, Maarten de Rijke
We employ a modified self-attention mechanism to estimate item importance in a session, which is then used to predict user's long-term preference.
no code implementations • LREC 2020 • Chuan Wu, Evangelos Kanoulas, Maarten de Rijke, Wei Lu
To support research on entity salience, we present a new dataset, the WikiNews Salience dataset (WN-Salience), which can be used to benchmark tasks such as entity salience detection and salient entity linking.
1 code implementation • 29 Apr 2020 • Pengjie Ren, Zhumin Chen, Zhaochun Ren, Evangelos Kanoulas, Christof Monz, Maarten de Rijke
In this paper, we address the problem of answering complex information needs by conversing conversations with search engines, in the sense that users can express their queries in natural language, and directly receivethe information they need from a short system response in a conversational manner.
1 code implementation • 7 Apr 2020 • Ziming Li, Sungjin Lee, Baolin Peng, Jinchao Li, Julia Kiseleva, Maarten de Rijke, Shahin Shayandeh, Jianfeng Gao
Reinforcement Learning (RL) methods have emerged as a popular choice for training an efficient and effective dialogue policy.