Search Results for author: Emine Yilmaz

Found 54 papers, 26 papers with code

Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation

1 code implementation Findings (EMNLP) 2021 Jarana Manotumruksa, Jeff Dalton, Edgar Meij, Emine Yilmaz

While state-of-the-art Dialogue State Tracking (DST) models show promising results, all of them rely on a traditional cross-entropy loss function during the training process, which may not be optimal for improving the joint goal accuracy.

Data Augmentation Dialogue State Tracking

Clarifying the Path to User Satisfaction: An Investigation into Clarification Usefulness

no code implementations2 Feb 2024 Hossein A. Rahmani, Xi Wang, Mohammad Aliannejadi, Mohammadmehdi Naghiaei, Emine Yilmaz

Clarifying questions are an integral component of modern information retrieval systems, directly impacting user satisfaction and overall system performance.

Information Retrieval

Benchmarking LLMs via Uncertainty Quantification

1 code implementation23 Jan 2024 Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek F. Wong, Emine Yilmaz, Shuming Shi, Zhaopeng Tu

The proliferation of open-source Large Language Models (LLMs) from various institutions has highlighted the urgent need for comprehensive evaluation methods.

Benchmarking Uncertainty Quantification

A Toolbox for Modelling Engagement with Educational Videos

no code implementations30 Dec 2023 Yuxiang Qiu, Karim Djemili, Denis Elezi, Aaneel Shalman, María Pérez-Ortiz, Emine Yilmaz, John Shawe-Taylor, Sahan Bulathwela

With the advancement and utility of Artificial Intelligence (AI), personalising education to a global population could be a cornerstone of new educational systems in the future.

Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation

1 code implementation25 Oct 2023 Xi Wang, Hossein A. Rahmani, Jiqun Liu, Emine Yilmaz

Conversational Recommendation System (CRS) is a rapidly growing research area that has gained significant attention alongside advancements in language modelling techniques.

Data Augmentation Language Modelling +2

Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting

1 code implementation15 Oct 2023 Fanghua Ye, Meng Fang, Shenghui Li, Emine Yilmaz

Furthermore, we propose distilling the rewriting capabilities of LLMs into smaller models to reduce rewriting latency.

Conversational Search Language Modelling +2

Schema-Guided User Satisfaction Modeling for Task-Oriented Dialogues

1 code implementation26 May 2023 Yue Feng, Yunlong Jiao, Animesh Prasad, Nikolaos Aletras, Emine Yilmaz, Gabriella Kazai

Further, it employs a fulfillment representation layer for learning how many task attributes have been fulfilled in the dialogue, an importance predictor component for calculating the importance of task attributes.

Attribute Language Modelling +1

A Survey on Asking Clarification Questions Datasets in Conversational Systems

1 code implementation25 May 2023 Hossein A. Rahmani, Xi Wang, Yue Feng, Qiang Zhang, Emine Yilmaz, Aldo Lipani

The ability to understand a user's underlying needs is critical for conversational systems, especially with limited input from users in a conversation.

Rethinking Semi-supervised Learning with Language Models

2 code implementations22 May 2023 Zhengxiang Shi, Francesco Tonolini, Nikolaos Aletras, Emine Yilmaz, Gabriella Kazai, Yunlong Jiao

Semi-supervised learning (SSL) is a popular setting aiming to effectively utilize unlabelled data to improve model performance in downstream natural language processing (NLP) tasks.

Pseudo Label Semi-Supervised Text Classification +1

Modeling User Satisfaction Dynamics in Dialogue via Hawkes Process

1 code implementation21 May 2023 Fanghua Ye, Zhiyuan Hu, Emine Yilmaz

It assumes that the performance of a dialogue system can be measured by user satisfaction and uses an estimator to simulate users.

Scalable Educational Question Generation with Pre-trained Language Models

1 code implementation13 May 2023 Sahan Bulathwela, Hamze Muse, Emine Yilmaz

We develop \textit{EduQG}, a novel educational question generation model built by adapting a large language model.

Language Modelling Large Language Model +2

Query-specific Variable Depth Pooling via Query Performance Prediction towards Reducing Relevance Assessment Effort

no code implementations23 Apr 2023 Debasis Ganguly, Emine Yilmaz

However, in this paper we argue that the annotation effort can be substantially reduced if the depth of the pool is made a variable quantity for each query, the rationale being that the number of documents relevant to the information need can widely vary across queries.


Task2KB: A Public Task-Oriented Knowledge Base

no code implementations24 Jan 2023 Procheta Sen, Xi Wang, Ruiqing Xu, Emine Yilmaz

Search engines and conversational assistants are commonly used to help users complete their every day tasks such as booking travel, cooking, etc.

Knowledge Graphs

Pre-Training With Scientific Text Improves Educational Question Generation

no code implementations7 Dec 2022 Hamze Muse, Sahan Bulathwela, Emine Yilmaz

With the boom of digital educational materials and scalable e-learning systems, the potential for realising AI-assisted personalised learning has skyrocketed.

Language Modelling Large Language Model +2

Just Mix Once: Worst-group Generalization by Group Interpolation

no code implementations21 Oct 2022 Giorgio Giannone, Serhii Havrylov, Jordan Massiah, Emine Yilmaz, Yunlong Jiao

Advances in deep learning theory have revealed how average generalization relies on superficial patterns in data.

Learning Theory

Evaluation Metrics for Measuring Bias in Search Engine Results

no code implementations19 Oct 2022 Gizem Gezici, Aldo Lipani, Yucel Saygin, Emine Yilmaz

However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point.

Can Population-based Engagement Improve Personalisation? A Novel Dataset and Experiments

no code implementations22 Jun 2022 Sahan Bulathwela, Meghana Verma, Maria Perez-Ortiz, Emine Yilmaz, John Shawe-Taylor

This work explores how population-based engagement prediction can address cold-start at scale in large learning resource collections.


Integrated Weak Learning

no code implementations19 Jun 2022 Peter Hayes, Mingtian Zhang, Raza Habib, Jordan Burgess, Emine Yilmaz, David Barber

We introduce a label model that can learn to aggregate weak supervision sources differently for different datapoints and takes into consideration the performance of the end-model during training.

ViralBERT: A User Focused BERT-Based Approach to Virality Prediction

1 code implementation17 May 2022 Rikaz Rameez, Hossein A. Rahmani, Emine Yilmaz

We collect a dataset of 330k tweets to train ViralBERT and validate the efficacy of our model using baselines from current studies in this field.

Sentiment Analysis

Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking

no code implementations ACL 2022 Yue Feng, Aldo Lipani, Fanghua Ye, Qiang Zhang, Emine Yilmaz

Existing approaches that have considered such relations generally fall short in: (1) fusing prior slot-domain membership relations and dialogue-aware dynamic slot relations explicitly, and (2) generalizing to unseen domains.

Dialogue State Tracking Multi-domain Dialogue State Tracking +1

ASSIST: Towards Label Noise-Robust Dialogue State Tracking

1 code implementation Findings (ACL) 2022 Fanghua Ye, Yue Feng, Emine Yilmaz

In this paper, instead of improving the annotation quality further, we propose a general framework, named ASSIST (lAbel noiSe-robuSt dIalogue State Tracking), to train DST models robustly from noisy labels.

Dialogue State Tracking

Semantic TrueLearn: Using Semantic Knowledge Graphs in Recommendation Systems

no code implementations8 Dec 2021 Sahan Bulathwela, María Pérez-Ortiz, Emine Yilmaz, John Shawe-Taylor

In informational recommenders, many challenges arise from the need to handle the semantic and hierarchical structure between knowledge areas.

Knowledge Graphs Recommendation Systems

Towards More Accountable Search Engines: Online Evaluation of Representation Bias

no code implementations17 Oct 2021 Aldo Lipani, Florina Piroi, Emine Yilmaz

Information availability affects people's behavior and perception of the world.

Sample Efficient Model Evaluation

no code implementations24 Sep 2021 Emine Yilmaz, Peter Hayes, Raza Habib, Jordan Burgess, David Barber

Labelling data is a major practical bottleneck in training and testing classifiers.

PEEK: A Large Dataset of Learner Engagement with Educational Videos

no code implementations3 Sep 2021 Sahan Bulathwela, Maria Perez-Ortiz, Erik Novak, Emine Yilmaz, John Shawe-Taylor

One of the main challenges in advancing this research direction is the scarcity of large, publicly available datasets.

Recommendation Systems

MS MARCO: Benchmarking Ranking Models in the Large-Data Regime

no code implementations9 May 2021 Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin

Evaluation efforts such as TREC, CLEF, NTCIR and FIRE, alongside public leaderboard such as MS MARCO, are intended to encourage research and track our progress, addressing big questions in our field.


TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime

no code implementations19 Apr 2021 Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff

The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available.

Selection bias Test

Overview of the TREC 2020 deep learning track

1 code implementation15 Feb 2021 Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos

This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime.

Passage Retrieval Retrieval

Slot Self-Attentive Dialogue State Tracking

1 code implementation22 Jan 2021 Fanghua Ye, Jarana Manotumruksa, Qiang Zhang, Shenghui Li, Emine Yilmaz

Then a stacked slot self-attention is applied on these features to learn the correlations among slots.

Dialogue State Tracking Task-Oriented Dialogue Systems

VLEngagement: A Dataset of Scientific Video Lectures for Evaluating Population-based Engagement

1 code implementation2 Nov 2020 Sahan Bulathwela, Maria Perez-Ortiz, Emine Yilmaz, John Shawe-Taylor

This paper introduces VLEngagement, a novel dataset that consists of content-based and video-specific features extracted from publicly available scientific video lectures and several metrics related to user engagement.

ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

no code implementations9 Jun 2020 Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo Billerbeck

Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval.

Information Retrieval Retrieval

Predicting Engagement in Video Lectures

1 code implementation31 May 2020 Sahan Bulathwela, María Pérez-Ortiz, Aldo Lipani, Emine Yilmaz, John Shawe-Taylor

The explosion of Open Educational Resources (OERs) in the recent years creates the demand for scalable, automatic approaches to process and evaluate OERs, with the end goal of identifying and recommending the most suitable educational materials for learners.

Recommendation Systems

On the Reliability of Test Collections for Evaluating Systems of Different Types

no code implementations28 Apr 2020 Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos

As deep learning based models are increasingly being used for information retrieval (IR), a major challenge is to ensure the availability of test collections for measuring their quality.

Fairness Information Retrieval +3

Overview of the TREC 2019 deep learning track

2 code implementations17 Mar 2020 Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees

The Deep Learning Track is a new track for TREC 2019, with the goal of studying ad hoc ranking in a large data regime.

Passage Retrieval Retrieval +2

Towards an Integrative Educational Recommender for Lifelong Learners

1 code implementation3 Dec 2019 Sahan Bulathwela, Maria Perez-Ortiz, Emine Yilmaz, John Shawe-Taylor

One of the most ambitious use cases of computer-assisted learning is to build a recommendation system for lifelong learning.

TrueLearn: A Family of Bayesian Algorithms to Match Lifelong Learners to Open Educational Resources

1 code implementation21 Nov 2019 Sahan Bulathwela, Maria Perez-Ortiz, Emine Yilmaz, John Shawe-Taylor

The recent advances in computer-assisted learning systems and the availability of open educational resources today promise a pathway to providing cost-efficient, high-quality education to large masses of learners.

Knowledge Tracing Test

Disentangling Interpretable Generative Parameters of Random and Real-World Graphs

1 code implementation12 Oct 2019 Niklas Stoehr, Emine Yilmaz, Marc Brockschmidt, Jan Stuehmer

While a wide range of interpretable generative procedures for graphs exist, matching observed graph topologies with such procedures and choices for its parameters remains an open problem.

Disentanglement Graph Embedding +2

Self-Attentive Hawkes Processes

1 code implementation17 Jul 2019 Qiang Zhang, Aldo Lipani, Omer Kirnap, Emine Yilmaz

The proposed method adapts self-attention to fit the intensity function of Hawkes processes.

Variational Self-attention Model for Sentence Representation

no code implementations30 Dec 2018 Qiang Zhang, Shangsong Liang, Emine Yilmaz

This paper proposes a variational self-attention model (VSAM) that employs variational inference to derive self-attention.

Sentence Stance Detection +1

Extracting Hierarchies of Search Tasks & Subtasks via a Bayesian Nonparametric Approach

no code implementations6 Jun 2017 Rishabh Mehrotra, Emine Yilmaz

As a result, significant amount of research has been devoted to extracting proper representations of tasks in order to enable search systems to help users complete their tasks, as well as providing the end user with better query suggestions, for better recommendations, for satisfaction prediction, and for improved personalization in terms of tasks.

Cannot find the paper you are looking for? You can Submit a new open access paper.