Search Results for author: Muhammed Yusuf Kocyigit

Found 7 papers, 4 papers with code

Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation

no code implementations30 Jan 2025 Muhammed Yusuf Kocyigit, Eleftheria Briakou, Daniel Deutsch, Jiaming Luo, Colin Cherry, Markus Freitag

Data contamination -- the accidental consumption of evaluation examples within the pre-training data -- can undermine the validity of evaluation benchmarks.

Machine Translation

Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?

no code implementations6 Nov 2024 Aaditya K. Singh, Muhammed Yusuf Kocyigit, Andrew Poulton, David Esiobu, Maria Lomeli, Gergely Szilvasy, Dieuwke Hupkes

We propose a novel analysis method called ConTAM, and show with a large scale survey of existing and novel n-gram based contamination metrics across 13 benchmarks and 7 models from 2 different families that ConTAM can be used to better understand evaluation data contamination and its effects.

Specificity

On Measuring Social Biases in Prompt-Based Multi-Task Learning

1 code implementation Findings (NAACL) 2022 Afra Feyza Akyürek, Sejin Paik, Muhammed Yusuf Kocyigit, Seda Akbiyik, Şerife Leman Runyun, Derry Wijaya

Large language models trained on a mixture of NLP tasks that are converted into a text-to-text format using prompts, can generalize into novel forms of language and handle novel tasks.

Language Modeling Language Modelling +4

Better Quality Estimation for Low Resource Corpus Mining

no code implementations Findings (ACL) 2022 Muhammed Yusuf Kocyigit, Jiho Lee, Derry Wijaya

We show that State-of-the-art QE models, when tested in a Parallel Corpus Mining (PCM) setting, perform unexpectedly bad due to a lack of robustness to out-of-domain examples.

Contrastive Learning Data Augmentation +3

Cannot find the paper you are looking for? You can Submit a new open access paper.