Search Results for author: Alexandra Vtyurina

Found 2 papers, 2 papers with code

Shallow pooling for sparse labels

1 code implementation31 Aug 2021 Negar Arabzadeh, Alexandra Vtyurina, Xinyi Yan, Charles L. A. Clarke

To test this observation, we employed crowdsourced workers to make preference judgments between the top item returned by a modern neural ranking stack and a judged relevant item.

Passage Ranking

Assessing top-$k$ preferences

2 code implementations22 Jul 2020 Charles L. A. Clarke, Alexandra Vtyurina, Mark D. Smucker

To measure the performance of a ranker, we compare its output to this preferred ordering by applying a rank similarity measure. We demonstrate the practical feasibility of this approach by crowdsourcing partial preferences for the TREC 2019 Conversational Assistance Track, replacing NDCG with a new measure named "compatibility".

Cannot find the paper you are looking for? You can Submit a new open access paper.