Search Results for author: Pedro Rodriguez

Found 16 papers, 6 papers with code

Evaluation Paradigms in Question Answering

no code implementations EMNLP 2021 Pedro Rodriguez, Jordan Boyd-Graber

Question answering (QA) primarily descends from two branches of research: (1) Alan Turing’s investigation of machine intelligence at Manchester University and (2) Cyril Cleverdon’s comparison of library card catalog indices at Cranfield University.

Position Question Answering

Clustering Examples in Multi-Dataset Benchmarks with Item Response Theory

no code implementations insights (ACL) 2022 Pedro Rodriguez, Phu Mon Htut, John Lalor, João Sedoc

In natural language processing, multi-dataset benchmarks for common tasks (e. g., SuperGLUE for natural language inference and MRQA for question answering) have risen in importance.

Clustering Natural Language Inference +1

PowerFlowMultiNet: Multigraph Neural Networks for Unbalanced Three-Phase Distribution Systems

no code implementations1 Mar 2024 Salah Ghamizi, Jun Cao, Aoxiang Ma, Pedro Rodriguez

PowerFlowMultiNet outperforms traditional methods and other deep learning approaches in terms of accuracy and computational speed.

Graph Embedding

MultiContrievers: Analysis of Dense Retrieval Representations

1 code implementation24 Feb 2024 Seraphina Goldfarb-Tarrant, Pedro Rodriguez, Jane Dwivedi-Yu, Patrick Lewis

Dense retrievers compress source documents into (possibly lossy) vector representations, yet there is little analysis of what information is lost versus preserved, and how it affects downstream tasks.

Retrieval

Instruction-tuned Language Models are Better Knowledge Learners

no code implementations20 Feb 2024 Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Victoria Lin, Wen-tau Yih, Srinivasan Iyer

The standard recipe for doing so involves continued pre-training on new documents followed by instruction-tuning on question-answer (QA) pairs.

Language Modelling Large Language Model

RA-DIT: Retrieval-Augmented Dual Instruction Tuning

no code implementations2 Oct 2023 Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Rich James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, Scott Yih

Retrieval-augmented language models (RALMs) improve performance by accessing long-tail and up-to-date knowledge from external data stores, but are challenging to build.

Few-Shot Learning Open-Domain Question Answering +1

Reimagining Retrieval Augmented Language Models for Answering Queries

no code implementations1 Jun 2023 Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Scott Yih

We present a reality check on large language models and inspect the promise of retrieval augmented language models in comparison.

Question Answering Retrieval

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks

1 code implementation ACL 2022 Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela

We introduce Dynatask: an open source system for setting up custom NLP tasks that aims to greatly lower the technical knowledge and effort required for hosting and evaluating state-of-the-art NLP models, as well as for conducting model in the loop data collection with crowdworkers.

Benchmarking

py-irt: A Scalable Item Response Theory Library for Python

1 code implementation2 Mar 2022 John P. Lalor, Pedro Rodriguez

py-irt is a Python library for fitting Bayesian Item Response Theory (IRT) models.

Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?

1 code implementation ACL 2021 Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, Jordan Boyd-Graber

While leaderboards are a straightforward ranking of NLP models, this simplicity can mask nuances in evaluation items (examples) and subjects (NLP models).

Information Seeking in the Spirit of Learning: a Dataset for Conversational Curiosity

1 code implementation EMNLP 2020 Pedro Rodriguez, Paul Crook, Seungwhan Moon, Zhiguang Wang

Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard-of-Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts related to what they know.

Mitigating Noisy Inputs for Question Answering

no code implementations8 Aug 2019 Denis Peskov, Joe Barrow, Pedro Rodriguez, Graham Neubig, Jordan Boyd-Graber

We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +6

Quizbowl: The Case for Incremental Question Answering

no code implementations9 Apr 2019 Pedro Rodriguez, Shi Feng, Mohit Iyyer, He He, Jordan Boyd-Graber

Throughout this paper, we show that collaborations with the vibrant trivia community have contributed to the quality of our dataset, spawned new research directions, and doubled as an exciting way to engage the public with research in machine learning and natural language processing.

BIG-bench Machine Learning Decision Making +1

Pathologies of Neural Models Make Interpretations Difficult

no code implementations EMNLP 2018 Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, Jordan Boyd-Graber

In existing interpretation methods for NLP, a word's importance is determined by either input perturbation---measuring the decrease in model confidence when that word is removed---or by the gradient with respect to that word.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.