no code implementations • 29 Jun 2021 • Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum -- those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data.
no code implementations • Findings (EMNLP) 2021 • Jeremy R. Cole, Nanjiang Jiang, Panupong Pasupat, Luheng He, Peter Shaw
The dominant paradigm for semantic parsing in recent years is to formulate parsing as a sequence-to-sequence task, generating predictions with auto-regressive sequence decoders.
no code implementations • 25 Sep 2022 • Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, William W. Cohen
We introduce a new in-context learning paradigm to measure Large Language Models' (LLMs) ability to learn novel words during inference.
no code implementations • 26 Sep 2022 • Fangyu Liu, Julian Martin Eisenschlos, Jeremy R. Cole, Nigel Collier
Language models (LMs) trained on raw texts have no direct access to the physical world.
no code implementations • 1 Mar 2023 • Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J. Q. Zhang, Eunsol Choi, Bhuwan Dhingra
We propose representing factual changes between paired documents as question-answer pairs, where the answer to the same question differs between two versions.
no code implementations • 22 Mar 2023 • Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra, Partha Talukdar
First, we find that SSM alone improves the downstream performance on three temporal tasks by an avg.
no code implementations • 23 May 2023 • Livio Baldini Soares, Daniel Gillick, Jeremy R. Cole, Tom Kwiatkowski
Neural document rerankers are extremely effective in terms of accuracy.
no code implementations • 24 May 2023 • Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, Jacob Eisenstein
We investigate question answering from this perspective, focusing on answering a subset of questions with a high degree of accuracy, from a set of questions in which many are inherently ambiguous.
no code implementations • 29 Mar 2024 • Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Hernandez Abrego, Weiqiang Shi, Nithi Gupta, Aditya Kusupati, Prateek Jain, Siddhartha Reddy Jonnalagadda, Ming-Wei Chang, Iftekhar Naim
On the Massive Text Embedding Benchmark (MTEB), Gecko with 256 embedding dimensions outperforms all existing entries with 768 embedding size.