Search Results for author: Howard Yen

Found 6 papers, 5 papers with code

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly

1 code implementation3 Oct 2024 Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, Danqi Chen

There have been many benchmarks for evaluating long-context language models (LCLMs), but developers often rely on synthetic tasks like needle-in-a-haystack (NIAH) or arbitrary subsets of tasks.

RAG

How to Train Long-Context Language Models (Effectively)

1 code implementation3 Oct 2024 Tianyu Gao, Alexander Wettig, Howard Yen, Danqi Chen

We study continued training and supervised fine-tuning (SFT) of a language model (LM) to make effective use of long-context information.

BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval

no code implementations16 Jul 2024 Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S. Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O. Arik, Danqi Chen, Tao Yu

To better benchmark retrieval on such challenging queries, we introduce BRIGHT, the first text retrieval benchmark that requires intensive reasoning to retrieve relevant documents.

Question Answering Text Retrieval

Long-Context Language Modeling with Parallel Context Encoding

1 code implementation26 Feb 2024 Howard Yen, Tianyu Gao, Danqi Chen

However, the substantial computational cost of transformers and limited generalization of positional encoding restrict the size of their context window.

In-Context Learning Instruction Following +1

Cannot find the paper you are looking for? You can Submit a new open access paper.