Characterizing the Efficiency vs. Accuracy Trade-off for Long-Context NLP Models

1 code implementation • nlppower (ACL) 2022 • Phyllis Ang, Bhuwan Dhingra, Lisa Wu Wills

In this work, we perform a systematic study of this accuracy vs. efficiency trade-off on two widely used long-sequence models - Longformer-Encoder-Decoder (LED) and Big Bird - during fine-tuning and inference on four datasets from the SCROLLS benchmark.

Playing the Game of 2048 Question Answering