no code implementations • 20 Dec 2022 • Kundan Krishna, Yao Zhao, Jie Ren, Balaji Lakshminarayanan, Jiaming Luo, Mohammad Saleh, Peter J. Liu
We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes.
no code implementations • 30 Sep 2022 • Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, Peter J. Liu
Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences.
Ranked #1 on Text Summarization on X-Sum
abstractive question answering Abstractive Text Summarization +4
no code implementations • 30 Sep 2022 • Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, Peter J. Liu
Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output.
Abstractive Text Summarization Out-of-Distribution Detection +1
1 code implementation • 8 Aug 2022 • Jason Phang, Yao Zhao, Peter J. Liu
While large pretrained Transformer models have proven highly capable at tackling natural language tasks, handling long sequence inputs continues to be a significant challenge.
Ranked #1 on Long-range modeling on SCROLLS (GovRep metric)
no code implementations • 1 Aug 2022 • Reinald Kim Amplayo, Peter J. Liu, Yao Zhao, Shashi Narayan
Specifically, We treat sentences as basic units of matching instead of tokens, and use a sentence matching function to soft-match candidate and reference sentences.
no code implementations • 18 Jun 2020 • Yao Zhao, Mohammad Saleh, Peter J. Liu
Most prior work in the sequence-to-sequence paradigm focused on datasets with input sequence lengths in the hundreds of tokens due to the computational constraints of common RNN and Transformer architectures.
17 code implementations • ICML 2020 • Jingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. Liu
Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summarization.
Ranked #1 on Abstractive Text Summarization on AESLC
39 code implementations • arXiv 2019 • Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
Ranked #1 on Sentiment Analysis on SST-2 Binary classification
2 code implementations • 2 Oct 2019 • Peter J. Liu, Yu-An Chung, Jie Ren
We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.
5 code implementations • NeurIPS 2019 • Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, Balaji Lakshminarayanan
We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics.
Out-of-Distribution Detection Out of Distribution (OOD) Detection
2 code implementations • 30 May 2019 • Ben Goodrich, Vinay Rao, Mohammad Saleh, Peter J. Liu
We propose a model-based metric to estimate the factual accuracy of generated text that is complementary to typical scoring schemes like ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BLEU (Bilingual Evaluation Understudy).
no code implementations • 28 May 2019 • Ethan Steinberg, Peter J. Liu
Massively multi-label prediction/classification problems arise in environments like health-care or biology where very precise predictions are useful.
no code implementations • ICLR 2019 • Ethan Steinberg, Peter J. Liu
Massively multi-label prediction/classification problems arise in environments like health-care or biology where it is useful to make very precise predictions.
3 code implementations • 12 Oct 2018 • Eric Chu, Peter J. Liu
Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review while not relying on any review-specific features.
no code implementations • 27 Sep 2018 • Eric Chu, Peter J. Liu
Our proposed model consists of an auto-encoder trained so that the mean of the representations of the input reviews decodes to a reasonable summary-review.
no code implementations • 8 Aug 2018 • Peter J. Liu
Clinicians spend a significant amount of time inputting free-form textual notes into Electronic Health Records (EHR) systems.
4 code implementations • ICLR 2018 • Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, Noam Shazeer
We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents.
no code implementations • 24 Jan 2018 • Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Peter J. Liu, Xiaobing Liu, Mimi Sun, Patrik Sundberg, Hector Yee, Kun Zhang, Gavin E. Duggan, Gerardo Flores, Michaela Hardt, Jamie Irvine, Quoc Le, Kurt Litsch, Jake Marcus, Alexander Mossin, Justin Tansuwan, De Wang, James Wexler, Jimbo Wilson, Dana Ludwig, Samuel L. Volchenboum, Katherine Chou, Michael Pearson, Srinivasan Madabushi, Nigam H. Shah, Atul J. Butte, Michael Howell, Claire Cui, Greg Corrado, Jeff Dean
Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality.
3 code implementations • ICLR 2018 • W. James Murdoch, Peter J. Liu, Bin Yu
On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction.
40 code implementations • ACL 2017 • Abigail See, Peter J. Liu, Christopher D. Manning
Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization (meaning they are not restricted to simply selecting and rearranging passages from the original text).
Ranked #12 on Extractive Text Summarization on CNN / Daily Mail
2 code implementations • ICML 2017 • Colin Raffel, Minh-Thang Luong, Peter J. Liu, Ron J. Weiss, Douglas Eck
Recurrent neural network models with an attention mechanism have proven to be extremely effective on a wide variety of sequence-to-sequence problems.
Ranked #20 on Speech Recognition on TIMIT
no code implementations • EMNLP 2017 • Prajit Ramachandran, Peter J. Liu, Quoc V. Le
We apply this method to challenging benchmarks in machine translation and abstractive summarization and find that it significantly improves the subsequent supervised models.