1 code implementation • 29 Sep 2023 • Kevin Clark, Paul Vicol, Kevin Swersky, David J Fleet
We present Direct Reward Fine-Tuning (DRaFT), a simple and effective method for fine-tuning diffusion models to maximize differentiable reward functions, such as scores from human preference models.
1 code implementation • 28 Sep 2023 • Priyank Jaini, Kevin Clark, Robert Geirhos
What is the best paradigm to recognize objects -- discriminative inference (fast but potentially prone to shortcut learning) or using a generative model (slow but potentially more robust)?
1 code implementation • 16 May 2023 • Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Nenad Tomasev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, Vivek Natarajan
Large language models (LLMs) have catalyzed significant progress in medical question answering; Med-PaLM was the first model to exceed a "passing" score in US Medical Licensing Examination (USMLE) style questions with a score of 67. 2% on the MedQA dataset.
no code implementations • 27 Mar 2023 • Kevin Clark, Priyank Jaini
The key idea is using a diffusion model's ability to denoise a noised image given a text description of a label as a proxy for that label's likelihood.
no code implementations • 5 Dec 2022 • Kevin Clark, Kelvin Guu, Ming-Wei Chang, Panupong Pasupat, Geoffrey Hinton, Mohammad Norouzi
Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens and substantially improves LM performance.
1 code implementation • EMNLP 2020 • Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning
We introduce Electric, an energy-based cloze model for representation learning over text.
18 code implementations • ICLR 2020 • Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning
Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample or not.
Ranked #7 on Question Answering on Quora Question Pairs
1 code implementation • ACL 2019 • Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal, Christopher D. Manning, Quoc V. Le
It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.
2 code implementations • WS 2019 • Kevin Clark, Urvashi Khandelwal, Omer Levy, Christopher D. Manning
Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to learn from unlabeled data.
2 code implementations • 21 May 2019 • Urvashi Khandelwal, Kevin Clark, Dan Jurafsky, Lukasz Kaiser
Language model (LM) pre-training has resulted in impressive performance and sample efficiency on a variety of language understanding tasks.
Ranked #1 on Text Summarization on DUC 2004 Task 1 (ROUGE-2 metric)
2 code implementations • EMNLP 2018 • Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Quoc V. Le
We therefore propose Cross-View Training (CVT), a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data.
Ranked #3 on CCG Supertagging on CCGbank
no code implementations • ICLR 2018 • Kevin Clark, Thang Luong, Quoc V. Le
The students can learn from the teacher (the full model) because the teacher sees more of each example.
Ranked #4 on Chunking on CoNLL 2000 (using extra training data)
1 code implementation • EMNLP 2016 • Kevin Clark, Christopher D. Manning
Coreference resolution systems are typically trained with heuristic loss functions that require careful tuning.
Ranked #24 on Coreference Resolution on OntoNotes
1 code implementation • EMNLP 2016 • William L. Hamilton, Kevin Clark, Jure Leskovec, Dan Jurafsky
A word's sentiment depends on the domain in which it is used.
1 code implementation • ACL 2016 • Kevin Clark, Christopher D. Manning
A long-standing challenge in coreference resolution has been the incorporation of entity-level information - features defined over clusters of mentions instead of mention pairs.
Ranked #25 on Coreference Resolution on OntoNotes
no code implementations • TACL 2016 • Tim Althoff, Kevin Clark, Jure Leskovec
Mental illness is one of the most pressing public health issues of our time.