Revisiting Topic-Guided Language Models

1 code implementation • 4 Dec 2023 • Carolina Zheng, Keyon Vafa, David M. Blei

A recent line of work in natural language processing has aimed to combine language models and topic models.

Language Modelling Topic Models

An Invariant Learning Characterization of Controlled Text Generation

1 code implementation • 31 May 2023 • Carolina Zheng, Claudia Shi, Keyon Vafa, Amir Feder, David M. Blei

In this paper, we show that the performance of controlled generation may be poor if the distributions of text in response to user prompts differ from the distribution the predictor was trained on.

Attribute Language Modelling +2

CAREER: A Foundation Model for Labor Sequence Data

1 code implementation • 16 Feb 2022 • Keyon Vafa, Emil Palikot, Tianyu Du, Ayush Kanodia, Susan Athey, David M. Blei

We fit CAREER to a dataset of 24 million job sequences from resumes, and adjust it on small longitudinal survey datasets.

Language Modelling Transfer Learning

Rationales for Sequential Predictions

2 code implementations • EMNLP 2021 • Keyon Vafa, Yuntian Deng, David M. Blei, Alexander M. Rush

Compared to existing baselines, greedy rationalization is best at optimizing the combinatorial objective and provides the most faithful rationales.

Combinatorial Optimization Language Modelling +2

Text-Based Ideal Points

1 code implementation • ACL 2020 • Keyon Vafa, Suresh Naidu, David M. Blei

In this paper, we introduce the text-based ideal point model (TBIP), an unsupervised probabilistic topic model that analyzes texts to quantify the political positions of its authors.

Discrete Flows: Invertible Generative Models of Discrete Data

2 code implementations • NeurIPS 2019 • Dustin Tran, Keyon Vafa, Kumar Krishna Agrawal, Laurent Dinh, Ben Poole

While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown.

Language Modelling