SentenceMIM: A Latent Variable Language Model

18 Feb 2020 Micha Livne Kevin Swersky David J. Fleet

SentenceMIM is a probabilistic auto-encoder for language data, trained with Mutual Information Machine (MIM) learning to provide a fixed length representation of variable length language observations (i.e., similar to VAE). Previous attempts to learn VAEs for language data faced challenges due to posterior collapse... (read more)

PDF Abstract

Results from the Paper


 Ranked #1 on Question Answering on YahooCQA (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Question Answering YahooCQA sMIM (1024) + P@1 0.757 # 1
MRR 0.863 # 1
Question Answering YahooCQA sMIM (1024) P@1 0.683 # 2
MRR 0.818 # 2

Methods used in the Paper


METHOD TYPE
MIM
Representation Learning
VAE
Generative Models
AE
Dimensionality Reduction