Local Prior Matching

Local Prior Matching is a semi-supervised objective for speech recognition that distills knowledge from a strong prior (e.g. a language model) to provide learning signal to a discriminative model trained on unlabeled speech. The LPM objective minimizes the cross entropy between the local prior and the model distribution, and is minimized when $q_{y\mid{x}} = \bar{p}_{y\mid{x}}$. Intuitively, LPM encourages the ASR model to assign posterior probabilities proportional to the linguistic probabilities of the proposed hypotheses.