Peak-First CTC: Reducing the Peak Latency of CTC Models by Applying Peak-First Regularization

no code implementations • 7 Nov 2022 • Zhengkun Tian, Hongyu Xiang, Min Li, Feifei Lin, Ke Ding, Guanglu Wan

To reduce the peak latency, we propose a simple and novel method named peak-first regularization, which utilizes a frame-wise knowledge distillation function to force the probability distribution of the CTC model to shift left along the time axis instead of directly modifying the calculation process of CTC loss and gradients.

Knowledge Distillation

SubCharacter Chinese-English Neural Machine Translation with Wubi encoding

no code implementations • 7 Nov 2019 • Wei Zhang, Feifei Lin, Xiaodong Wang, Zhenshuang Liang, Zhen Huang

However, when the translation task involves Chinese, semantic granularity remains at the word and character level, so there is still need more fine-grained translation model of Chinese.

Machine Translation Model Compression +2