Search Results for author:
Found 5 papers, 2 papers with code
Understanding Parameter Sharing in Transformers
Inspired by this, we tune the training hyperparameters related to model convergence in a targeted manner.
MobileNMT: Enabling Translation in 15MB and 30ms
With the co-design of model and engine, compared with the existing system, we speed up 47. 0x and save 99. 5% of memory with only 11. 6% loss of BLEU.
PARAGEN : A Parallel Generation Toolkit
PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation.
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation
Albeit being a prevalent architecture searching approach, differentiable architecture search (DARTS) is largely hindered by its substantial memory cost since the entire supernet resides in the memory.
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
Ranked #17 on
Common Sense Reasoning
on ReCoRD
