Search Results for author:
Found 7 papers, 5 papers with code
Tunable Soft Prompts are Messengers in Federated Learning
As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL.
Counterfactual Debiasing for Generating Factually Consistent Text Summaries
Despite substantial progress in abstractive text summarization to generate fluent and informative texts, the factual inconsistency in the generated summaries remains an important yet challenging problem to be solved.
Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learning
In this study, we further broaden the application scope of FL in NLP by proposing an Assign-Then-Contrast (denoted as ATC) framework, which enables clients with heterogeneous NLP tasks to construct an FL course and learn useful knowledge from each other.
A Benchmark for Federated Hetero-Task Learning
To investigate the heterogeneity in federated learning in real-world scenarios, we generalize the classic federated learning to federated hetero-task learning, which emphasizes the inconsistency across the participants in federated learning in terms of both data distribution and learning tasks.
A Survey of Natural Language Generation
This paper offers a comprehensive review of the research on Natural Language Generation (NLG) over the past two decades, especially in relation to data-to-text generation and text-to-text generation deep learning methods, as well as new applications of NLG technology.
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression
In this paper, we target to compress PLMs with knowledge distillation, and propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation
In this paper, we have a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA) since the computational cost of FFN is 2$\sim$3 times larger than MHA.
