Search Results for author:
Found 13 papers, 5 papers with code
NLP-PINGAN-TECH @ CL-SciSumm 2020
We focus on systems for TASK1 (TASK 1A and TASK 1B) of CL-SciSumm Shared Task 2020 in this paper.
paht_nlp @ MEDIQA 2021: Multi-grained Query Focused Multi-Answer Summarization
First a RoBERTa model is first applied to give a local ranking of the candidate sentences.
GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning
From this observation, we use mutual learning to improve BERT’s early exiting performances, that is, we ask each exit of a multi-exit BERT to distill knowledge from each other.
UltraFeedback: Boosting Language Models with High-quality Feedback
However, the scarcity of diverse, naturalistic datasets of human preferences on LLM outputs at scale poses a great challenge to RLHF as well as feedback learning research within the open-source community.
Unified Demonstration Retriever for In-Context Learning
To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice.
Ranked #1 on
Medical Relation Extraction
on CMeIE
Discovering Better Model Architectures for Medical Query Understanding
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection.
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.
AutoTrans: Automating Transformer Design via Reinforced Architecture Search
Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, especially the need for a principled way of warm-up which has shown importance for stable training of a transformer, as well as whether the task at hand prefer to scale the attention product or not.
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
PANLP at MEDIQA 2019: Pre-trained Language Models, Transfer Learning and Knowledge Distillation
Transfer learning from the NLI task to the RQE task is also experimented, which proves to be useful in improving the results of fine-tuning MT-DNN large.
