Search Results for author:
Found 16 papers, 6 papers with code
GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning
From this observation, we use mutual learning to improve BERT’s early exiting performances, that is, we ask each exit of a multi-exit BERT to distill knowledge from each other.
NLP-PINGAN-TECH @ CL-SciSumm 2020
We focus on systems for TASK1 (TASK 1A and TASK 1B) of CL-SciSumm Shared Task 2020 in this paper.
paht_nlp @ MEDIQA 2021: Multi-grained Query Focused Multi-Answer Summarization
First a RoBERTa model is first applied to give a local ranking of the candidate sentences.
ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue
In our online medical consultation scenario, a doctor responds to the texts and images provided by a patient in multiple rounds to diagnose her/his health condition, forming a multi-turn multimodal medical dialogue format.
IAPT: Instruction-Aware Prompt Tuning for Large Language Models
Thus, we propose to learn the idiosyncratic activation functions for prompt generators automatically with the help of rational functions.
Text2MDT: Extracting Medical Decision Trees from Medical Texts
In this work, we propose a novel task, Text2MDT, to explore the automatic extraction of MDTs from medical texts such as medical guidelines and textbooks.
UltraFeedback: Boosting Language Models with Scaled AI Feedback
Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models, serving as a solid foundation for future feedback learning research.
Unified Demonstration Retriever for In-Context Learning
To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice.
Ranked #1 on
Semantic Similarity
on CHIP-STS
Discovering Better Model Architectures for Medical Query Understanding
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection.
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.
AutoTrans: Automating Transformer Design via Reinforced Architecture Search
Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, especially the need for a principled way of warm-up which has shown importance for stable training of a transformer, as well as whether the task at hand prefer to scale the attention product or not.
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
Ranked #17 on
Common Sense Reasoning
on ReCoRD
PANLP at MEDIQA 2019: Pre-trained Language Models, Transfer Learning and Knowledge Distillation
Transfer learning from the NLI task to the RQE task is also experimented, which proves to be useful in improving the results of fine-tuning MT-DNN large.
