Search Results for author:
Found 7 papers, 5 papers with code
Harder Tasks Need More Experts: Dynamic Routing in MoE Models
In this paper, we introduce a novel dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adjusting the number of activated experts based on input difficulty.
Probing Multimodal Large Language Models for Global and Local Semantic Representations
The advancement of Multimodal Large Language Models (MLLMs) has greatly accelerated the development of applications in understanding integrated texts and images.
Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering
Open-ended question answering requires models to find appropriate evidence to form well-reasoned, comprehensive and helpful answers.
MC^2: A Multilingual Corpus of Minority Languages in China
However, existing LLMs exhibit limited abilities in understanding low-resource languages, including the minority languages in China, due to a lack of training data.
Lawyer LLaMA Technical Report
Specifically, we inject domain knowledge during the continual training stage and teach the model to learn professional skills using properly designed supervised fine-tuning tasks.
A Frustratingly Easy Improvement for Position Embeddings via Random Padding
Since the embeddings of rear positions are updated fewer times than the front position embeddings, the rear ones may not be properly trained.
Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study
Large pre-trained language models help to achieve state of the art on a variety of natural language processing (NLP) tasks, nevertheless, they still suffer from forgetting when incrementally learning a sequence of tasks.
