Search Results for author:
Found 13 papers, 11 papers with code
RESSA: Repair Sparse Vision-Language Models via Sparse Cross-Modality Adaptation
Moreover, while parameter-efficient LoRA finetuning has been proposed to repair the performance of sparse models, a significant challenge of weights merging arises due to the incompatibility of dense LoRA modules with sparse models that destroy the sparsity of pruned models.
Reformatted Alignment
This paper explores elevating the quality of existing instruction data to better align with human values, introducing a simple and effective approach named ReAlign, which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence.
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
Instruction tuning is critical to large language models (LLMs) for achieving better instruction following and task adaptation capabilities but its success heavily relies on the training data quality.
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
Data filtering for instruction tuning has proved important in improving both the efficiency and performance of the tuning process.
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation.
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts
Although a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters (e. g., one expert) for each input, its computation escalates significantly if increasing the number of activated experts, limiting its practical utility.
Accurate Prediction of Antibody Function and Structure Using Bio-Inspired Antibody Language Model
Remarkably, BALMFold outperforms those well-established methods like AlphaFold2, IgFold, ESMFold, and OmegaFold in the antibody benchmark, demonstrating significant potential to advance innovative engineering and streamline therapeutic antibody development by reducing the need for unnecessary trials.
MerA: Merging Pretrained Adapters For Few-Shot Learning
Adapter tuning, which updates only a few parameters, has become a mainstream method for fine-tuning pretrained language models to downstream tasks.
PAD-Net: An Efficient Framework for Dynamic Networks
The main contributions of our work are challenging the basic commonsense in dynamic networks and proposing a partially dynamic network, namely PAD-Net, to transform the redundant dynamic parameters into static ones.
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters
Adapter Tuning, which freezes the pretrained language models (PLMs) and only fine-tunes a few extra modules, becomes an appealing efficient alternative to the full model fine-tuning.
Vega-MT: The JD Explore Academy Translation System for WMT22
As for model sizes, we scale the Transformer-Big up to the extremely large model that owns nearly 4. 7 Billion parameters, to fully enhance the model capacity for our Vega-MT.
Ranked #1 on
Machine Translation
on WMT 2022 English-Russian
SD-Conv: Towards the Parameter-Efficiency of Dynamic Convolution
Dynamic convolution achieves better performance for efficient CNNs at the cost of negligible FLOPs increase.
Multi-modal Attention Network for Stock Movements Prediction
Traditionally, the prediction of future stock movements is based on the historical trading record.
