Search Results for author:
Found 18 papers, 6 papers with code
Fractured Chain-of-Thought Reasoning
Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation
This paper introduces Unilogit, a novel self-distillation method for machine unlearning in Large Language Models.
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning
As large language models (LLMs) scale, model compression is crucial for edge deployment and accessibility.
Reward-Guided Speculative Decoding for Efficient LLM Reasoning
We introduce Reward-Guided Speculative Decoding (RSD), a novel framework aimed at improving the efficiency of inference in large language models (LLMs).
3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability
One notable challenge involves the efficient deployment of LLMs equipped with multiple task- or user-specific adapters, particularly when different adapters are needed for distinct requests within the same batch.
IKUN for WMT24 General MT Task: LLMs Are here for Multilingual Machine Translation
The systems are based on a two-stage approach: first, continuous pre-training on monolingual data in 10 languages, followed by fine-tuning on high-quality parallel data for 11 language directions.
The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models
Following multiple instructions is a crucial ability for large language models (LLMs).
Is It a Free Lunch for Removing Outliers during Pretraining?
With the growing size of large language models, the role of quantization becomes increasingly significant.
ApiQ: Finetuning of 2-Bit Quantized Large Language Model
Memory-efficient finetuning of large language models (LLMs) has recently attracted huge attention with the increasing size of LLMs, primarily due to the constraints posed by GPU memory limitations and the effectiveness of these methods compared to full finetuning.
ITEm: Unsupervised Image-Text Embedding Learning for eCommerce
How to teach a model to learn embedding from different modalities without neglecting information from the less dominant modality is challenging.
Ask Language Model to Clean Your Noisy Translation Data
Lastly, experiments on C-MTNT showcased its effectiveness in evaluating the robustness of NMT models, highlighting the potential of advanced language models for data cleaning and emphasizing C-MTNT as a valuable resource.
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
One effective way to reduce the activation memory is to apply a reversible model, so the intermediate activations are not necessary to be cached and can be recomputed.
Parameter-Efficient Fine-Tuning without Introducing New Latency
Parameter-efficient fine-tuning (PEFT) of pre-trained language models has recently demonstrated remarkable achievements, effectively matching the performance of full fine-tuning while utilizing significantly fewer trainable parameters, and consequently addressing the storage and communication constraints.
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token
We show: (1) [MASK]s can indeed be appended at a later layer, being disentangled from the word embedding; (2) The gathering of contextualized information from unmasked tokens can be conducted with a few layers.
Towards prediction of turbulent flows at high Reynolds numbers using high performance computing data and deep learning
In this paper, deep learning (DL) methods are evaluated in the context of turbulent flows.
Back-translation for Large-Scale Multilingual Machine Translation
Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement.
Unifying Input and Output Smoothing in Neural Machine Translation
Soft contextualized data augmentation is a recent method that replaces one-hot representation of words with soft posterior distributions of an external language model, smoothing the input of neural machine translation systems.
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation
Mutual learning, where multiple agents learn collaboratively and teach one another, has been shown to be an effective way to distill knowledge for image classification tasks.
