Search Results for author:
Found 19 papers, 9 papers with code
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
However, even the top-performing 13B LLM-based translation models, like ALMA, does not match the performance of state-of-the-art conventional encoder-decoder translation models or larger-scale LLMs such as GPT-4.
PEMA: An Offsite-Tunable Plug-in External Memory Adaptation for Language Models
PEMA integrates with context representations from test data during inference to perform downstream tasks.
Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness
In our comprehensive analysis, we show that MoE models with 2-bit expert weights can deliver better model performance than the dense model trained on the same dataset.
A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models
In this study, we propose a novel fine-tuning approach for LLMs that is specifically designed for the translation task, eliminating the need for the abundant parallel data that traditional translation models usually depend on.
Ranked #5 on
Machine Translation
on FLoRes-200
Task-Based MoE for Multitask Multilingual Machine Translation
Mixture-of-experts (MoE) architecture has been proven a powerful method for diverse tasks in training deep models in many applications.
FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs
Large Language Models (LLMs) have achieved state-of-the-art performance across various language tasks but pose challenges for practical deployment due to their substantial memory requirements.
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
In this paper, we present a comprehensive evaluation of GPT models for machine translation, covering various aspects such as quality of different GPT models in comparison with state-of-the-art research and commercial systems, effect of prompting strategies, robustness towards domain shifts and document-level translation.
Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Production
Mixture of Experts (MoE) models with conditional execution of sparsely activated layers have enabled training models with a much larger number of parameters.
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Furthermore, existing MoE works do not consider computational constraints (e. g., FLOPs, latency) to guide their design.
Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPU
The proposed method speeds up the vocab projection step itself by up to 2. 6x.
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Sparsely activated transformers, such as Mixture of Experts (MoE), have received great interest due to their outrageous scaling capability which enables dramatical increases in model size without significant increases in computational cost.
Taming Sparsely Activated Transformer with Stochastic Experts
While most on-going research focuses on improving SAMs models by exploring methods of routing inputs to experts, our analysis reveals that such research might not lead to the solution we expect, i. e., the commonly-used routing methods based on gating mechanisms do not work better than randomly routing inputs to experts.
Scalable and Efficient MoE Training for Multitask Multilingual Models
By combining the efficient system and training methods, we are able to significantly scale up large multitask multilingual models for language generation which results in a great improvement in model accuracy.
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding
In this paper, we present FastFormers, a set of recipes to achieve efficient inference-time performance for Transformer-based models on various NLU tasks.
From Research to Production and Back: Ludicrously Fast Neural Machine Translation
Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models.
Automated Quality Control in Image Segmentation: Application to the UK Biobank Cardiac MR Imaging Study
Methods: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis.
Real-time Prediction of Segmentation Quality
Recent advances in deep learning based image segmentation methods have enabled real-time performance with human-level accuracy.
Automated cardiovascular magnetic resonance image analysis with fully convolutional networks
By combining FCN with a large-scale annotated dataset, the proposed automated method achieves a high performance on par with human experts in segmenting the LV and RV on short-axis CMR images and the left atrium (LA) and right atrium (RA) on long-axis CMR images.
