Search Results for author:
Found 47 papers, 34 papers with code
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis
Then, we conduct extensive experiments with the baseline within each class, covering models with various sizes (7B-70B), bitwidths, training levels (LLaMA1/2/3/3. 1), architectures (Mixtral, DeepSeekMoE and Mamba) and modality (LLaVA1. 5 and VILA1. 5) on a wide range of evaluation metrics. Through comparative analysis on the results, we summarize the superior of each PTQ strategy and modelsize-bitwidth trade-off considering the performance.
DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs
Efficient KV cache management in LLMs is crucial for long-context tasks like RAG and summarization.
Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering
Multi-hop question answering (MHQA) poses a significant challenge for large language models (LLMs) due to the extensive knowledge demands involved.
DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimization
To address this, we propose DRPruning, which incorporates distributionally robust optimization to restore balanced performance across domains, along with further improvements to enhance robustness.
NewTerm: Benchmarking Real-Time New Terms for Large Language Models with Annual Updates
Despite their remarkable abilities in various tasks, large language models (LLMs) still struggle with real-time information (e. g., new facts and terms) due to the knowledge cutoff in their development process.
DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory
Large language models (LLMs) have achieved reasonable quality improvements in machine translation (MT).
LPZero: Language Model Zero-cost Proxy Search from Zero
Specifically, we model the ZC proxy as a symbolic equation and incorporate a unified proxy search space that encompasses existing ZC proxies, which are composed of a predefined set of mathematical symbols.
Self-Powered LLM Modality Expansion for Large Speech-Text Models
This study aims to refine the use of speech datasets for LSM training by addressing the limitations of vanilla instruction tuning.
CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions
With instruction tuning, Large Language Models (LLMs) can enhance their ability to adhere to commands.
Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them.
TasTe: Teaching Large Language Models to Translate through Self-Reflection
The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore
The efficacy of an large language model (LLM) generated text detector depends substantially on the availability of sizable training data.
3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset
Experimental results show that MMT models trained on our dataset exhibit a greater ability to exploit visual information than those trained on other MMT datasets.
Improving Attributed Text Generation of Large Language Models via Preference Learning
Large language models have been widely adopted in natural language processing, yet they face the challenge of generating unreliable content.
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
In this work, we propose a novel approach, termed SelectIT, that capitalizes on the foundational capabilities of the LLM itself.
DB-LLM: Accurate Dual-Binarization for Efficient LLMs
Large language models (LLMs) have significantly advanced the field of natural language processing, while the expensive memory and computation consumption impede their practical deployment.
Revisiting Demonstration Selection Strategies in In-Context Learning
In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent.
Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrix
In multilingual translation research, the comprehension and utilization of language families are of paramount importance.
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
For video generation, various open-sourced models and public-available services have been developed to generate high-quality videos.
Holistic Exploration on Universal Decompositional Semantic Parsing: Architecture, Data Augmentation, and LLM Paradigm
In this paper, we conduct a holistic exploration of the Universal Decompositional Semantic (UDS) Parsing.
Pluggable Neural Machine Translation Models via Memory-augmented Adapters
Although neural machine translation (NMT) models perform well in the general domain, it remains rather challenging to control their generation behavior to satisfy the requirement of different users.
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers.
Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization
Moreover, the knowledge memorized by PLMs may quickly become outdated, which affects the generalization performance of PLMs on future data.
Towards Making the Most of ChatGPT for Machine Translation
We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community.
ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation
In this paper, we propose a novel transfer learning method for NMT, namely ConsistTL, which can continuously transfer knowledge from the parent model during the training of the child model.
Low Resource Neural Machine Translation
Low-Resource Neural Machine Translation
+4
Improving Simultaneous Machine Translation with Monolingual Data
Preliminary experiments on En-Zh and En-Ja news domain corpora demonstrate that monolingual data can significantly improve translation quality (e. g., +3. 15 BLEU on En-Zh).
Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling
Existing research generally treats Chinese character as a minimum unit for representation.
Ranked #1 on
Machine Translation
on WMT2017 Chinese-English
Revisiting Grammatical Error Correction Evaluation and Beyond
Pretraining-based (PT-based) automatic evaluation metrics (e. g., BERTScore and BARTScore) have been widely used in several sentence generation tasks (e. g., machine translation and text summarization) due to their better correlation with human judgments over traditional overlap-based methods.
BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation
Data augmentations (DA) are the cores to achieving robust sequence-to-sequence learning on various natural language processing (NLP) tasks.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation
Inspired by this, we design a new architecture, {\it ODE Transformer}, which is analogous to the Runge-Kutta method that is well motivated in ODE.
Variance-Aware Machine Translation Test Sets
We release 70 small and discriminative test sets for machine translation (MT) evaluation called variance-aware test sets (VAT), covering 35 translation directions from WMT16 to WMT20 competitions.
On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation
Pre-training (PT) and back-translation (BT) are two simple and powerful methods to utilize monolingual data for improving the model performance of neural machine translation (NMT).
Difficulty-Aware Machine Translation Evaluation
The high-quality translation results produced by machine translation (MT) systems still pose a huge challenge for automatic evaluation.
On the Copying Behaviors of Pre-Training for Neural Machine Translation
In response to this problem, we propose a simple and effective method named copying penalty to control the copying behaviors in decoding.
Progressive Multi-Granularity Training for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process via predicting the entire target sequence.
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation
Results demonstrate that the proposed approach can significantly and universally improve translation quality by reducing translation errors on low-frequency words.
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text.
Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation
Meta-learning has been sufficiently validated to be beneficial for low-resource neural machine translation (NMT).
Domain Adaptation
Low Resource Neural Machine Translation
+4
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effective on various NLP tasks.
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
To this end, we introduce an extra Kullback-Leibler divergence term derived by comparing the lexical choice of NAT model and that embedded in the raw data.
DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding
The table detection and handcrafted features in previous works cannot apply to all forms because of their requirements on formats.
AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting
Unlike previous works that merely employed visual features for text detection, this work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter), which learns both visual and linguistic features to significantly reduce ambiguity in text detection.
Norm-Based Curriculum Learning for Neural Machine Translation
We use the norm (aka length or module) of a word embedding as a measure of 1) the difficulty of the sentence, 2) the competence of the model, and 3) the weight of the sentence.
Scene Text Image Super-Resolution in the Wild
For example, it outperforms LapSRN by over 5% and 8%on the recognition accuracy of ASTER and CRNN.
Shared-Private Bilingual Word Embeddings for Neural Machine Translation
For similar source and target words, their embeddings tend to share a part of the features and they cooperatively learn these common representation units.
Pyramid Mask Text Detector
Scene text detection, an essential step of scene text recognition system, is to locate text instances in natural scene images automatically.
Ranked #1 on
Scene Text Detection
on ICDAR 2017 MLT
FOTS: Fast Oriented Text Spotting with a Unified Network
Incidental scene text spotting is considered one of the most difficult and valuable challenges in the document analysis community.
Ranked #4 on
Scene Text Detection
on ICDAR 2017 MLT
