Search Results for author:
Found 65 papers, 41 papers with code
CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
The current landscape lacks a comprehensive benchmark to effectively measure the literate capabilities of LMMs.
P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs
Recent advancements in large language models (LLMs) showcase varied multilingual capabilities across tasks like translation, code generation, and reasoning.
Language Models can Self-Lengthen to Generate Long Texts
The framework consists of two roles: the Generator and the Extender.
Aligning CodeLLMs with Direct Preference Optimization
We also contribute a pipeline for collecting preference pairs for DPO on CodeLLMs.
Aligning Large Language Models via Self-Steering Optimization
The key to automated alignment lies in providing learnable and accurate preference signals for preference learning without human annotation.
Rethinking Data Selection at Scale: Random Selection is Almost All You Need
Supervised fine-tuning (SFT) is crucial for aligning Large Language Models (LLMs) with human instructions.
A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generation
This work tackles the information loss bottleneck of vector-quantization (VQ) autoregressive image generation by introducing a novel model architecture called the 2-Dimensional Autoregression (DnD) Transformer.
Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference
Based on observing activations from large language models, outliers can be classified into channel-wise and spike outliers.
Analyzing and Mitigating Inconsistency in Discrete Audio Tokens for Neural Codec Language Models
This inconsistency can lead to a single audio segment being represented by multiple divergent sequences, which creates confusion in neural codec language models and results in omissions and repetitions during speech generation.
Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
This RM is then applied to the iterative evolution of data in supervised fine-tuning (SFT).
Ranked #1 on
Math Word Problem Solving
on MATH
(using extra training data)
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing.
Ranked #3 on
Temporal Relation Extraction
on Vinoground
Qwen2.5-Coder Technical Report
In this report, we introduce the Qwen2. 5-Coder series, a significant upgrade from its predecessor, CodeQwen1. 5.
Synthesizing Text-to-SQL Data from Weak and Strong LLMs
The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to-SQL tasks.
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
OpenDevin), a platform for the development of powerful and flexible AI agents that interact with the world in similar ways to those of a human developer: by writing code, interacting with a command line, and browsing the web.
Qwen2-Audio Technical Report
We introduce the latest progress of Qwen-Audio, a large-scale audio-language model called Qwen2-Audio, which is capable of accepting various audio signal inputs and performing audio analysis or direct textual responses with regard to speech instructions.
Qwen2 Technical Report
This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models.
Ranked #1 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
Furthermore, we introduce the concept of consistency score to quantitatively measure this inconsistency and analyze the potential for improvement in consistency by relative consistency score.
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
To this end, we introduce FastV, a versatile plug-and-play method designed to optimize computational efficiency by learning adaptive attention patterns in early layers and pruning visual tokens in subsequent ones.
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models
Zooter shows computation efficiency in inference as it introduces only a minor computation overhead of a routing function compared with reward model ranking methods.
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets, as the model selects new data points most distinct from any existing ones according to its current embedding space.
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
Ranked #3 on
Multi-Label Text Classification
on CC3M-TagMask
TouchStone: Evaluating Vision-Language Models by Language Models
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large language models (LLMs).
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
Ranked #3 on
Visual Question Answering (VQA)
on InfiMM-Eval
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
Based on this observation, we propose a data selector based on InsTag to select 6K diverse and complex samples from open-source datasets and fine-tune models on InsTag-selected data.
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts.
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
Ranked #1 on
Semantic Segmentation
on ADE20K
(using extra training data)
Transferring General Multimodal Pretrained Models to Text Recognition
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models
As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data.
Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.
Ranked #1 on
Zero-shot Image Retrieval
on MUGE Retrieval
Prompt Tuning for Generative Multimodal Pretrained Models
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
Ranked #2 on
Visual Entailment
on SNLI-VE test
Instance-wise Prompt Tuning for Pretrained Language Models
Prompt Learning has recently gained great popularity in bridging the gap between pretraining tasks and various downstream tasks.
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Recently, it has been observed that the best uni-modal network outperforms the jointly trained multi-modal network, which is counter-intuitive since multiple signals generally bring more information.
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
Ranked #1 on
Visual Question Answering
on VQA v2 test-std
(yes/no metric)
KNAS: Green Neural Architecture Search
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation
Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
Connecting Language and Vision for Natural Language-Based Vehicle Retrieval
In this paper, we apply one new modality, i. e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario.
Learning Relation Alignment for Calibrated Cross-modal Retrieval
To bridge the semantic gap between the two modalities, previous studies mainly focus on word-region alignment at the object level, lacking the matching between the linguistic relation among the words and the visual relation among the regions.
Ranked #5 on
Image-to-Text Retrieval
on MS COCO
CogView: Mastering Text-to-Image Generation via Transformers
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.
Ranked #53 on
Text-to-Image Generation
on MS COCO
(using extra training data)
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
A Gradient-based Kernel Approach for Efficient Network Architecture Search
Inspired by our new finding, we explore a simple yet effective network architecture search (NAS) approach that leverages gradient correlation and gradient values to find well-performing architectures.
Graph-based Multi-hop Reasoning for Long Text Generation
The reasoning module is responsible for searching skeleton paths from a knowledge graph to imitate the imagination process in the human writing for semantic transfer.
InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining
We pretrain the model with three pretraining tasks, including masked segment modeling (MSM), masked region modeling (MRM) and image-text matching (ITM); and finetune the model on a series of vision-and-language downstream tasks.
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks.
Understanding and Improving Layer Normalization
Unlike them, we find that the derivatives of the mean and variance are more important than forward normalization by re-centering and re-scaling backward gradients.
Ranked #5 on
Machine Translation
on IWSLT2015 English-Vietnamese
Specificity-Driven Cascading Approach for Unsupervised Sentiment Modification
The task of unsupervised sentiment modification aims to reverse the sentiment polarity of the input text while preserving its semantic content without any parallel data.
Sparse Transformer: Concentrated Attention Through Explicit Selection
Extensive experimental results on a series of natural language processing tasks, including neural machine translation, image captioning, and language modeling, all demonstrate the advantages of Sparse Transformer in model performance.
Towards Knowledge-Based Recommender Dialog System
In this paper, we propose a novel end-to-end framework called KBRD, which stands for Knowledge-Based Recommender Dialog System.
Ranked #5 on
Text Generation
on ReDial
A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification
In this way, we can reduce the dependence of the model on the label order, as well as capture high-order correlations between labels.
Imitation Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation models (NAT) have achieved impressive inference speedup.
Towards Knowledge-Based Personalized Product Description Generation in E-commerce
In order to make the description both informative and personalized, KOBE considers a variety of important factors during text generation, including product aspects, user categories, and knowledge base, etc.
Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation
Existing text generation methods tend to produce repeated and {''}boring{''} expressions.
A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification
However, the Seq2Seq model is not suitable for the MLTC task in essence.
Future-Prediction-Based Model for Neural Machine Translation
We propose a novel model for Neural Machine Translation (NMT).
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Different from conventional text generation tasks, the mapping between inputs and responses in conversations is more complicated, which highly demands the understanding of utterance-level semantic dependency, a relation between the whole meanings of inputs and outputs.
Ranked #2 on
Text Generation
on DailyDialog
Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
We propose a novel model for multi-label text classification, which is based on sequence-to-sequence learning.
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation
Most of the Neural Machine Translation (NMT) models are based on the sequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped with the attention mechanism.
Ranked #7 on
Machine Translation
on IWSLT2015 English-Vietnamese
Deconvolution-Based Global Decoding for Neural Machine Translation
A great proportion of sequence-to-sequence (Seq2Seq) models for Neural Machine Translation (NMT) adopt Recurrent Neural Network (RNN) to generate translation word by word following a sequential order.
Ranked #9 on
Machine Translation
on IWSLT2015 English-Vietnamese
Bag-of-Words as Target for Neural Machine Translation
However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage.
Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization
In this work, we supervise the learning of the representation of the source content with that of the summary.
Global Encoding for Abstractive Summarization
To tackle the problem, we propose a global encoding framework, which controls the information flow from the encoder to the decoder based on the global information of the source context.
Ranked #29 on
Text Summarization
on GigaWord
A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification
Text summarization and sentiment classification both aim to capture the main ideas of the text but at different levels.
Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation
ACA learns to control the attention by keeping track of the decoding history and the current information with a memory vector, so that the model can take the translated contents and the current information into consideration.
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Existing text generation methods tend to produce repeated and "boring" expressions.
