Search Results for author:
Found 47 papers, 30 papers with code
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models
Zooter shows computation efficiency in inference as it introduces only a minor computation overhead of a routing function compared with reward model ranking methods.
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets, as the model selects new data points most distinct from any existing ones according to its current embedding space.
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
TouchStone: Evaluating Vision-Language Models by Language Models
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large language models (LLMs).
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
Ranked #2 on
Visual Question Answering (VQA)
on CORE-MM
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
Based on this observation, we propose a data selector based on InsTag to select 6K diverse and complex samples from open-source datasets and fine-tune models on InsTag-selected data.
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts.
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
Ranked #1 on
Audio-Visual Question Answering (AVQA)
on AVQA
(using extra training data)
Transferring General Multimodal Pretrained Models to Text Recognition
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models
As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data.
Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.
Ranked #1 on
Zero-shot Image Retrieval
on MUGE Retrieval
Prompt Tuning for Generative Multimodal Pretrained Models
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
Ranked #2 on
Visual Entailment
on SNLI-VE test
Instance-wise Prompt Tuning for Pretrained Language Models
Prompt Learning has recently gained great popularity in bridging the gap between pretraining tasks and various downstream tasks.
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Recently, it has been observed that the best uni-modal network outperforms the jointly trained multi-modal network, which is counter-intuitive since multiple signals generally bring more information.
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
Ranked #1 on
Visual Question Answering
on VQA v2 test-std
(yes/no metric)
KNAS: Green Neural Architecture Search
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Connecting Language and Vision for Natural Language-Based Vehicle Retrieval
In this paper, we apply one new modality, i. e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation
Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness.
Learning Relation Alignment for Calibrated Cross-modal Retrieval
To bridge the semantic gap between the two modalities, previous studies mainly focus on word-region alignment at the object level, lacking the matching between the linguistic relation among the words and the visual relation among the regions.
Ranked #4 on
Visual Reasoning
on Winoground
CogView: Mastering Text-to-Image Generation via Transformers
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.
Ranked #55 on
Text-to-Image Generation
on COCO
(using extra training data)
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
A Gradient-based Kernel Approach for Efficient Network Architecture Search
Inspired by our new finding, we explore a simple yet effective network architecture search (NAS) approach that leverages gradient correlation and gradient values to find well-performing architectures.
Graph-based Multi-hop Reasoning for Long Text Generation
The reasoning module is responsible for searching skeleton paths from a knowledge graph to imitate the imagination process in the human writing for semantic transfer.
InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining
We pretrain the model with three pretraining tasks, including masked segment modeling (MSM), masked region modeling (MRM) and image-text matching (ITM); and finetune the model on a series of vision-and-language downstream tasks.
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks.
Understanding and Improving Layer Normalization
Unlike them, we find that the derivatives of the mean and variance are more important than forward normalization by re-centering and re-scaling backward gradients.
Ranked #5 on
Machine Translation
on IWSLT2015 English-Vietnamese
Specificity-Driven Cascading Approach for Unsupervised Sentiment Modification
The task of unsupervised sentiment modification aims to reverse the sentiment polarity of the input text while preserving its semantic content without any parallel data.
Sparse Transformer: Concentrated Attention Through Explicit Selection
Extensive experimental results on a series of natural language processing tasks, including neural machine translation, image captioning, and language modeling, all demonstrate the advantages of Sparse Transformer in model performance.
Towards Knowledge-Based Recommender Dialog System
In this paper, we propose a novel end-to-end framework called KBRD, which stands for Knowledge-Based Recommender Dialog System.
Ranked #5 on
Text Generation
on ReDial
A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification
In this way, we can reduce the dependence of the model on the label order, as well as capture high-order correlations between labels.
Imitation Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation models (NAT) have achieved impressive inference speedup.
Towards Knowledge-Based Personalized Product Description Generation in E-commerce
In order to make the description both informative and personalized, KOBE considers a variety of important factors during text generation, including product aspects, user categories, and knowledge base, etc.
Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation
Existing text generation methods tend to produce repeated and {''}boring{''} expressions.
A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification
However, the Seq2Seq model is not suitable for the MLTC task in essence.
Future-Prediction-Based Model for Neural Machine Translation
We propose a novel model for Neural Machine Translation (NMT).
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Different from conventional text generation tasks, the mapping between inputs and responses in conversations is more complicated, which highly demands the understanding of utterance-level semantic dependency, a relation between the whole meanings of inputs and outputs.
Ranked #2 on
Text Generation
on DailyDialog
Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
We propose a novel model for multi-label text classification, which is based on sequence-to-sequence learning.
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation
Most of the Neural Machine Translation (NMT) models are based on the sequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped with the attention mechanism.
Ranked #7 on
Machine Translation
on IWSLT2015 English-Vietnamese
Deconvolution-Based Global Decoding for Neural Machine Translation
A great proportion of sequence-to-sequence (Seq2Seq) models for Neural Machine Translation (NMT) adopt Recurrent Neural Network (RNN) to generate translation word by word following a sequential order.
Ranked #9 on
Machine Translation
on IWSLT2015 English-Vietnamese
Bag-of-Words as Target for Neural Machine Translation
However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage.
Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization
In this work, we supervise the learning of the representation of the source content with that of the summary.
Global Encoding for Abstractive Summarization
To tackle the problem, we propose a global encoding framework, which controls the information flow from the encoder to the decoder based on the global information of the source context.
Ranked #29 on
Text Summarization
on GigaWord
A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification
Text summarization and sentiment classification both aim to capture the main ideas of the text but at different levels.
Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation
ACA learns to control the attention by keeping track of the decoding history and the current information with a memory vector, so that the model can take the translated contents and the current information into consideration.
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Existing text generation methods tend to produce repeated and "boring" expressions.
