Search Results for author:
Found 26 papers, 10 papers with code
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
In addition, taking the gate as a probing, we validate the efficiency and effectiveness of the variable prefix.
Vision Langauge Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation
Cross-modal contrastive learning in vision language pretraining (VLP) faces the challenge of (partial) false negatives.
Transforming Visual Scene Graphs to Image Captions
In TSG, we apply multi-head attention (MHA) to design the Graph Neural Network (GNN) for embedding scene graphs.
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
Our code, pre-trained model, instruction-tuned models, and evaluation set are available at https://github. com/X-PLUG/mPLUG-Owl.
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.
Ranked #1 on
Visual Grounding
on RefCOCO+ testA
Learning Trajectory-Word Alignments for Video-Language Tasks
To amend this, we propose a novel TW-BERT to learn Trajectory-Word alignment by a newly designed trajectory-to-word (T2W) attention for solving video-language tasks.
Adaptively Clustering Neighbor Elements for Image Captioning
We design a novel global-local Transformer named \textbf{Ada-ClustFormer} (\textbf{ACF}) to generate captions.
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e. g., SSv2-Template and SSv2-Label) with 8. 6% and 11. 1% improvement respectively.
Ranked #1 on
Visual Question Answering (VQA)
on TGIF-QA
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
Ranked #1 on
Image Captioning
on COCO Captions
Image Captioning In the Transformer Age
This drawback inspires the researchers to develop a homogeneous architecture that facilitates end-to-end training, for which Transformer is the perfect one that has proven its huge potential in both vision and language domains and thus can be used as the basic component of the visual encoder and language decoder in an IC pipeline.
Achieving Human Parity on Visual Question Answering
The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image.
Ranked #12 on
Visual Question Answering (VQA)
on VQA v2 test-dev
EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching
The datasets will be released to facilitate the development of video captioning metrics.
Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training
Existing approaches to vision-language pre-training (VLP) heavily rely on an object detector based on bounding boxes (regions), where salient objects are first detected from images and then a Transformer-based model is used for cross-modal fusion.
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning
Vision-language pre-training (VLP) on large-scale image-text pairs has achieved huge success for the cross-modal downstream tasks.
We Know What You Want: An Advertising Strategy Recommender System for Online Advertising
To achieve this goal, the advertising platform needs to identify the advertiser's optimization objectives, and then recommend the corresponding strategies to fulfill the objectives.
SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels
Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations.
Learning to Infer User Hidden States for Online Sequential Advertising
To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important.
A Deep Prediction Network for Understanding Advertiser Intent and Satisfaction
For e-commerce platforms such as Taobao and Amazon, advertisers play an important role in the entire digital ecosystem: their behaviors explicitly influence users' browsing and shopping experience; more importantly, advertiser's expenditure on advertising constitutes a primary source of platform revenue.
Neural Topic Modeling with Bidirectional Adversarial Training
Recent years have witnessed a surge of interests of using neural topic models for automatic topic extraction from text, since they avoid the complicated mathematical derivations for model inference as in traditional topic models such as Latent Dirichlet Allocation (LDA).
Ranked #1 on
Text Clustering
on 20 Newsgroups
Learning Syntactic and Dynamic Selective Encoding for Document Summarization
Our approach has the following contributions: first, we incorporate syntactic information such as constituency parsing trees into the encoding sequence to learn both the semantic and syntactic information from the document, resulting in more accurate summary; second, we propose a dynamic gate network to select the salient information based on the context of the decoder state, which is essential to document summarization.
Adversarial Multi-Binary Neural Network for Multi-class Classification
Multi-class text classification is one of the key problems in machine learning and natural language processing.
Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization
Abstractive text summarization is a challenging task, and one need to design a mechanism to effectively extract salient information from the source text and then generate a summary.
Learning Alignment for Multimodal Emotion Recognition from Speech
Further, emotion recognition will be beneficial from using audio-textual multimodal information, it is not trivial to build a system to learn from multimodality.
Multimodal Emotion Recognition
Speech Emotion Recognition
+2
DELTA: A DEep learning based Language Technology plAtform
In this paper we present DELTA, a deep learning based language technology platform.
Ranked #3 on
Text Classification
on Yahoo! Answers
