Search Results for author:
Found 13 papers, 6 papers with code
Accommodating Audio Modality in CLIP for Multimodal Processing
In this paper, we extend the stateof-the-art Vision-Language model CLIP to accommodate the audio modality for Vision-Language-Audio multimodal processing.
Unifying Event Detection and Captioning as Sequence Generation via Pre-Training
Dense video captioning aims to generate corresponding text descriptions for a series of events in the untrimmed video, which can be divided into two sub-tasks, event detection and event captioning.
Some theoretical results on discrete contour trees
Contours are defined on a continuous scalar field.
Progressive Learning for Image Retrieval with Hybrid-Modality Queries
Image retrieval with hybrid-modality queries, also known as composing text and image for image retrieval (CTI-IR), is a retrieval task where the search intention is expressed in a more complex query format, involving both vision and text modalities.
Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training
Firstly, there are many specialized jargons in the product description, which are ambiguous to translate without the product image.
Team RUC_AIM3 Technical Report at ActivityNet 2021: Entities Object Localization
For the object grounding, we fine-tune the state-of-the-art detection model MDETR and design a post processing method to make the grounding results more faithful.
Towards Diverse Paragraph Captioning for Untrimmed Videos
Video paragraph captioning aims to describe multiple events in untrimmed videos with descriptive paragraphs.
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model.
Ranked #1 on
Image Retrieval
on RUC-CAS-WenLan
Team RUC_AIM3 Technical Report at Activitynet 2020 Task 2: Exploring Sequential Events Detection for Dense Video Captioning
Detecting meaningful events in an untrimmed video is essential for dense video captioning.
Ranked #3 on
Dense Video Captioning
on ActivityNet Captions
Integrating Temporal and Spatial Attentions for VATEX Video Captioning Challenge 2019
This notebook paper presents our model in the VATEX video captioning challenge.
Unpaired Cross-lingual Image Caption Generation with Self-Supervised Rewards
We employ self-supervision from mono-lingual corpus in the target language to provide fluency reward, and propose a multi-level visual semantic matching model to provide both sentence-level and concept-level visual relevancy rewards.
Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos
The overall system achieves the state-of-the-art performance on the dense-captioning events in video task with 9. 91 METEOR score on the challenge testing set.
RUC+CMU: System Report for Dense Captioning Events in Videos
This notebook paper presents our system in the ActivityNet Dense Captioning in Video task (task 3).
