Search Results for author:
Found 11 papers, 6 papers with code
Investigating Information-Theoretic Properties of the Typology of Spatial Demonstratives
Using data from Nintemann et al. (2020), we explore the variability in complexity and informativity across spatial demonstrative systems using spatial deictic lexicons from 223 languages.
EAVL: Explicitly Align Vision and Language for Referring Image Segmentation
In previous approaches, fused vision-language features are directly fed into a decoder and pass through a convolution with a fixed kernel to obtain the result, which follows a similar pattern as traditional image segmentation.
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
Due to the limited scale and quality of video-text training corpus, most vision-language foundation models employ image-text datasets for pretraining and primarily focus on modeling visually semantic representations while disregarding temporal semantic representations and correlations.
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst
We show that only language-paired two-modality data is sufficient to connect all modalities.
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending
Towards this goal, we propose a novel video-text pre-training method dubbed VLAB: Video Language pre-training by feature Adapting and Blending, which transfers CLIP representations to video pre-training tasks and develops unified video multimodal models for a wide range of video-text tasks.
Ranked #1 on
TGIF-Frame
on TGIF-QA
(using extra training data)
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner
In this paper, we propose a novel method called Joint QA and DC GEneration (JADE), which utilizes a pre-trained multimodal model and easily-crawled image-text pairs to automatically generate and filter large-scale VQA and dense captioning datasets.
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
Different from widely-studied vision-language pretraining models, VALOR jointly models relationships of vision, audio and language in an end-to-end manner.
Ranked #1 on
Video Captioning
on VATEX
(using extra training data)
Sounding Video Generator: A Unified Framework for Text-guided Sounding Video Generation
In this work, we concentrate on a rarely investigated problem of text guided sounding video generation and propose the Sounding Video Generator (SVG), a unified framework for generating realistic videos along with audio signals.
TJ4DRadSet: A 4D Radar Dataset for Autonomous Driving
The next-generation high-resolution automotive radar (4D radar) can provide additional elevation measurement and denser point clouds, which has great potential for 3D sensing in autonomous driving.
CPTR: Full Transformer Network for Image Captioning
Besides, we provide detailed visualizations of the self-attention between patches in the encoder and the "words-to-patches" attention in the decoder thanks to the full Transformer architecture.
Global-Local Propagation Network for RGB-D Semantic Segmentation
Depth information matters in RGB-D semantic segmentation task for providing additional geometric information to color images.
