Search Results for author:
Found 17 papers, 8 papers with code
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
On the other hand, there are also many existing models and systems (symbolic-based or neural-based) that can do some domain-specific tasks very well.
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation.
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
To this end, We build a system called \textbf{Visual ChatGPT}, incorporating different Visual Foundation Models, to enable the user to interact with ChatGPT by 1) sending and receiving not only languages but also images 2) providing complex visual questions or visual editing instructions that require the collaboration of multiple AI models with multi-steps.
Learning 3D Photography Videos via Self-supervised Diffusion on Single Images
3D photography renders a static image into a video with appealing 3D visual effects.
ReCo: Region-Controlled Text-to-Image Generation
Human evaluation on PaintSkill shows that ReCo is +19. 28% and +17. 21% more accurate in generating images with correct object count and spatial relationship than the T2I model.
HORIZON: A High-Resolution Panorama Synthesis Framework
Panorama synthesis aims to generate a visual scene with all 360-degree views and enables an immersive virtual world.
NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis
In this paper, we present NUWA-Infinity, a generative model for infinite visual synthesis, which is defined as the task of generating arbitrarily-sized high-resolution images or long-duration videos.
Ranked #1 on
Image Outpainting
on LHQC
BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning
Vision-Language (VL) models with the Two-Tower architecture have dominated visual-language representation learning in recent years.
DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder
Our work proposes a VQ-VAE architecture model with a diffusion decoder (DiVAE) to work as the reconstructing component in image synthesis.
VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers
Breakthroughs in transformer-based models have revolutionized not only the NLP field, but also vision and multimodal systems.
NÜWA-LIP: Language Guided Image Inpainting with Defect-free VQGAN
Language guided image inpainting aims to fill in the defective regions of an image under the guidance of text while keeping non-defective regions unchanged.
NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion
To cover language, image, and video at the same time for different scenarios, a 3D transformer encoder-decoder framework is designed, which can not only deal with videos as 3D data but also adapt to texts and images as 1D and 2D data, respectively.
Ranked #1 on
Text-to-Video Generation
on Kinetics
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation
Self-supervised vision-and-language pretraining (VLP) aims to learn transferable multi-modal representations from large-scale image-text data and to achieve strong performances on a broad scope of vision-language tasks after finetuning.
GEM: A General Evaluation Benchmark for Multimodal Tasks
Comparing with existing multimodal datasets such as MSCOCO and Flicker30K for image-language tasks, YouCook2 and MSR-VTT for video-language tasks, GEM is not only the largest vision-language dataset covering image-language tasks and video-language tasks at the same time, but also labeled in multiple languages.
GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions
Generating videos from text is a challenging task due to its high computational requirements for training and infinite possible answers for evaluation.
Ranked #4 on
Text-to-Video Generation
on MSR-VTT
Deep Reason: A Strong Baseline for Real-World Visual Reasoning
This paper presents a strong baseline for real-world visual reasoning (GQA), which achieves 60. 93% in GQA 2019 challenge and won the sixth place.
Chain of Reasoning for Visual Question Answering
A chain of reasoning (CoR) is constructed for supporting multi-step and dynamic reasoning on changed relations and objects.
