Search Results for author:
Found 44 papers, 27 papers with code
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
We introduce VASA, a framework for generating lifelike talking faces with appealing visual affective skills (VAS) given a single static image and a speech audio clip.
GaussianCube: Structuring Gaussian Splatting using Optimal Transport for 3D Generative Modeling
To address the problem, we introduce GaussianCube, a structured GS representation that is both powerful and efficient for generative modeling.
Simplified Diffusion Schrödinger Bridge
This paper introduces a novel theoretical simplification of the Diffusion Schr\"odinger Bridge (DSB) that facilitates its unification with Score-based Generative Models (SGMs), addressing the limitations of DSB in complex data generation and enabling faster convergence and enhanced performance.
VisualCritic: Making LMMs Perceive Visual Quality Like Humans
In this paper, we explore this question and provide the answer "Yes!".
RelationVLM: Making Large Vision-Language Models Understand Visual Relations
The development of Large Vision-Language Models (LVLMs) is striving to catch up with the success of Large Language Models (LLMs), yet it faces more challenges to be resolved.
CCA: Collaborative Competitive Agents for Image Editing
This paper presents a novel generative model, Collaborative Competitive Agents (CCA), which leverages the capabilities of multiple Large Language Models (LLMs) based agents to execute complex tasks.
VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder
The 3D volumes are then trained on a diffusion model for text-to-3D generation using a 3D U-Net.
MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation
We present MicroCinema, a straightforward yet effective framework for high-quality and coherent text-to-video generation.
COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design
Our COLE system comprises multiple fine-tuned Large Language Models (LLMs), Large Multimodal Models (LMMs), and Diffusion Models (DMs), each specifically tailored for design-aware layer-wise captioning, layout planning, reasoning, and the task of generating images and text.
CCEdit: Creative and Controllable Video Editing via Diffusion Models
The versatility of our framework is demonstrated through a diverse range of choices in both structure representations and personalized T2I models, as well as the option to provide the edited key frame.
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks
We present InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions.
V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection
We introduce a highly performant 3D object detector for point clouds using the DETR framework.
Ranked #2 on
3D Object Detection
on ScanNetV2
Adaptive Frequency Filters As Efficient Global Token Mixers
With this insight, we propose Adaptive Frequency Filtering (AFF) token mixer.
Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding
We pretrained a large {\SST} model on a synthetic Structured3D dataset, which is an order of magnitude larger than the ScanNet dataset.
Ranked #2 on
3D Object Detection
on S3DIS
(using extra training data)
IRGen: Generative Modeling for Image Retrieval
While generative modeling has been ubiquitous in natural language processing and computer vision, its application to image retrieval remains unexplored.
Efficient Diffusion Training via Min-SNR Weighting Strategy
Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence.
Ranked #1 on
Image Generation
on ImageNet 256x256
RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch
A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen.
Improving CLIP Fine-tuning Performance
Experiments suggest that the feature map distillation approach significantly boosts the fine-tuning performance of CLIP models on several typical downstream vision tasks.
iCLIP: Bridging Image Classification and Contrastive Language-Image Pre-Training for Visual Recognition
This paper presents a method that effectively combines two prevalent visual recognition methods, i. e., image classification and contrastive language-image pre-training, dubbed iCLIP.
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion
This paper presents a 3D generative model that uses diffusion models to automatically generate 3D digital avatars represented as neural radiance fields.
Fine-Grained Image Style Transfer with Visual Transformers
Such a design usually destroys the spatial information of the input images and fails to transfer fine-grained style patterns into style transfer results.
Implicit Conversion of Manifold B-Rep Solids by Neural Halfspace Representation
We present a novel implicit representation -- neural halfspace representation (NH-Rep), to convert manifold B-Rep solids to implicit representations.
Language-Guided Face Animation by Recurrent StyleGAN-based Generator
Specifically, we propose a recurrent motion generator to extract a series of semantic and motion information from the language and feed it along with visual information to a pre-trained StyleGAN to generate high-quality frames.
ComplexGen: CAD Reconstruction by B-Rep Chain Complex Generation
We view the reconstruction of CAD models in the boundary representation (B-Rep) as the detection of geometric primitives of different orders, i. e. vertices, edges and surface patches, and the correspondence of primitives, which are holistically modeled as a chain complex, and show that by modeling such comprehensive structures more complete and regularized reconstructions can be achieved.
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation
These properties, which we aggregately refer to as optimization friendliness, are identified and analyzed by a set of attention- and optimization-related diagnosis tools.
Ranked #2 on
Instance Segmentation
on COCO test-dev
(using extra training data)
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition
Second, we convert the image classification problem from learning parametric category classifier weights to learning a text encoder as a meta network to generate category classifier weights.
Protecting Celebrities from DeepFake with Identity Consistency Transformer
In this work we propose Identity Consistency Transformer, a novel face forgery detection method that focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions.
StyleSwin: Transformer-based GAN for High-resolution Image Generation
To this end, we believe that local attention is crucial to strike the balance between computational efficiency and modeling capacity.
Ranked #1 on
Image Generation
on CelebA 256x256
(FID metric)
VirtualCube: An Immersive 3D Video Communication System
The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies.
Vector Quantized Diffusion Model for Text-to-Image Synthesis
Our experiments indicate that the VQ-Diffusion model with the reparameterization is fifteen times faster than traditional AR methods while achieving a better image quality.
Ranked #1 on
Text-to-Image Generation
on Oxford 102 Flowers
(using extra training data)
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
To enable VL pre-training, we jointly optimize the HD-VILA model by a hybrid Transformer that learns rich spatiotemporal features, and a multimodal Transformer that enforces interactions of the learned video features with diversified texts.
Ranked #16 on
Video Retrieval
on MSR-VTT
Swin Transformer V2: Scaling Up Capacity and Resolution
Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images.
Ranked #4 on
Image Classification
on ImageNet V2
(using extra training data)
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
By further pretraining on the larger dataset ImageNet-21K, we achieve 87. 5% Top-1 accuracy on ImageNet-1K and high segmentation performance on ADE20K with 55. 7 mIoU.
Ranked #25 on
Semantic Segmentation
on ADE20K val
Aggregated Contextual Transformations for High-Resolution Image Inpainting
For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task.
Ranked #9 on
Image Inpainting
on Places2
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision.
Ranked #2 on
Image Classification
on OmniBenchmark
Identity-Driven DeepFake Detection
Our approach takes as input the suspect image/video as well as the target identity information (a reference image or video).
Learning Texture Transformer Network for Image Super-Resolution
In this paper, we propose a novel Texture Transformer Network for Image Super-Resolution (TTSR), in which the LR and Ref images are formulated as queries and keys in a transformer, respectively.
Face X-ray for More General Face Forgery Detection
For this reason, face X-ray provides an effective way for detecting forgery generated by most existing face manipulation algorithms.
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
As the missing content can be filled by attention transfer from deep to shallow in a pyramid fashion, both visual and semantic coherence for image inpainting can be ensured.
Compressing Neural Networks using the Variational Information Bottelneck
Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture.
Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders
With the growing popularity of short-form video sharing platforms such as \em{Instagram} and \em{Vine}, there has been an increasing need for techniques that automatically extract highlights from video.
Orientational Pyramid Matching for Recognizing Indoor Scenes
The novelty lies in that OPM uses the 3D orientations to form the pyramid and produce the pooling regions, which is unlike SPM that uses the spatial positions to form the pyramid.
Fast Neighborhood Graph Search using Cartesian Concatenation
This structure augments the neighborhood graph with a bridge graph.
