Search Results for author:
Found 49 papers, 25 papers with code
VividFace: A Diffusion-Based Hybrid Framework for High-Fidelity Video Face Swapping
Our approach effectively mitigates key challenges in video face swapping, including temporal flickering, identity preservation, and robustness to occlusions and pose variations.
EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM
To effectively exploit consistent visual elements within multiple images, we leverage the multi-image comprehension and instruction-following capabilities of the multimodal large language model (MLLM), prompting it to capture consistent visual elements based on the instruction.
See Further When Clear: Curriculum Consistency Model
Specifically, we regard the distillation process at each timestep as a curriculum and introduce a metric based on Peak Signal-to-Noise Ratio (PSNR) to quantify the learning complexity of this curriculum, then ensure that the curriculum maintains consistent learning complexity across different timesteps by having the teacher model iterate more steps when the noise intensity is low.
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning
Multimodal task specification is essential for enhanced robotic performance, where \textit{Cross-modality Alignment} enables the robot to holistically understand complex task instructions.
MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines
We further present error analysis to unveil current LMMs still struggle to fully grasp the multimodal search tasks, and conduct ablation study to indicate the potential of scaling test-time computation for AI search engine.
Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models
To deal with this issue, we propose a novel framework to fully harness the capabilities of LLMs.
Phased Consistency Models
In this paper, we identify three key flaws in the current design of Latent Consistency Models (LCMs).
Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models
In this study, we propose Deep Reward Tuning (DRTune), an algorithm that directly supervises the final output image of a text-to-image diffusion model and back-propagates through the iterative sampling process to the input noise.
MoVA: Adapting Mixture of Vision Experts to Multimodal Context
In the coarse-grained stage, we design a context-aware expert routing strategy to dynamically select the most suitable vision experts according to the user instruction, input image, and expertise of vision experts.
Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance
Classifier-Free Guidance (CFG) has been widely used in text-to-image diffusion models, where the CFG scale is introduced to control the strength of text guidance on the whole image space.
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching
We further attribute this phenomenon to the diffusion model's insufficient condition utilization, which is caused by its training paradigm.
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
To address these challenges, we collect and introduce the large-scale Visual CoT dataset comprising 438k question-answer pairs, annotated with intermediate bounding boxes highlighting key regions essential for answering the questions.
Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptation
We introduce MOTIA Mastering Video Outpainting Through Input-Specific Adaptation, a diffusion-based pipeline that leverages both the intrinsic data-specific patterns of the source video and the image/video generative prior for effective outpainting.
FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis
In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions.
AnimateLCM: Computation-Efficient Personalized Style Video Generation without Personalized Video Data
This paper introduces an effective method for computation-efficient personalized style video generation without requiring access to any personalized video data.
Towards Large-scale Masked Face Recognition
During the COVID-19 coronavirus epidemic, almost everyone is wearing masks, which poses a huge challenge for deep learning-based face recognition algorithms.
Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection
We observe that different regions of interest in the visual feature map are suitable for performing query classification and box localization tasks, even for the same object.
RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
Text-to-image generation has recently witnessed remarkable achievements.
Ranked #11 on
Text-to-Image Generation
on MS COCO
Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising
To address this challenge, we introduce a novel paradigm dubbed as Gen-L-Video, capable of extending off-the-shelf short video diffusion models for generating and editing videos comprising hundreds of frames with diverse semantic segments without introducing additional training, all while preserving content consistency.
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
The HoP approach is straightforward: given the current timestamp t, we generate a pseudo Bird's-Eye View (BEV) feature of timestamp t-k from its adjacent frames and utilize this feature to predict the object set at timestamp t-k. Our approach is motivated by the observation that enforcing the detector to capture both the spatial location and temporal motion of objects occurring at historical timestamps can lead to more accurate BEV feature learning.
Ranked #3 on
3D Object Detection
on nuScenes Camera Only
UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors
Bridging this semantic gap now requires case-by-case algorithm design which is time-consuming and heavily relies on experienced adjustment.
Masked Autoencoders Are Stronger Knowledge Distillers
In MKD, random patches of the input image are masked, and the corresponding missing feature is recovered by forcing it to imitate the output of the teacher.
DETRs with Collaborative Hybrid Assignments Training
This new training scheme can easily enhance the encoder's learning ability in end-to-end detectors by training the multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS and Faster RCNN.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(using extra training data)
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
Large-batch Optimization for Dense Visual Predictions
To address this challenge, we propose a simple yet effective algorithm, named Adaptive Gradient Variance Modulator (AGVM), which can train dense visual predictors with very large batch size, enabling several benefits more appealing than prior arts.
Towards Robust Face Recognition with Comprehensive Search
To eliminate the bias of single-aspect research and provide an overall understanding of the face recognition model design, we first carefully design the search space for each aspect, then a comprehensive search method is introduced to jointly search optimal data cleaning, architecture, and loss function design.
Unifying Visual Perception by Dispersible Points Learning
The method, called UniHead, views different visual perception tasks as the dispersible points learning via the transformer encoder architecture.
Rethinking Robust Representation Learning Under Fine-grained Noisy Faces
To better understand this, we reformulate the noise type of each class in a more fine-grained manner as N-identities|K^C-clusters.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Finally, we integrate configurable operators and DSMs into a unified search space and search with a Reinforcement Learning-based search algorithm to fully explore the optimal combination of the operators.
Ranked #12 on
Neural Architecture Search
on ImageNet
UniFormer: Unifying Convolution and Self-attention for Visual Recognition
Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing to tackle both redundancy and dependency for efficient and effective representation learning.
Ranked #162 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 9% and 71. 2% top-1 accuracy respectively.
Self-slimmed Vision Transformer
Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs by dynamic token aggregation.
INTERN: A New Learning Paradigm Towards General Vision
Enormous waves of technological innovations over the past several years, marked by the advances in AI technologies, are profoundly reshaping the industry and the society.
Rectifying the Data Bias in Knowledge Distillation
In this paper, we first quantitatively define the uniformity of the sampled data for training, providing a unified view for methods that learn from biased data.
Ranked #1 on
Face Verification
on IJB-C
(training dataset metric)
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Recently, transformer and multi-layer perceptron (MLP) architectures have achieved impressive results on various vision tasks.
Ranked #254 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatial-Temporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 8% and 71. 4% top-1 accuracy respectively.
Ranked #9 on
Action Recognition
on Something-Something V1
Self-Slimming Vision Transformer
Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs by dynamic token aggregation.
FNAS: Uncertainty-Aware Fast Neural Architecture Search
Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times.
Switchable K-Class Hyperplanes for Noise-Robust Representation Learning
When collaborated with the popular ArcFace on million-level data representation learning, we found that the switchable manner in SKH can effectively eliminate the gradient conflict generated by real-world label noise on a single K-class hyperplane.
Discriminability Distillation in Group Representation Learning
We show the discrimiability knowledge has good properties that can be distilled by a light-weight distillation network and can be generalized on the unseen target set.
1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
This technical report introduces our winning solution to the spatio-temporal action localization track, AVA-Kinetics Crossover, in ActivityNet Challenge 2020.
Revisiting the Sibling Head in Object Detector
The ``shared head for classification and localization'' (sibling head), firstly denominated in Fast RCNN~\cite{girshick2015fast}, has been leading the fashion of the object detection community in the past five years.
Ranked #80 on
Object Detection
on COCO test-dev
KPNet: Towards Minimal Face Detector
The small receptive field and capacity of minimal neural networks limit their performance when using them to be the backbone of detectors.
1st Place Solutions for OpenImage2019 -- Object Detection and Instance Segmentation
Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
Top-1 Solution of Multi-Moments in Time Challenge 2019
In this technical report, we briefly introduce the solutions of our team 'Efficient' for the Multi-Moments in Time challenge in ICCV 2019.
Towards Flops-constrained Face Recognition
Large scale face recognition is challenging especially when the computational budget is limited.
Transductive Centroid Projection for Semi-supervised Large-scale Recognition
It is inspired by the observation of the weights in classification layer (called extit{anchors}) converge to the central direction of each class in hyperspace.
Beyond Trade-off: Accelerate FCN-based Face Detector with Higher Accuracy
Fully convolutional neural network (FCN) has been dominating the game of face detection task for a few years with its congenital capability of sliding-window-searching with shared kernels, which boiled down all the redundant calculation, and most recent state-of-the-art methods such as Faster-RCNN, SSD, YOLO and FPN use FCN as their backbone.
Region-based Quality Estimation Network for Large-scale Person Re-identification
One of the major restrictions on the performance of video-based person re-id is partial noise caused by occlusion, blur and illumination.
