Search Results for author:
Found 388 papers, 200 papers with code
Adversarial Training with Bi-directional Likelihood Regularization for Visual Classification
We call such a new robust training strategy the adversarial training with bi-directional likelihood regularization (ATBLR) method.
Video Object Detection via Object-level Temporal Aggregation
While single-image object detectors can be naively applied to videos in a frame-by-frame fashion, the prediction is often temporally inconsistent.
HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior
Text-to-image diffusion models have emerged as powerful priors for real-world image super-resolution (Real-ISR).
Hierarchical Information Flow for Generalized Efficient Image Restoration
To strike a balance between efficiency and model capacity for a generalized transformer-based IR method, we propose a hierarchical information flow mechanism for image restoration, dubbed Hi-IR, which progressively propagates information among pixels in a bottom-up manner.
Ranked #1 on
Image Deblurring
on HIDE (trained on GOPRO)
OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection
Moreover, we propose a vision-centric 3D open-world object detection baseline and further introduce an ensemble method by fusing general and specialized models to address the issue of lower precision in existing open-world methods for the OpenAD benchmark.
Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation
In this work, we introduce a novel approach that overcomes this limitation by incorporating object-level contextual knowledge within images.
Extending Video Masked Autoencoders to 128 frames
Nevertheless, the majority of prior works that leverage MAE pre-training have focused on relatively short video representations (16 / 32 frames in length) largely due to hardware memory and compute limitations that scale poorly with video length due to the dense memory-intensive self-attention decoding.
Ranked #1 on
Action Recognition
on Diving-48
(using extra training data)
PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing
We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations.
No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images
We utilize the reconstructed 3D Gaussians for novel view synthesis and pose estimation tasks and propose a two-stage coarse-to-fine pipeline for accurate pose estimation.
Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint
Starting with a 2D layout provided by a user or generated from a text description, we first create a coarse 3D scene using a carefully designed initialization process based on efficient reconstruction models.
OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities
To address this, OmnixR offers two evaluation variants: (1)synthetic subset: a synthetic dataset generated automatically by translating text into multiple modalities--audio, images, video, and hybrids (Omnify).
A Simple Approach to Unifying Diffusion-based Conditional Generation
Recent progress in image generation has sparked research into controlling these models through condition signals, with various methods addressing specific challenges in conditional generation.
KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities
Using KITTEN, we conduct a systematic study on the fidelity of entities in text-to-image generation models, focusing on their ability to generate a wide range of real-world visual entities, such as landmark buildings, aircraft, plants, and animals.
Tex4D: Zero-shot 4D Scene Texturing with Video Diffusion Models
In this work, we present Tex4D, a zero-shot approach that integrates inherent 3D geometry knowledge from mesh sequences with the expressiveness of video diffusion models to produce multi-view and temporally consistent 4D textures.
PredFormer: Transformers Are Effective Spatial-Temporal Predictive Learners
On Moving MNIST, PredFormer achieves a 51. 3% reduction in MSE relative to SimVP.
Ranked #1 on
Video Prediction
on Moving MNIST
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion
Estimating geometry from dynamic scenes, where objects move and deform over time, remains a core challenge in computer vision.
LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentation
Despite the promising performance of current video segmentation models on existing benchmarks, these models still struggle with complex scenes.
Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th LSVOS
Video object segmentation (VOS) is a crucial task in computer vision, but current VOS methods struggle with complex scenes and prolonged object motions.
Re-boosting Self-Collaboration Parallel Prompt GAN for Unsupervised Image Restoration
As SE increases computational complexity during inference, we propose a re-boosting module to the SC (Reb-SC) to improve the SC strategy further by incorporating SE into SC without increasing inference time.
Cropper: Vision-Language Model for Image Cropping through In-Context Learning
The goal of image cropping is to identify visually appealing crops within an image.
LLAVADI: What Matters For Multimodal Large Language Models Distillation
The recent surge in Multimodal Large Language Models (MLLMs) has showcased their remarkable potential for achieving generalized intelligence by integrating visual understanding into Large Language Models. Nevertheless, the sheer model size of MLLMs leads to substantial memory and computational demands that hinder their widespread deployment.
Learning Spatial-Semantic Features for Robust Video Object Segmentation
Tracking and segmenting multiple similar objects with complex or separate parts in long-term videos is inherently challenging due to the ambiguity of target parts and identity confusion caused by occlusion, background clutter, and long-term variations.
Chat-Edit-3D: Interactive 3D Scene Editing via Text Prompts
Recent work on image content manipulation based on vision-language pre-training models has been effectively extended to text-driven 3D scene editing.
PVUW 2024 Challenge on Complex Video Understanding: Methods and Results
Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments.
1st Place Solution for MOSE Track in CVPR 2024 PVUW Workshop: Complex Video Object Segmentation
In this challenge, we propose a semantic embedding video object segmentation model and use the salient features of objects as query representations.
SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow
For image synthesis, we propose a finite perturbation approach to enhance the diversity of generated results without changing the semantic categories.
Sharing Key Semantics in Transformer Makes Efficient Image Restoration
Additionally, for IR, it is commonly noted that small segments of a degraded image, particularly those closely aligned semantically, provide particularly relevant information to aid in the restoration process, as they contribute essential contextual cues crucial for accurate reconstruction.
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model
This foundational estimation facilitates a detailed, coarse-to-fine segmentation strategy that significantly enhances the precision of object identification and segmentation.
Efficient Visual State Space Model for Image Deblurring
Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration.
Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance
To address this issue, we introduce an effective approach to stabilize the proposal-target matching in point-based methods.
Ranked #1 on
Crowd Counting
on NWPU-Crowd
Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidance
In subject-driven text-to-image synthesis, the synthesis process tends to be heavily influenced by the reference images provided by users, often overlooking crucial attributes detailed in the text prompt.
Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring
In particular, we use a motion estimation network to capture motion information from neighborhoods, thereby adaptively estimating spatially-variant motion flow, mask, kernels, weights, and offsets to obtain the MISC Filter.
AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters
Towards this goal, we propose AdaIR, a novel framework that enables low storage cost and efficient training without sacrificing performance.
Taming Latent Diffusion Model for Neural Radiance Field Inpainting
These two problems are further reinforced with the use of pixel-distance losses.
No More Ambiguity in 360° Room Layout via Bi-Layout Estimation
A unique property of our Bi-Layout model is its ability to inherently detect ambiguous regions by comparing the two predictions.
Gaga: Group Any Gaussians via 3D-aware Memory Bank
We introduce Gaga, a framework that reconstructs and segments open-world 3D scenes by leveraging inconsistent 2D masks predicted by zero-shot segmentation models.
Spatial-Temporal Multi-level Association for Video Object Segmentation
In addition, we propose a spatial-temporal memory to assist feature association and temporal ID assignment and correlation.
Efficient Concertormer for Image Deblurring and Beyond
By retaining partial information in additional dimensions independent from the self-attention calculations, our method effectively captures global contextual representations with complexity linear to the image size.
HENet: Hybrid Encoding for End-to-end Multi-task 3D Perception from Multi-view Cameras
Three-dimensional perception from multi-view cameras is a crucial component in autonomous driving systems, which involves multiple tasks like 3D object detection and bird's-eye-view (BEV) semantic segmentation.
Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration
All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation.
RTracker: Recoverable Tracking via PN Tree Structured Memory
Upon the PN tree memory, we develop corresponding walking rules for determining the state of the target and define a set of control flows to unite the tracker and the detector in different tracking scenarios.
Ranked #1 on
Visual Object Tracking
on VideoCube
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation.
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Next, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation.
Interactive Multi-Head Self-Attention with Linear Complexity
In this work, we propose an effective method to decompose the attention operation into query- and key-less components.
Scene Prior Filtering for Depth Super-Resolution
Specifically, we design an All-in-one Prior Propagation that computes the similarity between multi-modal scene priors, i. e., RGB, normal, semantic, and depth, to reduce the texture interference.
VideoPrism: A Foundational Visual Encoder for Video Understanding
We introduce VideoPrism, a general-purpose video encoder that tackles diverse video understanding tasks with a single frozen model.
StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing
Given a script, the challenge in Movie Dubbing (Visual Voice Cloning, V2C) is to generate speech that aligns well with the video in both time and emotion, based on the tone of a reference audio track.
Training Class-Imbalanced Diffusion Model Via Overlap Optimization
To address the observed appearance overlap between synthesized images of rare classes and tail classes, we propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting
We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation.
PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal
For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts.
Generalizable Entity Grounding via Assistance of Large Language Model
In this work, we propose a novel approach to densely ground visual entities from a long caption.
RAP-SAM: Towards Real-Time All-Purpose Segment Anything
Segment Anything Model (SAM) is one remarkable model that can achieve generalized segmentation.
No More Ambiguity in 360deg Room Layout via Bi-Layout Estimation
Specifically on the MatterportLayout dataset it improves 3DIoU from 81. 70% to 82. 57% across the full test set and notably from 54. 80% to 59. 97% in subsets with significant ambiguity.
VideoGrounding-DINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding
Our contributions include a novel spatio-temporal video grounding model surpassing state-of-the-art results in closed-set evaluations on multiple datasets and demonstrating superior performance in open-vocabulary scenarios.
Weakly Supervised Video Individual Counting
To provide a more realistic reflection of the underlying practical challenge we introduce a weakly supervised VIC task wherein trajectory labels are not provided.
Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding
Our contributions include a novel spatio-temporal video grounding model, surpassing state-of-the-art results in closed-set evaluations on multiple datasets and demonstrating superior performance in open-vocabulary scenarios.
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #4 on
Text-to-Video Generation
on MSR-VTT
VidToMe: Video Token Merging for Zero-Shot Video Editing
In this work, we propose a novel approach to enhance temporal consistency in generated videos by merging self-attention tokens across frames.
DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
We present DrivingGaussian, an efficient and effective framework for surrounding dynamic autonomous driving scenes.
PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection
Recent temporal LiDAR-based 3D object detectors achieve promising performance based on the two-stage proposal-based approach.
Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection
\Eg, achieving 85. 4 mAD that surpasses UniAD by +3. 0 for the MVTec AD dataset, and it requires only 1. 1 hours and 2. 3G GPU memory to complete model training on a single V100 that can serve as a strong baseline to facilitate the development of future research.
Weakly Supervised 3D Object Detection via Multi-Level Visual Guidance
Finally, the training-level constraint is utilized by producing accurate and consistent 3D pseudo-labels that align with the visual data.
Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting
% To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided.
CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen
Addressing Out-Of-Distribution (OOD) Segmentation and Zero-Shot Semantic Segmentation (ZS3) is challenging, necessitating segmenting unseen classes.
Towards 4D Human Video Stylization
To overcome these limitations, we leverage Neural Radiance Fields (NeRFs) to represent videos, conducting stylization in the rendered feature space.
DreaMo: Articulated 3D Reconstruction From A Single Casual Video
In this work, we study articulated 3D shape reconstruction from a single and casually captured internet video, where the subject's view coverage is incomplete.
Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection
Semi-supervised object detection is crucial for 3D scene understanding, efficiently addressing the limitation of acquiring large-scale 3D bounding box annotations.
Fine-grained Controllable Video Generation via Object Appearance and Context
To achieve detailed control, we propose a unified framework to jointly inject control signals into the existing text-to-video model.
Multi-task Image Restoration Guided By Robust DINO Features
To guide the restoration model with the features of DINOv2, we develop a DINO-Restore adaption and fusion module to adjust the channel of fused features from PSF and then integrate them with the features from the restoration model.
UniGS: Unified Representation for Image Generation and Segmentation
On the other hand, the progressive dichotomy module can efficiently decode the synthesized colormap to high-quality entity-level masks in a depth-first binary search without knowing the cluster numbers.
Effective Adapter for Face Recognition in the Wild
Traditional heuristic approaches-either training models directly on these degraded images or their enhanced counterparts using face restoration techniques-have proven ineffective, primarily due to the degradation of facial features and the discrepancy in image domains.
Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection
To address these limitations, we propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection, which learns multi-scale temporal features.
Exploiting Diffusion Prior for Generalizable Dense Prediction
Contents generated by recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate due to the immitigable domain gap.
Text-Driven Image Editing via Learnable Regions
Language has emerged as a natural interface for image editing.
Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence
This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing.
Ranked #1 on
Semantic correspondence
on SPair-71k
Pyramid Diffusion for Fine 3D Large Scene Generation
We introduce a framework, the Pyramid Discrete Diffusion model (PDD), which employs scale-varied diffusion models to progressively generate high-quality outdoor scenes.
Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion
We benchmarked the proposed evaluation metrics on 12 open-vocabulary methods of three segmentation tasks.
GLaMM: Pixel Grounding Large Multimodal Model
In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
One-for-All: Towards Universal Domain Translation with a Single StyleGAN
In this paper, we propose a novel translation model, UniTranslator, for transforming representations between visually distinct domains under conditions of limited training data and significant visual differences.
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Video Timeline Modeling For News Story Understanding
In this paper, we present a novel problem, namely video timeline modeling.
SAMPLING: Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image
Recent novel view synthesis methods obtain promising results for relatively small scenes, e. g., indoor environments and scenes with a few objects, but tend to fail for unbounded outdoor scenes with a single image as input.
Editing 3D Scenes via Text Prompts without Retraining
To tackle these issues, we introduce a text-driven editing method, termed DN2N, which allows for the direct acquisition of a NeRF model with universal editing capabilities, eliminating the requirement for retraining.
CiteTracker: Correlating Image and Text for Visual Tracking
Existing visual tracking methods typically take an image patch as the reference of the target to perform tracking.
Delving into Motion-Aware Matching for Monocular 3D Object Tracking
In this paper, we find that the motion cue of objects along different time frames is critical in 3D multi-object tracking, which is less explored in existing monocular-based approaches.
Dual Associated Encoder for Face Restoration
Restoring facial details from low-quality (LQ) images has remained a challenging problem due to its ill-posedness induced by various degradations in the wild.
Ranked #2 on
Blind Face Restoration
on WIDER
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
CLR: Channel-wise Lightweight Reprogramming for Continual Learning
Reprogramming parameters are task-specific and exclusive to each task, which makes our method immune to catastrophic forgetting.
Self-regulating Prompts: Foundational Model Adaptation without Forgetting
To the best of our knowledge, this is the first regularization framework for prompt learning that avoids overfitting by jointly attending to pre-trained model features, the training trajectory during prompting, and the textual diversity.
Ranked #2 on
Prompt Engineering
on ImageNet V2
VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate the video understanding capabilities of existing foundation models (FMs) using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition, temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring an FM for downstream tasks.
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs
In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such as images or videos.
Counting Crowds in Bad Weather
Crowd counting has recently attracted significant attention in the field of computer vision due to its wide applications to image understanding.
AIMS: All-Inclusive Multi-Level Segmentation
Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-interest selections remains unsolved.
A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
Text-to-image diffusion models have made significant advances in generating and editing high-quality images.
Ranked #1 on
Dense Pixel Correspondence Estimation
on TSS
Dense Pixel Correspondence Estimation
Representation Learning
+2
Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark
Our model features two main components: a Dual Degradation Representation Network (DDR-Net) and a Restoration Network.
Motion-Conditioned Diffusion Model for Controllable Video Synthesis
In response to this gap, we introduce MCDiff, a conditional diffusion model that generates a video from a starting image frame and a set of strokes, which allow users to specify the intended content and dynamics for synthesis.
Video Generation Beyond a Single Clip
We tackle the long video generation problem, i. e.~generating videos beyond the output length of video generation models.
Burstormer: Burst Image Restoration and Enhancement Transformer
Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
Structured Video-Language Modeling with Temporal Grouping and Spatial Grounding
Existing video-language pre-training methods primarily focus on instance-level alignment between video clips and captions via global contrastive learning but neglect rich fine-grained local information in both videos and text, which is of importance to downstream tasks requiring temporal localization and semantic reasoning.
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Using our proposed efficient additive attention, we build a series of models called "SwiftFormer" which achieves state-of-the-art performance in terms of both accuracy and mobile inference speed.
Unified Visual Relationship Detection with Vision and Language Models
To address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs).
Human-Object Interaction Detection
Relationship Detection
+2
InfiniCity: Infinite-Scale City Synthesis
Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises.
Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation
Few-Shot Instance Segmentation (FSIS) requires detecting and segmenting novel classes with limited support examples.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
MiniROAD: Minimal RNN Framework for Online Action Detection
Online Action Detection (OAD) is the task of identifying actions in streaming videos without access to future frames.
Ranked #1 on
Online Action Detection
on TVSeries
Self-Supervised Super-Plane for Neural 3D Reconstruction
In this work, we introduce a self-supervised super-plane constraint by exploring the free geometry cues from the predicted surface, which can further regularize the reconstruction of plane regions without any other ground truth annotations.
High Quality Entity Segmentation
Given the high-quality and -resolution nature of the dataset, we propose CropFormer which is designed to tackle the intractability of instance-level segmentation on high-resolution images.
Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble
Automatically estimating 3D skeleton, shape, camera viewpoints, and part articulation from sparse in-the-wild image ensembles is a severely under-constrained and challenging problem.
Learning Object-level Point Augmentor for Semi-supervised 3D Object Detection
In this work, we propose an object-level point augmentor (OPA) that performs local transformations for semi-supervised 3D object detection.
BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios
Based on the property of outdoor point clouds in autonomous driving scenarios, i. e., the point clouds of distant objects are more sparse, we propose point density prediction to enable the 3D encoder to learn location information, which is essential for object detection.
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
Physics-based Indirect Illumination for Inverse Rendering
We present a physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images.
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks.
Progressive Multi-resolution Loss for Crowd Counting
Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth.
Learning to Dub Movies via Hierarchical Prosody Models
Given a piece of text, a video clip and a reference audio, the movie dubbing (also known as visual voice clone V2C) task aims to generate speeches that match the speaker's emotion presented in the video using the desired speaker voice as reference.
Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection
Weakly supervised video anomaly detection aims to identify abnormal events in videos using only video-level labels.
Consistency-Aware Anchor Pyramid Network for Crowd Localization
Crowd localization aims to predict the spatial position of humans in a crowd scenario.
Self-supervised AutoFlow
Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets, four other datasets, and other model architectures such as LiT.
Exploiting Category Names for Few-Shot Classification with Vision-Language Models
Vision-language foundation models pretrained on large-scale data provide a powerful tool for many visual understanding tasks.
Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training
Generating images from graph-structured inputs, such as scene graphs, is uniquely challenging due to the difficulty of aligning nodes and connections in graphs with objects and their relations in images.
High-Quality Entity Segmentation
It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image.
ScoreMix: A Scalable Augmentation Strategy for Training GANs with Limited Data
Then, we optimize the augmented samples by minimizing the norms of the data scores, i. e., the gradients of the log-density functions.
GAN-based Facial Attribute Manipulation
Facial Attribute Manipulation (FAM) aims to aesthetically modify a given face image to render desired attributes, which has received significant attention due to its broad practical applications ranging from digital entertainment to biometric forensics.
Diffusion Models: A Comprehensive Survey of Methods and Applications
This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.
Learning Visibility for Robust Dense Human Body Estimation
An alternative approach is to estimate dense vertices of a predefined template body in the image space.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning
In the first stage, we obtain a robust feature extractor, which could serve for all images with base and novel categories.
Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models
Utilizing vision and language models (VLMs) pre-trained on large-scale image-text pairs is becoming a promising paradigm for open-vocabulary visual recognition.
Ranked #1 on
Zero-Shot Action Recognition
on HMDB51
LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery
In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse).
FlowNAS: Neural Architecture Search for Optical Flow Estimation
To address this issue, we propose a neural architecture search method named FlowNAS to automatically find the better encoder architecture for flow estimation task.
Unveiling The Mask of Position-Information Pattern Through the Mist of Image Features
Recent studies show that paddings in convolutional neural networks encode absolute position information which can negatively affect the model performance for certain tasks.