Search Results for author:
Found 91 papers, 68 papers with code
GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models
OpenBias detects and quantifies biases, while GradBias determines the contribution of individual prompt words on biases.
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
We discover that simply concatenating visual tokens from a set of complementary vision encoders is as effective as more complex mixing architectures or strategies.
Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis
We present Zero-Painter, a novel training-free framework for layout-conditional text-to-image synthesis that facilitates the creation of detailed and controlled imagery from textual prompts.
Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment
To adapt the source model to the synthetic domain of the unconditional diffusion model, we introduce a Synthetic-Domain Alignment (SDA) framework to fine-tune the source model with synthetic data.
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts
Recent advancements in Multimodal Large Language Models (LLMs) have focused primarily on scaling by increasing text-image pair data and enhancing LLMs to improve performance on multimodal tasks.
Ranked #1 on
visual instruction following
on LLaVA-Bench
UVMap-ID: A Controllable and Personalized UV Map Generative Model
To solve the above problems, we introduce a novel method, UVMap-ID, which is a controllable and personalized UV Map generative model.
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set.
Learning Trimaps via Clicks for Image Matting
Despite significant advancements in image matting, existing models heavily depend on manually-drawn trimaps for accurate results in natural image scenarios.
Benchmarking Object Detectors with COCO: A New Path Forward
With these findings, we advocate using COCO-ReM for future object detection research.
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
To overcome these limitations, we introduce StreamingT2V, an autoregressive approach for long video generation of 80, 240, 600, 1200 or more frames with smooth transitions.
Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level
We observe that our fused kernels successfully circumvent some of the unavoidable inefficiencies in unfused implementations.
Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community
These findings highlight the relevance and effectiveness of Social Reward in assessing community appreciation for AI-generated artworks, establishing a closer alignment with users' creative goals: creating popular visual art.
VASE: Object-Centric Appearance and Shape Manipulation of Real Videos
Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models.
Brush2Prompt: Contextual Prompt Generator for Object Inpainting
Object inpainting is a task that involves adding objects to real images and seamlessly compositing them.
PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor
We propose PAIR Diffusion a generic framework that enables a diffusion model to control the structure and appearance properties of each object in the image.
VCoder: Versatile Vision Encoders for Multimodal Large Language Models
Secondly, we leverage the images from COCO and outputs from off-the-shelf vision perception models to create our COCO Segmentation Text (COST) dataset for training and evaluating MLLMs on the object perception task.
HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models
Recent progress in text-guided image inpainting, based on the unprecedented success of text-to-image diffusion models, has led to exceptionally realistic and visually plausible results.
Diffusion for Natural Image Matting
However, the presence of high computational overhead and the inconsistency of noise sampling between the training and inference processes pose significant obstacles to achieving this goal.
Ranked #1 on
Image Matting
on Distinctions-646
Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
Specifically, we introduce Step-wise Variation Regularization to enforce the proportion between the variations of an arbitrary input latent and that of the output image is a constant at any diffusion training step.
HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models
We further extend our method to a novel image editing task: substituting the subject in an image through textual manipulations.
Video Instance Matting
To remedy this deficiency, we propose Video Instance Matting~(VIM), that is, estimating alpha mattes of each instance at each frame of a video sequence.
Combating Label Noise With A General Surrogate Model For Sample Selection
Sample selection is an effective way to deal with label noise.
Multi-Concept T2I-Zero: Tweaking Only The Text Embeddings and Nothing Else
To achieve this goal, we identify the limitations in the text embeddings used for the pre-trained text-to-image diffusion models.
Learning Mask-aware CLIP Representations for Zero-Shot Segmentation
However, in the paper, we reveal that CLIP is insensitive to different mask proposals and tends to produce similar predictions for various mask proposals of the same image.
Open Vocabulary Semantic Segmentation
Zero Shot Segmentation
Interactive Neural Painting
This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP.
Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap
We propose Reference-based Painterly Inpainting, a novel task that crosses the wild reference domain gap and implants novel objects into artworks.
FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude
Whole-body biometric recognition is an important area of research due to its vast applications in law enforcement, border security, and surveillance.
Matting Anything
In this paper, we propose the Matting Anything Model (MAM), an efficient and versatile framework for estimating the alpha matte of any instance in an image with flexible and interactive visual or linguistic user prompt guidance.
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches.
Zero-shot Generative Model Adaptation via Image-specific Prompt Learning
Recently, CLIP-guided image synthesis has shown appealing performance on adapting a pre-trained source-domain generator to an unseen target domain.
Automatic High Resolution Wire Segmentation and Removal
In this paper, we present an automatic wire clean-up system that eases the process of wire segmentation and removal/inpainting to within a few seconds.
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models
The unlearning problem of deep learning models, once primarily an academic concern, has become a prevalent issue in the industry.
PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor
We propose PAIR Diffusion, a generic framework that can enable a diffusion model to control the structure and appearance properties of each object in the image.
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
Recent text-to-video generation approaches rely on computationally heavy training and require large-scale video datasets.
Graph Transformer GANs for Graph-Constrained House Generation
We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices
In this paper we present a simple image inpainting baseline, Mobile Inpainting GAN (MI-GAN), which is approximately one order of magnitude computationally cheaper and smaller than existing state-of-the-art inpainting models, and can be efficiently deployed on mobile devices.
Specialist Diffusion: Plug-and-Play Sample-Efficient Fine-Tuning of Text-to-Image Diffusion Models To Learn Any Unseen Style
Diffusion models have demonstrated impressive capability of text-conditioned image synthesis, and broader application horizons are emerging by personalizing those pretrained diffusion models toward generating some specialized target object or style.
Mask Matching Transformer for Few-Shot Segmentation
Typical methods follow the paradigm to firstly learn prototypical features from support images and then match query features in pixel-level to obtain segmentation results.
Boosted Dynamic Neural Networks
To optimize the model, these prediction heads together with the network backbone are trained on every batch of training data.
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
In this work, we expand the existing single-flow diffusion pipeline into a multi-task multimodal network, dubbed Versatile Diffusion (VD), that handles multiple flows of text-to-image, image-to-text, and variations in one unified model.
OneFormer: One Transformer to Rule Universal Image Segmentation
However, such panoptic architectures do not truly unify image segmentation because they need to be trained individually on the semantic, instance, or panoptic segmentation to achieve the best performance.
Ranked #1 on
Panoptic Segmentation
on COCO minival
StyleNAT: Giving Each Head a New Perspective
Image generation has been a long sought-after but challenging task, and performing the generation task in an efficient manner is similarly difficult.
Ranked #2 on
Image Generation
on FFHQ 256 x 256
Image Completion with Heterogeneously Filtered Spectral Hints
We also prove the effectiveness of our design via ablation studies, from which one may notice that the aforementioned challenges, i. e. pattern unawareness, blurry textures, and structure distortion, can be noticeably resolved.
Ranked #1 on
Image Inpainting
on FFHQ 512 x 512
Dilated Neighborhood Attention Transformer
These models typically employ localized attention mechanisms, such as the sliding-window Neighborhood Attention (NA) or Swin Transformer's Shifted Window Self Attention.
Ranked #4 on
Panoptic Segmentation
on COCO minival
AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition
Recent research has revealed that reducing the temporal and spatial redundancy are both effective approaches towards efficient video recognition, e. g., allocating the majority of computation to a task-relevant subset of frames or the most valuable image regions of each frame.
VMFormer: End-to-End Video Matting with Transformer
In this paper, we propose VMFormer: a transformer-based end-to-end method for video matting.
Keys to Better Image Inpainting: Structure and Texture Go Hand in Hand
We claim that the performance of inpainting algorithms can be better judged by the generated structures and textures.
Point-to-Box Network for Accurate Object Detection via Single Point Supervision
However, the performance gap between point supervised object detection (PSOD) and bounding box supervised detection remains large.
VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
The learned implicit neural representation can be decoded to videos of arbitrary spatial resolution and frame rate.
Space-time Video Super-resolution
Video Frame Interpolation
+1
DiSparse: Disentangled Sparsification for Multitask Model Compression
We analyzed the pruning masks generated with DiSparse and observed strikingly similar sparse network architecture identified by each task even before the training starts.
Towards Layer-wise Image Vectorization
Image rasterization is a mature technique in computer graphics, while image vectorization, the reverse path of rasterization, remains a major challenge.
Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN
To study the motion features in the latent space of StyleGAN, in this paper, we hypothesize and demonstrate that a series of meaningful, natural, and versatile small, local movements (referred to as "micromotion", such as expression, head movement, and aging effect) can be represented in low-rank spaces extracted from the latent space of a conventionally pre-trained StyleGAN-v2 model for face generation, with the guidance of proper "anchors" in the form of either short text or video clips.
Neighborhood Attention Transformer
We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision.
Ranked #120 on
Semantic Segmentation
on ADE20K
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image
Despite the rapid development of Neural Radiance Field (NeRF), the necessity of dense covers largely prohibits its wider applications.
Object Localization under Single Coarse Point Supervision
In this study, we propose a POL method using coarse point annotations, relaxing the supervision signals from accurate key points to freely spotted points.
AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition
Recent works have shown that the computational efficiency of video recognition can be significantly improved by reducing the spatial redundancy.
SeMask: Semantically Masked Transformers for Semantic Segmentation
To achieve this, we propose SeMask, a simple and effective framework that incorporates semantic information into the encoder with the help of a semantic attention operation.
Ranked #10 on
Semantic Segmentation
on Cityscapes val
More Control for Free! Image Synthesis with Semantic Diffusion Guidance
We investigate fine-grained, continuous control of this model class, and introduce a novel unified framework for semantic diffusion guidance, which allows either language or image guidance, or both.
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Efficient video architecture is the key to deploying video recognition systems on devices with limited computing resources.
Global and Local Alignment Networks for Unpaired Image-to-Image Translation
The global alignment network aims to transfer the input image from the source domain to the target domain.
Feudal Reinforcement Learning by Reading Manuals
The manager agent is a multi-hop plan generator dealing with high-level abstract information and generating a series of sub-goals in a backward manner.
ConvMLP: Hierarchical Convolutional MLPs for Vision
MLP-based architectures, which consist of a sequence of consecutive multi-layer perceptron blocks, have recently been found to reach comparable results to convolutional and transformer-based methods.
Ranked #7 on
Image Classification
on Flowers-102
(using extra training data)
Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS), with high performance, low cost, and in-depth interpretation.
MSN: Efficient Online Mask Selection Network for Video Instance Segmentation
Our method improves the masks from segmentation and propagation branches in an online manner using the Mask Selection Network (MSN) hence limiting the noise accumulation during mask tracking.
Ranked #28 on
Video Instance Segmentation
on YouTube-VIS validation
RSCA: Real-time Segmentation-based Context-Aware Scene Text Detection
Segmentation-based scene text detection methods have been widely adopted for arbitrary-shaped text detection recently, since they make accurate pixel-level predictions on curved text instances and can facilitate real-time inference without time-consuming processing on anchors.
Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization
Post-training quantization methods use a set of calibration data to compute quantization ranges for network parameters and activations.
Pseudo-IoU: Improving Label Assignment in Anchor-Free Object Detection
Current anchor-free object detectors are quite simple and effective yet lack accurate label assignment methods, which limits their potential in competing with classic anchor-based models that are supported by well-designed assignment methods based on the Intersection-over-Union~(IoU) metric.
Escaping the Big Data Paradigm with Compact Transformers
Our models are flexible in terms of model size, and can have as little as 0. 28M parameters while achieving competitive results.
Ranked #1 on
Image Classification
on Flowers-102
(using extra training data)
Fine-Grained Image Classification
Superpixel Image Classification
Learning to Track Instances without Video Annotations
Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation, and 2) the complexity of two-stage approaches.
UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions in which we deeply integrated spatial coordinates and periodic encoding with the implicit neural representation.
Study Group Learning: Improving Retinal Vessel Segmentation Trained with Noisy Labels
Retinal vessel segmentation from retinal images is an essential task for developing the computer-aided diagnosis system for retinal diseases.
Ranked #1 on
Retinal Vessel Segmentation
on CHASE_DB1
Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
To better exploit the value of both pre-trained weights and unlabeled target examples, we introduce adaptive consistency regularization that consists of two complementary components: Adaptive Knowledge Consistency (AKC) on the examples between the source and target model, and Adaptive Representation Consistency (ARC) on the target model between labeled and unlabeled examples.
A Multi-Mode Modulator for Multi-Domain Few-Shot Classification
Most existing few-shot classification methods only consider generalization on one dataset (i. e., single-domain), failing to transfer across various seen and unseen domains.
Clique: Spatiotemporal Object Re-identification at the City Scale
Object re-identification (ReID) is a key application of city-scale cameras.
CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation
Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video.
Ranked #43 on
Video Instance Segmentation
on YouTube-VIS validation
Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
We also introduce Text Refinement Network (TexRNet), a novel text segmentation approach that adapts to the unique properties of text, e. g. non-convex boundary, diverse texture, etc., which often impose burdens on traditional segmentation models.
Interpretable Visual Reasoning via Induced Symbolic Space
Finally, we achieve a higher level of interpretability by imposing OCCAM on the objects represented in the induced symbolic concept space.
Ranked #3 on
Visual Question Answering (VQA)
on CLEVR
Human-Object Interaction Detection:A Quick Survey and Examination of Methods
In order to provide insight to future researchers, we perform an individualized study that examines the performance of each component of a multi-stream convolutional neural network architecture for human-object interaction detection.
Deep Learning for 3D Point Cloud Understanding: A Survey
The development of practical applications, such as autonomous driving and robotics, has brought increasing attention to 3D point cloud understanding.
The 1st Tiny Object Detection Challenge:Methods and Results
The 1st Tiny Object Detection (TOD) Challenge aims to encourage research in developing novel and accurate methods for tiny object detection in images which have wide views, with a current focus on tiny person detection.
High-Resolution Deep Image Matting
Image matting is a key technique for image and video editing and composition.
Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation
In this paper, we propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation.
Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining
By combining the new CS-NL prior with local and in-scale non-local priors in a powerful recurrent fusion cell, we can find more cross-scale feature correlations within a single low-resolution (LR) image.
Ranked #10 on
Image Super-Resolution
on Manga109 - 3x upscaling
Deep Learning-Based Automated Image Segmentation for Concrete Petrographic Analysis
The standard petrography test method for measuring air voids in concrete (ASTM C457) requires a meticulous and long examination of sample phase composition under a stereomicroscope.
Pyramid Attention Networks for Image Restoration
Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales.
Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation
A key challenge of this task is how to alleviate the data distribution discrepancy between the source and target domains, i. e. reducing domain shift.
Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation
We consider the problem of unsupervised domain adaptation for semantic segmentation by easing the domain shift between the source domain (synthetic data) and the target domain (real data) in this work.
Ranked #8 on
Semantic Segmentation
on DensePASS
AlignSeg: Feature-Aligned Segmentation Networks
Aggregating features in terms of different convolutional blocks or contextual embeddings has been proven to be an effective way to strengthen feature representations for semantic segmentation.
CCNet: Criss-Cross Attention for Semantic Segmentation
Compared with the non-local block, the proposed recurrent criss-cross attention module requires 11x less GPU memory usage.
Ranked #7 on
Semantic Segmentation
on FoodSeg103
(using extra training data)
A Simple Non-i.i.d. Sampling Approach for Efficient Training and Better Generalization
While training on samples drawn from independent and identical distribution has been a de facto paradigm for optimizing image classification networks, humans learn new concepts in an easy-to-hard manner and on the selected examples progressively.
Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection
In particular, DCR places a separate classification network in parallel with the localization network (base detector).
