Search Results for author:
Found 38 papers, 24 papers with code
Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
In response to this observation, we present a novel diffusion transformer framework incorporating an additional set of mediator tokens to engage with queries and keys separately.
Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual Visual Text Rendering
With the combination of these techniques, we deliver a powerful customized multilingual text encoder, Glyph-ByT5-v2, and a strong aesthetic graphic generation model, Glyph-SDXL-v2, that can support accurate spelling in 10 different languages.
FontStudio: Shape-Adaptive Diffusion Model for Coherent and Consistent Font Effect Generation
This task essentially requires generating coherent and consistent visual content within the confines of a font-shaped canvas, as opposed to a traditional rectangular canvas.
Step-aware Preference Optimization: Aligning Preference with Denoising Performance at Each Step
To address this limitation, we propose Step-aware Preference Optimization (SPO), a novel post-training approach that independently evaluates and adjusts the denoising performance at each step, using a step-aware preference model and a step-wise resampler to ensure accurate step-aware supervision.
DesignEdit: Multi-Layered Latent Decomposition and Fusion for Unified & Accurate Image Editing
Second, we propose an instruction-guided latent fusion that pastes the multi-layered latent representations onto a canvas latent.
Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Rendering
Visual text rendering poses a fundamental challenge for contemporary text-to-image generation models, with the core problem lying in text encoder deficiencies.
Customize-It-3D: High-Quality 3D Creation from A Single Image Using Subject-Specific Knowledge Prior
In this paper, we present a novel two-stage approach that fully utilizes the information provided by the reference image to establish a customized knowledge prior for image-to-3D generation.
ART$\boldsymbol{\cdot}$V: Auto-Regressive Text-to-Video Generation with Diffusion Models
Second, it preserves the high-fidelity generation ability of the pre-trained image diffusion models by making only minimal network modifications.
MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation
We present MicroCinema, a straightforward yet effective framework for high-quality and coherent text-to-video generation.
COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design
Our COLE system comprises multiple fine-tuned Large Language Models (LLMs), Large Multimodal Models (LMMs), and Diffusion Models (DMs), each specifically tailored for design-aware layer-wise captioning, layout planning, reasoning, and the task of generating images and text.
CCEdit: Creative and Controllable Video Editing via Diffusion Models
The versatility of our framework is demonstrated through a diverse range of choices in both structure representations and personalized T2I models, as well as the option to provide the edited key frame.
Mask-Attention-Free Transformer for 3D Instance Segmentation
Therefore, we abandon the mask attention design and resort to an auxiliary center regression task instead.
Exploring Predicate Visual Context in Detecting Human-Object Interactions
Recently, the DETR framework has emerged as the dominant approach for human--object interaction (HOI) research.
Ranked #2 on
Human-Object Interaction Detection
on HICO-DET
V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection
We introduce a highly performant 3D object detector for point clouds using the DETR framework.
Ranked #2 on
3D Object Detection
on ScanNetV2
Mask Frozen-DETR: High Quality Instance Segmentation with One GPU
In this paper, we aim to study how to build a strong instance segmenter with minimal training time and GPUs, as opposed to the majority of current approaches that pursue more accurate instance segmenter by building more advanced frameworks at the cost of longer training time and higher GPU requirements.
Ranked #3 on
Instance Segmentation
on COCO minival
(using extra training data)
DETR Doesn't Need Multi-Scale or Locality Design
This paper presents an improved DETR detector that maintains a "plain" nature: using a single-scale feature map and global cross-attention calculations without specific locality constraints, in contrast to previous leading DETR-based detectors that reintroduce architectural inductive biases of multi-scale and locality into the decoder.
Revisiting DETR Pre-training for Object Detection
Motivated by the remarkable achievements of DETR-based approaches on COCO object detection and segmentation benchmarks, recent endeavors have been directed towards elevating their performance through self-supervised pre-training of Transformers while preserving a frozen backbone.
LISA: Reasoning Segmentation via Large Language Model
In this work, we propose a new segmentation task -- reasoning segmentation.
Space Engage: Collaborative Space Supervision for Contrastive-based Semi-Supervised Semantic Segmentation
To improve the robustness of representations, powerful methods introduce a pixel-wise contrastive learning approach in latent space (i. e., representation space) that aggregates the representations to their prototypes in a fully supervised manner.
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
GlyphControl: Glyph Conditional Control for Visual Text Generation
We evaluate the effectiveness of our approach by measuring OCR-based metrics, CLIP score, and FID of the generated visual text.
DETR Does Not Need Multi-Scale or Locality Design
This paper presents an improved DETR detector that maintains a "plain" nature: using a single-scale feature map and global cross-attention calculations without specific locality constraints, in contrast to previous leading DETR-based detectors that reintroduce architectural inductive biases of multi-scale and locality into the decoder.
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Image cropping has progressed tremendously under the data-driven paradigm.
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning
Vision transformers have recently achieved competitive results across various vision tasks but still suffer from heavy computation costs when processing a large number of tokens.
DETRs with Hybrid Matching
One-to-one set matching is a key design for DETR to establish its end-to-end capability, so that object detection does not require a hand-crafted NMS (non-maximum suppression) to remove duplicate detections.
Region Rebalance for Long-Tailed Semantic Segmentation
In this paper, we study the problem of class imbalance in semantic segmentation.
Ranked #21 on
Semantic Segmentation
on ADE20K
RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation
Given an input image or video, our framework first conducts multi-label classification over the complete label, then sorts the complete label and selects a small subset according to their class confidence scores.
HRFormer: High-Resolution Vision Transformer for Dense Predict
We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and has high memory and computational cost.
HRFormer: High-Resolution Transformer for Dense Prediction
We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and has high memory and computational cost.
Ranked #3 on
Pose Estimation
on AIC
Conditional DETR for Fast Training Convergence
Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention.
Ranked #108 on
Object Detection
on COCO minival
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
Our approach imposes the consistency on two segmentation networks perturbed with different initialization for the same input image.
Ranked #2 on
Semi-Supervised Semantic Segmentation
on WoodScape
SegFix: Model-Agnostic Boundary Refinement for Segmentation
We present a model-agnostic post-processing scheme to improve the boundary quality for the segmentation result that is generated by any existing segmentation model.
Beyond Human Parts: Dual Part-Aligned Representations for Person Re-Identification
On the other hand, there still exist many useful contextual cues that do not fall into the scope of predefined human parts or attributes.
Ranked #59 on
Person Re-Identification
on DukeMTMC-reID
Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
We empirically demonstrate that the proposed approach achieves competitive performance on various challenging semantic segmentation benchmarks: Cityscapes, ADE20K, LIP, PASCAL-Context, and COCO-Stuff.
Ranked #5 on
Semantic Segmentation
on LIP val
Interlaced Sparse Self-Attention for Semantic Segmentation
There are two successive attention modules each estimating a sparse affinity matrix.
OCNet: Object Context Network for Scene Parsing
To capture richer context information, we further combine our interlaced sparse self-attention scheme with the conventional multi-scale context schemes including pyramid pooling~\citep{zhao2017pyramid} and atrous spatial pyramid pooling~\citep{chen2018deeplab}.
Ranked #9 on
Semantic Segmentation
on Trans10K
Feature Incay for Representation Regularization
Thus, we propose feature incay to also regularize representation learning, which favors feature vectors with large norm when the samples can be correctly classified.
Hard-Aware Deeply Cascaded Embedding
This motivates us to ensemble a set of models with different complexities in cascaded manner and mine hard examples adaptively, a sample is judged by a series of models with increasing complexities and only updates models that consider the sample as a hard case.
Ranked #14 on
Image Retrieval
on SOP
