Search Results for author:
Found 60 papers, 26 papers with code
An Empirical Study of GPT-4o Image Generation Capabilities
The landscape of image generation has rapidly evolved, from early GAN-based approaches to diffusion models and, most recently, to unified generative architectures that seek to bridge understanding and generation tasks.
POSTA: A Go-to Framework for Customized Artistic Poster Generation
To train our models, we develop the PosterArt dataset, comprising high-quality artistic posters annotated with layout, typography, and pixel-level stylized text segmentation.
MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesis
To address these challenges, we propose MagicDistillation, a novel framework designed to reduce inference overhead while ensuring the generalization of VDMs for portrait video synthesis.
CoRe^2: Collect, Reflect and Refine to Generate Better and Faster
Making text-to-image (T2I) generative model sample both fast and well represents a promising research direction.
MagicInfinite: Generating Infinite Talking Videos with Your Words and Voice
We present MagicInfinite, a novel diffusion Transformer (DiT) framework that overcomes traditional portrait animation limitations, delivering high-fidelity results across diverse character types-realistic humans, full-body figures, and stylized anime characters.
Magic 1-For-1: Generating One Minute Video Clips within One Minute
The key idea is simple: factorize the text-to-video generation task into two separate easier tasks for diffusion step distillation, namely text-to-image generation and image-to-video generation.
ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioning
Using DEGC as a building block, we propose a novel CNN-GNN architecture, ClusterViG, for CV tasks.
Adversarial Training in Low-Label Regimes with Margin-Based Interpolation
In this paper, we introduce a novel semi-supervised adversarial training approach that enhances both robustness and natural accuracy by generating effective adversarial examples.
Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer
Text-to-image diffusion models (DMs) develop at an unprecedented pace, supported by thorough theoretical exploration and empirical analysis.
Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
We present Meissonic, which elevates non-autoregressive masked image modeling (MIM) text-to-image to a level comparable with state-of-the-art diffusion models like SDXL.
Unsupervised Low-light Image Enhancement with Lookup Tables and Diffusion Priors
As diffusion models are sensitive to noise, diffusion priors are introduced to achieve high-performance noise suppression.
Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint
Recent advancements in adverse weather restoration have shown potential, yet the unpredictable and varied combinations of weather degradations in the real world pose significant challenges.
Cross-conditioned Diffusion Model for Medical Image to Image Translation
First, we propose a Modality-specific Representation Model (MRM) to model the distribution of target modalities.
Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
Language models have demonstrated remarkable performance in solving reasoning tasks; however, even the strongest models still occasionally make reasoning mistakes.
Timeline and Boundary Guided Diffusion Network for Video Shadow Detection
Moreover, we are the first to introduce the Diffusion model for VSD in which we explore a Space-Time Encoded Embedding (STEE) to inject the temporal guidance for Diffusion to conduct shadow detection.
RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models
RestoreAgent autonomously assesses the type and extent of degradation in input images and performs restoration through (1) determining the appropriate restoration tasks, (2) optimizing the task sequence, (3) selecting the most suitable models, and (4) executing the restoration.
AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement
Existing low-light image enhancement (LIE) methods have achieved noteworthy success in solving synthetic distortions, yet they often fall short in practical applications.
STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
We begin our exploration with a vanilla large language model, augmenting it with a vision encoder and an action codebase trained on our collected high-quality dataset STEVE-21K.
Parallel Cross Strip Attention Network for Single Image Dehazing
The objective of single image dehazing is to restore hazy images and produce clear, high-quality visuals.
MovieChat+: Question-aware Sparse Memory for Long Video Question Answering
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Ranked #4 on
Question Answering
on NExT-QA (Open-ended VideoQA)
NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results
This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results.
FACTUAL: A Novel Framework for Contrastive Learning Based Robust SAR Image Classification
FACTUAL consists of two components: (1) Differing from existing works, a novel perturbation scheme that incorporates realistic physical adversarial attacks (such as OTSA) to build a supervised adversarial pre-training network.
VersaT2I: Improving Text-to-Image Models with Versatile Reward
Recent text-to-image (T2I) models have benefited from large-scale and high-quality data, demonstrating impressive performance.
Uncertainty-Aware SAR ATR: Defending Against Adversarial Attacks via Bayesian Neural Networks
Adversarial attacks have demonstrated the vulnerability of Machine Learning (ML) image classifiers in Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) systems.
Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation
To assess organizational behavior, we design a series of navigation tasks in the Minecraft environment, which includes searching and exploring.
SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation
Our SegMamba, in contrast to Transformer-based methods, excels in whole volume feature modeling from a state space model standpoint, maintaining superior processing speed, even with volume features at a resolution of {$64\times 64\times 64$}.
Learning Diffusion Texture Priors for Image Restoration
When adopting diffusion models for image restoration the crucial challenge lies in how to preserve high-level image fidelity in the randomness diffusion process and generate accurate background structures and realistic texture details.
VQCNIR: Clearer Night Image Restoration with Vector-Quantized Codebook
In this paper, we propose Clearer Night Image Restoration with Vector-Quantized Codebook (VQCNIR) to achieve remarkable and consistent restoration outcomes on real-world and synthetic benchmarks.
Benchmarking Deep Learning Classifiers for SAR Automatic Target Recognition
Synthetic Aperture Radar SAR Automatic Target Recognition ATR is a key technique of remote-sensing image recognition which can be supported by deep neural networks The existing works of SAR ATR mostly focus on improving the accuracy of the target recognition while ignoring the systems performance in terms of speed and storage which is critical to real-world applications of SAR ATR For decision-makers aiming to identify a proper deep learning model to deploy in a SAR ATR system it is important to understand the performance of different candidate deep learning models and determine the best model accordingly This paper comprehensively benchmarks several advanced deep learning models for SAR ATR with multiple distinct SAR imagery datasets Specifically we train and test five SAR image classifiers based on Residual Neural Networks ResNet18 ResNet34 ResNet50 Graph Neural Network GNN and Vision Transformer for Small-Sized Datasets (SS-ViT) We select three datasets MSTAR GBSAR and SynthWakeSAR that offer heterogeneity We evaluate and compare the five classifiers concerning their classification accuracy runtime performance in terms of inference throughput and analytical performance in terms of number of parameters number of layers model size and number of operations Experimental results show that the GNN classifier outperforms with respect to throughput and latency However it is also shown that no clear model winner emerges from all of our chosen metrics and a one model rules all case is doubtful in the domain of SAR ATR
Realistic Scatterer Based Adversarial Attacks on SAR Image Classifiers
Instead, adversarial attacks should be able to be implemented by physical actions, for example, placing additional false objects as scatterers around the on-ground target to perturb the SAR image and fool the SAR ATR.
See and Think: Embodied Agent in Virtual Environment
This paper proposes STEVE, a comprehensive and visionary embodied agent in the Minecraft virtual environment.
Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
These priors are subsequently utilized by RGformer to guide the decomposition of image features into their respective reflectance and illumination components.
Integrating View Conditions for Image Synthesis
In the field of image processing, applying intricate semantic modifications within existing images remains an enduring challenge.
Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks
In the real world, image degradations caused by rain often exhibit a combination of rain streaks and raindrops, thereby increasing the challenges of recovering the underlying clean image.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Multiple-choice
Video-based Generative Performance Benchmarking (Consistency)
+11
MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Estimating 3D human poses only from a 2D human pose sequence is thoroughly explored in recent years.
NightHazeFormer: Single Nighttime Haze Removal Using Prior Query Transformer
Extensive experiments on several synthetic and real-world datasets demonstrate the superiority of our NightHazeFormer over state-of-the-art nighttime haze removal methods in terms of both visually and quantitatively.
Five A$^{+}$ Network: You Only Need 9K Parameters for Underwater Image Enhancement
In this work, we propose the Five A$^{+}$ Network (FA$^{+}$Net), a highly efficient and lightweight real-time underwater image enhancement network with only $\sim$ 9k parameters and $\sim$ 0. 01s processing time.
DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes
Varicolored haze caused by chromatic casts poses haze removal and depth estimation challenges.
RSFDM-Net: Real-time Spatial and Frequency Domains Modulation Network for Underwater Image Enhancement
Underwater images typically experience mixed degradations of brightness and structure caused by the absorption and scattering of light by suspended particles.
Adverse Weather Removal with Codebook Priors
Inspired by recent advancements in codebook and vector quantization (VQ) techniques, we present a novel Adverse Weather Removal network with Codebook Priors (AWRCP) to address the problem of unified adverse weather removal.
Dual-former: Hybrid Self-attention Transformer for Efficient Image Restoration
Recently, image restoration transformers have achieved comparable performance with previous state-of-the-art CNNs.
AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results
The homepage of this challenge is at https://github. com/RenYang-home/AIM22_CompressSR.
SnowFormer: Context Interaction Transformer with Scale-awareness for Single Image Desnowing
Due to various and complicated snow degradations, single image desnowing is a challenging image restoration task.
Ranked #2 on
Single Image Desnowing
on CSD
Towards Real-time High-Definition Image Snow Removal: Efficient Pyramid Network with Asymmetrical Encoder-decoder Architecture
In winter scenes, the degradation of images taken under snow can be pretty complex, where the spatial distribution of snowy degradation is varied from image to image.
MSP-Former: Multi-Scale Projection Transformer for Single Image Desnowing
Snow removal causes challenges due to its characteristic of complex degradations.
Prior-Guided One-shot Neural Architecture Search
In this paper, we present Prior-Guided One-shot NAS (PGONAS) to strengthen the ranking correlation of supernets.
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Towards Efficient Single Image Dehazing and Desnowing
A single expert network efficiently addresses specific degradation in nasty winter scenes relying on the compact architecture and three novel components.
Self-Calibrated Efficient Transformer for Lightweight Super-Resolution
Recently, deep learning has been successfully applied to the single-image super-resolution (SISR) with remarkable performance.
Underwater Light Field Retention : Neural Rendering for Underwater Imaging
To this end, we propose a neural rendering method for underwater imaging, dubbed UWNR (Underwater Neural Rendering).
Mutual Learning for Domain Adaptation: Self-distillation Image Dehazing Network with Sample-cycle
Specifically, we first devise two siamese networks: a teacher network in the synthetic domain and a student network in the real domain, and then optimize them in a mutual learning manner by leveraging EMA and joint loss.
Perceiving and Modeling Density is All You Need for Image Dehazing
However, due to the paradox caused by the variation of real captured haze and the fixed degradation parameters of the current networks, the generalization ability of recent dehazing methods on real-world hazy images is not ideal. To address the problem of modeling real-world haze degradation, we propose to solve this problem by perceiving and modeling density for uneven haze distribution.
Ranked #7 on
Image Dehazing
on Haze4k
Efficient Re-parameterization Residual Attention Network For Nonhomogeneous Image Dehazing
This paper proposes an end-to-end Efficient Re-parameterizationResidual Attention Network(ERRA-Net) to directly restore the nonhomogeneous hazy image.
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization
We study the asymmetric low-rank factorization problem: \[\min_{\mathbf{U} \in \mathbb{R}^{m \times d}, \mathbf{V} \in \mathbb{R}^{n \times d}} \frac{1}{2}\|\mathbf{U}\mathbf{V}^\top -\mathbf{\Sigma}\|_F^2\] where $\mathbf{\Sigma}$ is a given matrix of size $m \times n$ and rank $d$.
Aligning Videos in Space and Time
During training, given a pair of videos, we compute cycles that connect patches in a given frame in the first video by matching through frames in the second video.
DEED: A General Quantization Scheme for Communication Efficiency in Bits
In the infrequent communication setting, DEED combined with Federated averaging requires a smaller total number of bits than Federated Averaging.
Graph Star Net for Generalized Multi-Task Learning
In this work, we present graph star net (GraphStar), a novel and unified graph neural net architecture which utilizes message-passing relay and attention mechanism for multiple prediction tasks - node classification, graph classification and link prediction.
Ranked #1 on
Link Prediction
on Pubmed (biased evaluation)
Interpretable Intuitive Physics Model
In order to demonstrate that our system models these underlying physical properties, we train our model on collisions of different shapes (cube, cone, cylinder, spheres etc.)
