Search Results for author:
Found 42 papers, 16 papers with code
VideoComposer: Compositional Video Synthesis with Motion Controllability
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis.
Cones 2: Customizable Image Synthesis with Multiple Subjects
Synthesizing images with user-specified subjects has received growing attention due to its practical applications.
MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
To address these issues, we develop a Motion-augmented Long-short Contrastive Learning (MoLo) method that contains two crucial components, including a long-short contrastive objective and a motion autodecoder.
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
A diffusion probabilistic model (DPM), which constructs a forward diffusion process by gradually adding noise to data points and learns the reverse denoising process to generate new samples, has been shown to handle complex data distribution.
Ranked #4 on
Video Generation
on UCF-101
Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos
Video temporal grounding aims to pinpoint a video segment that matches the query description.
ViM: Vision Middleware for Unified Downstream Transferring
ViM consists of a zoo of lightweight plug-in modules, each of which is independently learned on a midstream dataset with a shared frozen backbone.
Cones: Concept Neurons in Diffusion Models for Customized Generation
Concatenating multiple clusters of concept neurons can vividly generate all related concepts in a single image.
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task.
Rethinking Efficient Tuning Methods from a Unified Perspective
The U-Tuning framework can simultaneously encompass existing methods and derive new approaches for parameter-efficient transfer learning, which prove to achieve on-par or better performances on CIFAR-100 and FGVC datasets when compared with existing PETL methods.
Composer: Creative and Controllable Image Synthesis with Composable Conditions
Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability.
UKnow: A Unified Knowledge Protocol for Common-Sense Reasoning and Vision-Language Pre-training
This work presents a unified knowledge protocol, called UKnow, which facilitates knowledge-based studies from the perspective of data.
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples, which tends to be more practical in real-world applications.
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
(1) Any image region can be linked to the latent space, even if the region is pre-selected before training and fixed for all instances.
LipFormer: High-Fidelity and Generalizable Talking Face Generation With a Pre-Learned Facial Codebook
Generating a talking face video from the input audio sequence is a practical yet challenging task.
RA-CLIP: Retrieval Augmented Contrastive Language-Image Pre-Training
This process can be considered as an open-book exam: with the reference set as a cheat sheet, the proposed method doesn't need to memorize all visual concepts in the training data.
Dimensionality-Varying Diffusion Process
Diffusion models, which learn to reverse a signal destruction process to generate new data, typically require the signal at each step to have the same dimension.
Neural Dependencies Emerging from Learning Massive Categories
Through investigating the properties of the problem solution, we confirm that neural dependency is guaranteed by a redundant logit covariance matrix, which condition is easily met given massive categories, and that neural dependency is highly sparse, implying that one category correlates to only a few others.
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough.
Improving GANs with A Dynamic Discriminator
Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator.
Rank Diminishing in Deep Neural Networks
By virtue of our numerical tools, we provide the first empirical analysis of the per-layer behavior of network rank in practical settings, i. e., ResNets, deep MLPs, and Transformers on ImageNet.
Principled Knowledge Extrapolation with GANs
Human can extrapolate well, generalize daily knowledge into unseen scenarios, raise and answer counterfactual questions.
Region-Based Semantic Factorization in GANs
Despite the rapid advancement of semantic discovery in the latent space of Generative Adversarial Networks (GANs), existing approaches either are limited to finding global attributes or rely on a number of segmentation masks to identify local attributes.
Low-Rank Subspaces in GANs
Concretely, given an arbitrary image and a region of interest (e. g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces.
On Noise Injection in Generative Adversarial Networks
Noise injection has been proved to be one of the key technique advances in generating high-fidelity images.
In-Domain GAN Inversion for Real Image Editing
A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code.
Perceptual Image Super-Resolution with Progressive Adversarial Network
Progressive growing enhances image resolution gradually, thereby preserving precision of recovered image.
Latent Variables on Spheres for Autoencoders in High Dimensions
Variational Auto-Encoder (VAE) has been widely applied as a fundamental generative model in machine learning.
LIA: Latently Invertible Autoencoder with Adversarial Learning
A two-stage stochasticity-free training scheme is designed to train LIA via adversarial learning, in the sense that the decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from an autoencoder by detaching the invertible network from LIA.
Latent Variables on Spheres for Sampling and Inference
Variational inference is a fundamental problem in Variational AutoEncoder (VAE).
Curriculum Learning for Deep Generative Models with Clustering
The experiments on cat and human-face data validate that our algorithm is able to learn the optimal generative models (e. g. ProGAN) with respect to specified quality metrics for noisy data.
Disentangled Inference for GANs with Latently Invertible Autoencoder
In this paper, we show that the entanglement of the latent space for the VAE/GAN framework poses the main challenge for encoder learning.
DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning
Based on these sub-images, a local exposure for each sub-image is automatically learned by virtue of policy network sequentially while the reward of learning is globally designed for striking a balance of overall exposures.
Few Shot Learning with Simplex
During testing, combined with the test sample and the points in the class, a new simplex is formed.
Sparse Coding and Dictionary Learning With Linear Dynamical Systems
To enhance the performance of LDSs, in this paper, we address the challenging issue of performing sparse coding on the space of LDSs, where both data and dictionary atoms are LDSs.
Network Representation Learning with Rich Text Information
Representation learning has shown its effectiveness in many tasks such as image classification and text mining.
A New Retraction for Accelerating the Riemannian Three-Factor Low-Rank Matrix Completion Algorithm
In the line search step, R3MC approximates the minimum point on the searching curve by minimizing on the line tangent to the curve.
Zeta Hull Pursuits: Learning Nonconvex Data Hulls
Selecting a small informative subset from a given dataset, also called column sampling, has drawn much attention in machine learning.
Errata: Distant Supervision for Relation Extraction with Matrix Completion
The essence of distantly supervised relation extraction is that it is an incomplete multi-label classification problem with sparse and noisy features.
Homophilic Clustering by Locally Asymmetric Geometry
Clustering is indispensable for data analysis in many scientific disciplines.
Graph Degree Linkage: Agglomerative Clustering on a Directed Graph
We explore the different roles of two fundamental concepts in graph theory, indegree and outdegree, in the context of clustering.
Ranked #1 on
Image Clustering
on Coil-20
(Accuracy metric)
Cyclizing Clusters via Zeta Function of a Graph
A mathematical tool, Zeta function of a graph, is introduced for the integration of all cycles, leading to a structural descriptor of the cluster in determinantal form.
