Search Results for author:
Found 30 papers, 16 papers with code
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
A primary limitation is the substantial number of image tokens required for AR models, which constrains both training and inference efficiency, as well as image resolution.
Transfer between Modalities with MetaQueries
Unified multimodal models aim to integrate understanding (text output) and generation (pixel output), but aligning these different modalities within a single architecture often demands complex training recipes and careful data balancing.
MoCha: Towards Movie-Grade Talking Character Synthesis
We introduce Talking Characters, a more realistic task to generate talking character animations directly from speech and text.
An Egocentric Vision-Language Model based Portable Real-time Smart Assistant
We present Vinci, a vision-language system designed to provide real-time, comprehensive AI assistance on portable devices.
Bayesian inference for dynamic spatial quantile models with interactive effects
The proposed model captures the dynamic structure of panel data, high-dimensional cross-sectional dependence, and allows for heterogeneous regression coefficients.
Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts
This challenge arises due to the lack of a mechanism to link each concept with its specific reference image.
Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model
We introduce Vinci, a real-time embodied smart assistant built upon an egocentric vision-language model.
Movie Gen: A Cast of Media Foundation Models
Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation.
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames.
ControlRoom3D: Room Generation using Semantic Proxy Rooms
Central to our approach is a user-defined 3D semantic proxy room that outlines a rough room layout based on semantic bounding boxes and a textual description of the overall room style.
Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text.
A Close Look at Spatial Modeling: From Attention to Convolution
Vision Transformers have shown great promise recently for many vision tasks due to the insightful architecture design and attention mechanism.
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
To address this, we propose to finetune CLIP on a collection of masked image regions and their corresponding text descriptions.
Ranked #4 on
Semantic Segmentation
on Replica
S^2-Transformer for Mask-Aware Hyperspectral Image Reconstruction
Besides, CASSI entangles the spatial and spectral information into a 2D measurement, placing a barrier for information disentanglement and modeling.
Billion-user Customer Lifetime Value Prediction: An Industrial-scale Solution from Kuaishou
Customer Life Time Value (LTV) is the expected total revenue that a single user can bring to a business.
Dual Lottery Ticket Hypothesis
In this work, we regard the winning ticket from LTH as the subnetwork which is in trainable condition and its performance as our benchmark, then go from a complementary direction to articulate the Dual Lottery Ticket Hypothesis (DLTH): Randomly selected subnetworks from a randomly initialized dense network can be transformed into a trainable condition and achieve admirable performance compared with LTH -- random tickets in a given lottery pool can be transformed into winning tickets.
Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble
Current Sign Language Recognition (SLR) methods usually extract features via deep neural networks and suffer overfitting due to limited and noisy data.
MR Image Super-Resolution With Squeeze and Excitation Reasoning Attention Network
They also fail to sense the entire space of the input, which is critical for high-quality MR image SR. To address those problems, we propose squeeze and excitation reasoning attention networks (SERAN) for accurate MR image SR. We propose to squeeze attention from global spatial information of the input and obtain global descriptors.
Ranked #2 on
Image Super-Resolution
on IXI
Skeleton Aware Multi-modal Sign Language Recognition
Sign language is commonly used by deaf or speech impaired people to communicate but requires significant effort to master.
Ranked #2 on
Sign Language Recognition
on AUTSL
(using extra training data)
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts
To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.
Exploring Sub-Pseudo Labels for Learning from Weakly-Labeled Web Videos
To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.
Recent Developments on Factor Models and its Applications in Econometric Learning
This paper makes a selective survey on the recent development of the factor model and its application on statistical learnings.
Screencast Tutorial Video Understanding
It is very popular for both novice and experienced users to learn new skills, compared to other tutorial media such as text, because of the visual guidance and the ease of understanding.
Adversarial Feature Hallucination Networks for Few-Shot Learning
The recent flourish of deep learning in various tasks is largely accredited to the rich and accessible labeled data.
Attention Bridging Network for Knowledge Transfer
With weights sharing and domain adversary training, this knowledge can be successfully transferred by regularizing the network's response to the same category in the target domain.
Visual Semantic Reasoning for Image-Text Matching
It outperforms the current best method by 6. 8% relatively for image retrieval and 4. 8% relatively for caption retrieval on MS-COCO (Recall@1 using 1K test set).
Ranked #8 on
Image Retrieval
on Flickr30K 1K test
Residual Non-local Attention Networks for Image Restoration
To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts.
Support Neighbor Loss for Person Re-Identification
To ensure scalability and separability, a softmax-like function is formulated to push apart the positive and negative support sets.
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
To solve these problems, we propose the very deep residual channel attention networks (RCAN).
Ranked #21 on
Image Super-Resolution
on BSD100 - 4x upscaling
Tell Me Where to Look: Guided Attention Inference Network
Weakly supervised learning with only coarse labels can obtain visual explanations of deep neural network such as attention maps by back-propagating gradients.
