Search Results for author:
Found 36 papers, 12 papers with code
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames.
Efficient Quantization Strategies for Latent Diffusion Models
By treating the quantization discrepancy as relative noise and identifying sensitive part(s) of a model, we propose an efficient quantization approach encompassing both global and local strategies.
ControlRoom3D: Room Generation using Semantic Proxy Rooms
Central to our approach is a user-defined 3D semantic proxy room that outlines a rough room layout based on semantic bounding boxes and a textual description of the overall room style.
Cache Me if You Can: Accelerating Diffusion Models through Block Caching
However, one of the major drawbacks of diffusion models is that the image generation process is costly.
Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text.
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence.
DIME-FM: DIstilling Multimodal and Efficient Foundation Models
We transfer the knowledge from the pre-trained CLIP-ViTL/14 model to a ViT-B/32 model, with only 40M public images and 28. 4M unpaired public sentences.
3D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusion
We note that the novelty of our model lies in that we introduce contrastive learning during training the diffusion prior which enables the generation of the valid view-invariant latent code.
Pruning Compact ConvNets for Efficient Inference
The resulting family of pruned models can consistently obtain better performance than existing FBNetV3 models at the same level of computation, and thus provide state-of-the-art results when trading off between computational complexity and generalization performance on the ImageNet benchmark.
DIME-FM : DIstilling Multimodal and Efficient Foundation Models
In this paper, we introduce a new distillation mechanism (DIME-FM) that allows us to transfer the knowledge contained in large VLFMs to smaller, customized foundation models using a relatively small amount of inexpensive, unpaired images and sentences.
INGeo: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors
We present a method that accelerates reconstruction of 3D scenes and objects, aiming to enable instant reconstruction on edge devices such as mobile phones and AR/VR headsets.
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference
Vision Transformers (ViTs) have shown impressive performance but still require a high computation cost as compared to convolutional neural networks (CNNs), one reason is that ViTs' attention measures global similarities and thus has a quadratic complexity with the number of input tokens.
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
We hope that our work will stimulate research and lead to the development of a new generation of ML systems for XR use cases.
3D-Aware Encoding for Style-based Neural Radiance Fields
In the first stage, we introduce a base encoder that converts the input image to a latent code.
UmeTrack: Unified multi-view end-to-end hand tracking for VR
In this paper, we present a unified end-to-end differentiable framework for multi-view, multi-frame hand tracking that directly predicts 3D hand pose in world space.
Token Merging: Your ViT But Faster
Off-the-shelf, ToMe can 2x the throughput of state-of-the-art ViT-L @ 512 and ViT-H @ 518 models on images and 2. 2x the throughput of ViT-L on video with only a 0. 2-0. 3% accuracy drop in each case.
Ranked #13 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
To address this, we propose to finetune CLIP on a collection of masked image regions and their corresponding text descriptions.
Hydra Attention: Efficient Attention with Many Heads
While transformers have begun to dominate many tasks in vision, applying them to large images is still computationally difficult.
Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation
Traditional computer vision models are trained to predict a fixed set of predefined categories.
FBNetV5: Neural Architecture Search for Multiple Tasks in One Run
To tackle these challenges, we propose FBNetV5, a NAS framework that can search for neural architectures for a variety of vision tasks with much reduced computational cost and human effort.
Ranked #7 on
Neural Architecture Search
on ImageNet
An Investigation on Hardware-Aware Vision Transformer Scaling
In particular, when transferred to PiT, our scaling strategies lead to a boosted ImageNet top-1 accuracy of from $74. 6\%$ to $76. 7\%$ ($\uparrow2. 1\%$) under the same 0. 7G FLOPs; and when transferred to the COCO object detection task, the average precision is boosted by $\uparrow0. 7\%$ under a similar throughput on a V100 GPU.
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation
Our model transfers knowledge from pretrained image and sentence encoders and achieves strong performance with only 3M image text pairs, 133x smaller than CLIP.
Unbiased Teacher for Semi-Supervised Object Detection
To address this, we introduce Unbiased Teacher, a simple yet effective approach that jointly trains a student and a gradually progressing teacher in a mutually-beneficial manner.
Visual Transformers: Where Do Transformers Really Belong in Vision Models?
A recent trend in computer vision is to replace convolutions with transformers.
Low Bandwidth Video-Chat Compression using Deep Generative Models
To unlock video chat for hundreds of millions of people hindered by poor connectivity or unaffordable data costs, we propose to authentically reconstruct faces on the receiver's device using facial landmarks extracted at the sender's side and transmitted over the network.
FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge
More efficient variants of FBWave can achieve up to 109x fewer MACs while still delivering acceptable audio quality.
FP-NAS: Fast Probabilistic Neural Architecture Search
Furthermore, to search fast in the multi-variate space, we propose a coarse-to-fine strategy by using a factorized distribution at the beginning which can reduce the number of architecture parameters by over an order of magnitude.
One Shot 3D Photography
3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual Reality devices, where viewing it also includes stereo.
Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild
We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild.
Visual Transformers: Token-based Image Representation and Processing for Computer Vision
In this work, we challenge this paradigm by (a) representing images as semantic visual tokens and (b) running transformers to densely model token relationships.
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
To address this, we present Neural Architecture-Recipe Search (NARS) to search both (a) architectures and (b) their corresponding training recipes, simultaneously.
Ranked #5 on
Neural Architecture Search
on ImageNet
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands.
Ranked #68 on
Neural Architecture Search
on ImageNet
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
We formulate platform-aware NN architecture search in an optimization framework and propose a novel algorithm to search for optimal architectures aided by efficient accuracy and resource (latency and/or energy) predictors.
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Due to this, previous neural architecture search (NAS) methods are computationally expensive.
Ranked #893 on
Image Classification
on ImageNet
Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices with limited computational and memory resources.
