Search Results for author:
Found 37 papers, 18 papers with code
Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text.
NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detection
We present NeRF-Det, a novel method for indoor 3D detection with posed RGB images as input.
Pruning Compact ConvNets for Efficient Inference
The resulting family of pruned models can consistently obtain better performance than existing FBNetV3 models at the same level of computation, and thus provide state-of-the-art results when trading off between computational complexity and generalization performance on the ImageNet benchmark.
INGeo: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors
We present a method that accelerates reconstruction of 3D scenes and objects, aiming to enable instant reconstruction on edge devices such as mobile phones and AR/VR headsets.
A Practical Stereo Depth System for Smart Glasses
We present the design of a productionized end-to-end stereo depth sensing system that does pre-processing, online stereo rectification, and stereo depth estimation with a fallback to monocular depth estimation when rectification is unreliable.
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference
Vision Transformers (ViTs) have shown impressive performance but still require a high computation cost as compared to convolutional neural networks (CNNs), one reason is that ViTs' attention measures global similarities and thus has a quadratic complexity with the number of input tokens.
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
We hope that our work will stimulate research and lead to the development of a new generation of ML systems for XR use cases.
3D-Aware Encoding for Style-based Neural Radiance Fields
In the first stage, we introduce a base encoder that converts the input image to a latent code.
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
To address this, we propose to finetune CLIP on a collection of masked image regions and their corresponding text descriptions.
Open-Set Semi-Supervised Object Detection
To address this problem, we consider online and offline OOD detection modules, which are integrated with SSOD methods.
Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation
Traditional computer vision models are trained to predict a fixed set of predefined categories.
Cross-Domain Adaptive Teacher for Object Detection
To mitigate this problem, we propose a teacher-student framework named Adaptive Teacher (AT) which leverages domain adversarial learning and weak-strong data augmentation to address the domain gap.
FBNetV5: Neural Architecture Search for Multiple Tasks in One Run
To tackle these challenges, we propose FBNetV5, a NAS framework that can search for neural architectures for a variety of vision tasks with much reduced computational cost and human effort.
Ranked #7 on
Neural Architecture Search
on ImageNet
Adaptive Unbiased Teacher for Cross-Domain Object Detection
This enables the student model to capture domain-invariant features.
An Investigation on Hardware-Aware Vision Transformer Scaling
In particular, when transferred to PiT, our scaling strategies lead to a boosted ImageNet top-1 accuracy of from $74. 6\%$ to $76. 7\%$ ($\uparrow2. 1\%$) under the same 0. 7G FLOPs; and when transferred to the COCO object detection task, the average precision is boosted by $\uparrow0. 7\%$ under a similar throughput on a V100 GPU.
Tackling the Ill-Posedness of Super-Resolution Through Adaptive Target Generation
By the one-to-many nature of the super-resolution (SR) problem, a single low-resolution (LR) image can be mapped to many high-resolution (HR) images.
Ranked #3 on
Blind Super-Resolution
on DIV2KRK - 4x upscaling
Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Models
We discover that we can indeed use the same architecture and pretrained weights of a neural net model to understand both images and point-clouds.
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation
Our model transfers knowledge from pretrained image and sentence encoders and achieves strong performance with only 3M image text pairs, 133x smaller than CLIP.
Unbiased Teacher for Semi-Supervised Object Detection
To address this, we introduce Unbiased Teacher, a simple yet effective approach that jointly trains a student and a gradually progressing teacher in a mutually-beneficial manner.
Visual Transformers: Where Do Transformers Really Belong in Vision Models?
A recent trend in computer vision is to replace convolutions with transformers.
FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge
More efficient variants of FBWave can achieve up to 109x fewer MACs while still delivering acceptable audio quality.
One Shot 3D Photography
3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual Reality devices, where viewing it also includes stereo.
Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild
We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild.
Visual Transformers: Token-based Image Representation and Processing for Computer Vision
In this work, we challenge this paradigm by (a) representing images as semantic visual tokens and (b) running transformers to densely model token relationships.
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
To address this, we present Neural Architecture-Recipe Search (NARS) to search both (a) architectures and (b) their corresponding training recipes, simultaneously.
Ranked #5 on
Neural Architecture Search
on ImageNet
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands.
Ranked #68 on
Neural Architecture Search
on ImageNet
Deep Space-Time Video Upsampling Networks
Video super-resolution (VSR) and frame interpolation (FI) are traditional computer vision problems, and the performance have been improving by incorporating deep learning recently.
SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation
Using standard convolutions to process such LiDAR images is problematic, as convolution filters pick up local features that are only active in specific regions in the image.
Ranked #20 on
3D Semantic Segmentation
on SemanticKITTI
Learning the Loss Functions in a Discriminative Space for Video Restoration
Our framework is similar to GANs in that we iteratively train two networks - a generator and a loss network.
Efficient Segmentation: Learning Downsampling Near Semantic Boundaries
Many automated processes such as auto-piloting rely on a good semantic segmentation as a critical component.
Learning to Generate Grounded Visual Captions without Localization Supervision
When automatically generating a sentence description for an image or video, it often remains unclear how well the generated caption is grounded, that is whether the model uses the correct image regions to output particular words, or if the model is hallucinating based on priors in the dataset and/or the language model.
Precision Highway for Ultra Low-Precision Quantization
We also report that the proposed method significantly outperforms the existing method in the 2-bit quantization of an LSTM for language modeling.
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
We formulate platform-aware NN architecture search in an optimization framework and propose a novel algorithm to search for optimal architectures aided by efficient accuracy and resource (latency and/or energy) predictors.
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Due to this, previous neural architecture search (NAS) methods are computationally expensive.
Ranked #847 on
Image Classification
on ImageNet
Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices with limited computational and memory resources.
Value-aware Quantization for Training and Inference of Neural Networks
We propose a novel value-aware quantization which applies aggressively reduced precision to the majority of data while separately handling a small amount of large data in high precision, which reduces total quantization errors under very low precision.
DSD: Dense-Sparse-Dense Training for Deep Neural Networks
We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance.
