Search Results for author:
Found 26 papers, 17 papers with code
FastSR-NeRF: Improving NeRF Efficiency on Consumer Devices with A Simple Super-Resolution Pipeline
Compared to existing NeRF+SR methods, our pipeline mitigates the SR computing overhead and can be trained up to 23x faster, making it feasible to run on consumer devices such as the Apple MacBook.
Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and Applications
While these models can achieve high-quality lip articulation for speakers in the training set, they are unable to capture the full and diverse distribution of 3D facial motions that accompany speech in the real world.
HUGS: Human Gaussian Splats
We achieve state-of-the-art rendering quality with a rendering speed of 60 FPS while being ~100x faster to train over previous work.
Novel-View Acoustic Synthesis from 3D Reconstructed Rooms
Given audio recordings from 2-4 microphones and the 3D geometry and material of a scene containing multiple unknown sound sources, we estimate the sound anywhere in the scene.
Pointersect: Neural Rendering with Cloud-Ray Intersection
Specifically, we train a set transformer that, given a small number of local neighbor points along a light ray, provides the intersection point, the surface normal, and the material blending weights, which are used to render the outcome of this light ray.
FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction
Recent works on 3D reconstruction from posed images have demonstrated that direct inference of scene-level 3D geometry without test-time optimization is feasible using deep neural networks, showing remarkable promise and high efficiency.
FaceLit: Neural 3D Relightable Faces
We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation.
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization
To this end, we introduce a novel token mixing operator, RepMixer, a building block of FastViT, that uses structural reparameterization to lower the memory access cost by removing skip-connections in the network.
Naturalistic Head Motion Generation from Speech
Synthesizing natural head motion to accompany speech for an embodied conversational agent is necessary for providing a rich interactive experience.
SPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks
In this paper, we perform an empirical evaluation on methods for sharing parameters in isotropic networks (SPIN).
MobileOne: An Improved One millisecond Mobile Backbone
Furthermore, we show that our model generalizes to multiple tasks - image classification, object detection, and semantic segmentation with significant improvements in latency and accuracy as compared to existing efficient architectures when deployed on a mobile device.
Ranked #583 on
Image Classification
on ImageNet
NeuMan: Neural Human Radiance Field from a Single Video
Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences.
Token Pooling in Vision Transformers
Token Pooling is a simple and effective operator that can benefit many architectures.
LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time
Our models require no retraining, thus our subspace of models can be deployed entirely on-device to allow adaptive network compression at inference time.
MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias
Images generated using MorphGAN conserve the identity of the person in the original image, and the provided control over head pose and facial expression allows test sets to be created to identify robustness issues of a facial recognition deep network with respect to pose and expression.
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
To create our dataset, we leverage a large repository of synthetic scenes created by professional artists, and we generate 77, 400 images of 461 indoor scenes with detailed per-pixel labels and corresponding ground truth geometry.
GIF: Generative Interpretable Faces
Specifically, we condition StyleGAN2 on FLAME, a generative 3D face model.
Learning Multi-Human Optical Flow
Therefore, we develop a dataset of multi-human optical flow and train optical flow networks on this dataset.
Attacking Optical Flow
In this paper, we extend adversarial patch attacks to optical flow networks and show that such attacks can compromise their performance.
Learning to Dress 3D People in Generative Clothing
To our knowledge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses.
Capture, Learning, and Synthesis of 3D Speaking Styles
To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.
Unsupervised Learning of Multi-Frame Optical Flow with Occlusions
In this paper, we propose a framework for unsupervised learning of optical flow and occlusions over multiple frames.
Generating 3D faces using Convolutional Mesh Autoencoders
To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface.
Ranked #4 on
Face Alignment
on FaceScape
Learning Human Optical Flow
Given this, we devise an optical flow algorithm specifically for human motion and show that it is superior to generic flow methods.
Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation
We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical flow, and segmentation of a video into the static scene and moving regions.
Ranked #65 on
Monocular Depth Estimation
on KITTI Eigen split
Optical Flow Estimation using a Spatial Pyramid Network
We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning.
Ranked #8 on
Dense Pixel Correspondence Estimation
on HPatches
Dense Pixel Correspondence Estimation
Optical Flow Estimation
