Search Results for author:
Found 34 papers, 17 papers with code
Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion
Our method uses raymap conditioning to both augment visual features with spatial information from different viewpoints, as well as to guide the generation of images and depth maps from novel views.
SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction
Sign language processing has traditionally relied on task-specific models, limiting the potential for transfer learning across tasks.
SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale
A persistent challenge in sign language video processing, including the task of sign language to written language translation, is how we learn representations of sign language in an effective and efficient way that can preserve the important attributes of these languages, while remaining invariant to irrelevant visual differences.
Alpha Invariance: On Inverse Scaling Between Distance and Volume Density in Neural Radiance Fields
Scale-ambiguity in 3D scene dimensions leads to magnitude-ambiguity of volumetric densities in neural radiance fields, i. e., the densities double when scene size is halved, and vice versa.
Generative Models: What Do They Know? Do They Know Things? Let's Find Out!
Generative models excel at mimicking real scenes, suggesting they might inherently encode important intrinsic scene properties.
HyperFields: Towards Zero-Shot Generation of NeRFs from Text
We introduce HyperFields, a method for generating text-conditioned Neural Radiance Fields (NeRFs) with a single forward pass and (optionally) some fine-tuning.
LoopDraw: a Loop-Based Autoregressive Model for Shape Synthesis and Editing
There is no settled universal 3D representation for geometry with many alternatives such as point clouds, meshes, implicit functions, and voxels to name a few.
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
We propose to apply chain rule on the learned gradients, and back-propagate the score of a diffusion model through the Jacobian of a differentiable renderer, which we instantiate to be a voxel radiance field.
Ranked #6 on
Text to 3D
on T$^3$Bench
Text-Free Learning of a Natural Language Interface for Pretrained Face Generators
We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis.
Depth Field Networks for Generalizable Multi-view Scene Representation
Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching.
Open-Domain Sign Language Translation Learned from Online Video
Existing work on sign language translation - that is, translation from sign language videos into sentences in a written language - has focused mainly on (1) data collected in a controlled environment or (2) data in a specific domain, which limits the applicability to real-world settings.
Ranked #2 on
Gloss-free Sign Language Translation
on OpenASL
Gloss-free Sign Language Translation
Sign Language Translation
+1
Adapting CLIP For Phrase Localization Without Further Training
Our method for phrase localization requires no human annotations or additional training.
Neural Neighbor Style Transfer
We propose Neural Neighbor Style Transfer (NNST), a pipeline that offers state-of-the-art quality, generalization, and competitive efficiency for artistic style transfer.
Searching for fingerspelled content in American Sign Language
This is an important task since significant content in sign language is often conveyed via fingerspelling, and to our knowledge the task has not been studied before.
Boosting Barely Robust Learners: A New Perspective on Adversarial Robustness
We present an oracle-efficient algorithm for boosting the adversarial robustness of barely robust learners.
Self-Supervised Camera Self-Calibration from Video
Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.
Fingerspelling Detection in American Sign Language
We propose a benchmark and a suite of evaluation metrics, some of which reflect the effect of detection on the downstream fingerspelling recognition task.
Full Surround Monodepth from Multiple Cameras
In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs.
Information-Theoretic Segmentation by Inpainting Error Maximization
We study image segmentation from an information-theoretic perspective, proposing a novel adversarial method that performs unsupervised segmentation by partitioning images into maximally independent sets.
Ranked #1 on
Unsupervised Image Segmentation
on Flowers
Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion
Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.
Controlling Length in Image Captioning
We develop and evaluate captioning models that allow control of caption length.
Pixel Consensus Voting for Panoptic Segmentation
The core of our approach, Pixel Consensus Voting, is a framework for instance segmentation based on the Generalized Hough transform.
Ranked #36 on
Panoptic Segmentation
on COCO test-dev
Space-Time-Aware Multi-Resolution Video Enhancement
We consider the problem of space-time super-resolution (ST-SR): increasing spatial resolution of video frames and simultaneously interpolating frames to increase the frame rate.
Detection and Description of Change in Visual Streams
This paper presents a framework for the analysis of changes in visual streams: ordered sequences of images, possibly separated by significant time gaps.
Fingerspelling recognition in the wild with iterative visual attention
In this paper we focus on recognition of fingerspelling sequences in American Sign Language (ASL) videos collected in the wild, mainly from YouTube and Deaf social media.
Style Transfer by Relaxed Optimal Transport and Self-Similarity
Style transfer algorithms strive to render the content of one image using the style of another.
Deep Back-Projection Networks for Single Image Super-resolution
Previous feed-forward architectures of recently proposed deep super-resolution networks learn the features of low-resolution inputs and the non-linear mapping from those to a high-resolution output.
Ranked #1 on
Image Super-Resolution
on BSDS100 - 8x upscaling
Recurrent Back-Projection Network for Video Super-Resolution
We proposed a novel architecture for the problem of video super-resolution.
American Sign Language fingerspelling recognition in the wild
As the first attempt at fingerspelling recognition in the wild, this work is intended to serve as a baseline for future work on sign language recognition in realistic conditions.
Task-Driven Super Resolution: Object Detection in Low-resolution Images
We consider how image super resolution (SR) can contribute to an object detection task in low-resolution images.
Deep Back-Projection Networks For Super-Resolution
The feed-forward architectures of recently proposed deep super-resolution networks learn representations of low-resolution inputs, and the non-linear mapping from those to high-resolution output.
Ranked #21 on
Video Super-Resolution
on Vid4 - 4x upscaling
Self-Supervised Relative Depth Learning for Urban Scene Understanding
As an agent moves through the world, the apparent motion of scene elements is (usually) inversely proportional to their depth.
Discriminative Metric Learning by Neighborhood Gerrymandering
We formulate the problem of metric learning for k nearest neighbor classification as a large margin structured prediction problem, with a latent variable representing the choice of neighbors and the task loss directly corresponding to classification error.
