Search Results for author:
Found 55 papers, 37 papers with code
Probing Visual Language Priors in VLMs
Despite recent advances in Vision-Language Models (VLMs), many still over-rely on visual language priors present in their training data rather than true visual reasoning.
Lightplane: Highly-Scalable Components for Neural 3D Fields
Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision.
Probing the 3D Awareness of Visual Foundation Models
Given that such models can classify, delineate, and localize objects in 2D, we ask whether they also represent their 3D structure?
View Selection for 3D Captioning via Diffusion Ranking
Scalable annotation approaches are crucial for constructing extensive 3D-text datasets, facilitating a broader range of applications.
PointInfinity: Resolution-Invariant Point Diffusion Models
We present PointInfinity, an efficient family of point cloud diffusion models.
Benchmarking Object Detectors with COCO: A New Path Forward
With these findings, we advocate using COCO-ReM for future object detection research.
FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation
Estimating relative camera poses between images has been a central problem in computer vision.
FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation
Estimating relative camera poses between images has been a central problem in computer vision.
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis
This interaction field guides the sampling of an object-conditioned human motion diffusion model, so as to encourage plausible contacts and affordance semantics.
Learning to Predict Scene-Level Implicit 3D from Posed RGBD Data
We introduce a method that can learn to predict scene-level implicit functions for 3D reconstruction from posed RGBD data.
Scalable 3D Captioning with Pretrained Models
We introduce Cap3D, an automatic approach for generating descriptive text for 3D objects.
Hyperbolic Image-Text Representations
Visual and linguistic concepts naturally organize themselves in a hierarchy, where a textual concept "dog" entails all images that contain dogs.
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
We present Text2Room, a method for generating room-scale textured 3D meshes from a given text prompt as input.
Learning Visual Representations via Language-Guided Sampling
Our approach diverges from image-based contrastive learning by sampling view pairs using language similarity instead of hand-crafted augmentations or learned clusters.
Text-To-4D Dynamic Scene Generation
We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions.
HexPlane: A Fast Representation for Dynamic Scenes
HexPlane is a simple and effective solution for representing 4D volumes, and we hope they can broadly contribute to modeling spacetime for dynamic 3D scenes.
Multiview Compressive Coding for 3D Reconstruction
We introduce a simple framework that operates on 3D points of single objects or whole scenes coupled with category-agnostic large-scale training from diverse RGB-D videos.
Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Program
On Text2Shape, ShapeGlot, ABO, Genre, and Program Synthetic datasets, Neural Shape Compiler shows strengths in $\textit{Text}$ $\Longrightarrow$ $\textit{Point Cloud}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Text}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Program}$, and Point Cloud Completion tasks.
Self-Supervised Correspondence Estimation via Multiview Registration
To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences.
RGB no more: Minimally-decoded JPEG Vision Transformers
However, these RGB images are commonly encoded in JPEG before saving to disk; decoding them imposes an unavoidable overhead for RGB networks.
The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs
We present a simple baseline for directly estimating the relative pose (rotation and translation, including scale) between two images.
Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild
In 3D, existing benchmarks are small in size and approaches specialize in few object categories and specific domains, e. g. urban driving scenes.
3D Object Detection
3D Object Detection From Monocular Images
+3
FWD: Real-time Novel View Synthesis with Forward Warping and Depth
Novel view synthesis (NVS) is a challenging task requiring systems to generate photorealistic images of scenes from new viewpoints, where both quality and speed are important for applications.
Learning 3D Object Shape and Layout without 3D Supervision
A 3D scene consists of a set of objects, each with a shape and a layout giving their position in space.
What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction
We present an approach for full 3D scene reconstruction from a single unseen image.
Recognizing Scenes from Novel Viewpoints
Humans can perceive scenes in 3D from a handful of 2D views.
StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions
Style transfer typically operates on 2D images, making stylization of a mesh challenging.
RedCaps: web-curated image-text data created by the people, for the people
We introduce RedCaps -- a large-scale dataset of 12M image-text pairs collected from Reddit.
PixelSynth: Generating a 3D-Consistent Experience from a Single Image
Recent advancements in differentiable rendering and 3D reasoning have driven exciting results in novel view synthesis from a single image.
Inverting and Understanding Object Detectors
As a core problem in computer vision, the performance of object detection has improved drastically in the past few years.
Bootstrap Your Own Correspondences
Our approach combines classic ideas from point cloud registration with more recent representation learning approaches.
Rethinking "Batch" in BatchNorm
BatchNorm is a critical building block in modern convolutional neural networks.
UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering
Aligning partial views of a scene into a single whole is essential to understanding one's environment and is a key component of numerous robotics tasks such as SLAM and SfM.
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations
Recent advances in self-supervised learning (SSL) have largely closed the gap with supervised ImageNet pretraining.
Accelerating 3D Deep Learning with PyTorch3D
We address these challenges by introducing PyTorch3D, a library of modular, efficient, and differentiable operators for 3D deep learning.
VirTex: Learning Visual Representations from Textual Annotations
The de-facto approach to many vision tasks is to start from pretrained visual representations, typically learned via supervised training on ImageNet.
Ranked #1 on
Object Detection
on COCO test-dev
(Hardware Burden metric)
SynSin: End-to-end View Synthesis from a Single Image
Single image view synthesis allows for the generation of new views of a scene given a single input image.
Temporal Reasoning via Audio Question Answering
In this paper, we use the task of Audio Question Answering (AQA) to study the temporal reasoning abilities of machine learning models.
Ranked #1 on
Audio Question Answering
on DAQA
PHYRE: A New Benchmark for Physical Reasoning
The benchmark is designed to encourage the development of learning algorithms that are sample-efficient and generalize well across puzzles.
Ranked #3 on
Visual Reasoning
on PHYRE-1B-Within
Mesh R-CNN
We propose a system that detects objects in real-world images and produces a triangle mesh giving the full 3D shape of each detected object.
Ranked #1 on
3D Shape Modeling
on Pix3D S2
On Network Design Spaces for Visual Recognition
Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape.
HiDDeN: Hiding Data With Deep Networks
We show that these encodings are competitive with existing data hiding algorithms, and further that they can be made robust to noise: our models learn to reconstruct hidden information in an encoded image despite the presence of Gaussian blurring, pixel-wise dropout, cropping, and JPEG compression.
Image Generation from Scene Graphs
To overcome this limitation we propose a method for generating images from scene graphs, enabling explicitly reasoning about objects and their relationships.
Ranked #4 on
Layout-to-Image Generation
on Visual Genome 64x64
Image Generation from Scene Graphs
Layout-to-Image Generation
DDRprog: A CLEVR Differentiable Dynamic Reasoning Programmer
We present a novel Dynamic Differentiable Reasoning (DDR) framework for jointly learning branching programs and the functions composing them; this resolves a significant nondifferentiability inhibiting recent dynamic architectures.
Ranked #9 on
Visual Question Answering (VQA)
on CLEVR
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments.
Ranked #4 on
Trajectory Prediction
on ETH
Inferring and Executing Programs for Visual Reasoning
Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes.
Ranked #5 on
Visual Question Answering (VQA)
on CLEVR-Humans
Characterizing and Improving Stability in Neural Style Transfer
Recent progress in style transfer on images has focused on improving the quality of stylized images and speed of methods.
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings.
A Hierarchical Approach for Generating Descriptive Image Paragraphs
Recent progress on image captioning has made it possible to generate novel sentences describing images in natural language, but compressing an image into a single sentence can describe visual content in only coarse detail.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
We consider image transformation problems, where an input image is transformed into an output image.
Ranked #4 on
Nuclear Segmentation
on Cell17
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering.
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
We introduce the dense captioning task, which requires a computer vision system to both localize and describe salient regions in images in natural language.
Ranked #3 on
Object Detection
on Visual Genome
Love Thy Neighbors: Image Annotation by Exploiting Image Metadata
Some images that are difficult to recognize on their own may become more clear in the context of a neighborhood of related images with similar social-network metadata.
Visualizing and Understanding Recurrent Networks
Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data.
Image Retrieval Using Scene Graphs
We introduce a novel dataset of 5, 000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval.
