Search Results for author:
Found 37 papers, 23 papers with code
Is CLIP ideal? No. Can we fix it? Yes!
Contrastive Language-Image Pre-Training (CLIP) is a popular method for learning multimodal latent spaces with well-organized semantics.
Visual Agentic AI for Spatial Reasoning with a Dynamic API
We show that our method outperforms prior zero-shot models for visual reasoning in 3D and empirically validate the effectiveness of our agentic framework for 3D spatial reasoning tasks.
Find Any Part in 3D
We study open-world part segmentation in 3D: segmenting any part in any object based on any text query.
Reconstructing Hand-Held Objects in 3D from Images and Videos
In order to obtain the best performing single frame model, we first present MCC-Hand-Object (MCC-HO), which jointly reconstructs hand and object geometry given a single RGB image and inferred 3D hand as inputs.
Caltech Aerial RGB-Thermal Dataset in the Wild
We present the first publicly-available RGB-thermal dataset designed for aerial robotics operating in natural environments.
TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
We conclude that TOTEM matches or outperforms existing state-of-the-art models in both the canonical specialist setting (i. e., training one model on one domain) as well as the generalist setting (i. e., training a single model on many domains), which demonstrates the efficacy of tokenization for general time series analysis.
A Foundation Model for Cell Segmentation
Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.
Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection
We present PARQ - a multi-view 3D object detector with transformer and pixel-aligned recurrent queries.
Multiview Compressive Coding for 3D Reconstruction
We introduce a simple framework that operates on 3D points of single objects or whole scenes coupled with category-agnostic large-scale training from diverse RGB-D videos.
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild
In 3D, existing benchmarks are small in size and approaches specialize in few object categories and specific domains, e. g. urban driving scenes.
3D Object Detection
3D Object Detection From Monocular Images
+3
Learning 3D Object Shape and Layout without 3D Supervision
A 3D scene consists of a set of objects, each with a shape and a layout giving their position in space.
Recognizing Scenes from Novel Viewpoints
Humans can perceive scenes in 3D from a handful of 2D views.
Differentiable Stereopsis: Meshes from multiple views using differentiable rendering
We propose Differentiable Stereopsis, a multi-view stereo approach that reconstructs shape and texture from few input views and noisy cameras.
Compressed Object Detection
With our approach, we are able to compress a state-of-the-art object detection model by 30. 0% without a loss in performance.
Accelerating 3D Deep Learning with PyTorch3D
We address these challenges by introducing PyTorch3D, a library of modular, efficient, and differentiable operators for 3D deep learning.
3D Shape Reconstruction from Vision and Touch
When a toddler is presented a new toy, their instinctual behaviour is to pick it upand inspect it with their hand and eyes in tandem, clearly searching over its surface to properly understand what they are playing with.
SynSin: End-to-end View Synthesis from a Single Image
Single image view synthesis allows for the generation of new views of a scene given a single input image.
Bayesian Relational Memory for Semantic Visual Navigation
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards.
Mesh R-CNN
We propose a system that detects objects in real-world images and produces a triangle mesh giving the full 3D shape of each detected object.
Ranked #1 on
3D Shape Modeling
on Pix3D S2
Multi-Target Embodied Question Answering
To address this, we propose a modular architecture composed of a program generator, a controller, a navigator, and a VQA module.
Embodied Question Answering in Photorealistic Environments with Point Cloud Perception
To help bridge the gap between internet vision-style problems and the goal of vision for embodied perception we instantiate a large-scale navigation task -- Embodied Question Answering [1] in photo-realistic environments (Matterport 3D).
Neural Modular Control for Embodied Question Answering
We use imitation learning to warm-start policies at each level of the hierarchy, dramatically increasing sample efficiency, followed by reinforcement learning.
Learning and Planning with a Semantic Model
Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI.
Building Generalizable Agents with a Realistic and Rich 3D Environment
To generalize to unseen environments, an agent needs to be robust to low-level variations (e. g. color, texture, object changes), and also high-level variations (e. g. layout changes of the environment).
Detect-and-Track: Efficient Pose Estimation in Videos
This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video.
Ranked #7 on
Keypoint Detection
on COCO test-challenge
Data Distillation: Towards Omni-Supervised Learning
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data.
Embodied Question Answering
We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- where an agent is spawned at a random location in a 3D environment and asked a question ("What color is the car?").
Detecting and Recognizing Human-Object Interactions
Our hypothesis is that the appearance of a person -- their pose, clothing, action -- is a powerful cue for localizing the objects they are interacting with.
Ranked #55 on
Human-Object Interaction Detection
on HICO-DET
Mask R-CNN
Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance.
Ranked #1 on
Keypoint Detection
on COCO
Chained Predictions Using Convolutional Neural Networks
In this model the output variables for a given input are predicted sequentially using neural networks.
Contextual Action Recognition with R*CNN
In this work, we exploit the simple observation that actions are accompanied by contextual cues to build a strong action recognition system.
Ranked #4 on
Weakly Supervised Object Detection
on HICO-DET
Actions and Attributes from Wholes and Parts
We investigate the importance of parts for the tasks of action and attribute classification.
Finding Action Tubes
We address the problem of action detection in videos.
Ranked #4 on
Action Detection
on UCF Sports
R-CNNs for Pose Estimation and Action Detection
We present convolutional neural networks for the tasks of keypoint (pose) prediction and action classification of people in unconstrained images.
Using k-Poselets for Detecting People and Localizing Their Keypoints
A k-poselet is a deformable part model (DPM) with k parts, where each of the parts is a poselet, aligned to a specific configuration of keypoints based on ground-truth annotations.
Articulated Pose Estimation Using Discriminative Armlet Classifiers
We propose a novel approach for human pose estimation in real-world cluttered scenes, and focus on the challenging problem of predicting the pose of both arms for each person in the image.
