Search Results for author: Angjoo Kanazawa

Found 65 papers, 36 papers with code

Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind

no code implementations7 Apr 2024 Chiara Plizzari, Shubham Goel, Toby Perrett, Jacob Chalk, Angjoo Kanazawa, Dima Damen

As humans move around, performing their daily tasks, they are able to recall where they have positioned objects in their environment, even if these objects are currently out of sight.

The More You See in 2D, the More You Perceive in 3D

no code implementations4 Apr 2024 Xinyang Han, Zelin Gao, Angjoo Kanazawa, Shubham Goel, Yossi Gandelsman

Inspired by this behavior, we introduce SAP3D, a system for 3D reconstruction and novel view synthesis from an arbitrary number of unposed images.

3D Reconstruction Image to 3D +1

GARField: Group Anything with Radiance Fields

1 code implementation17 Jan 2024 Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Goldberg, Matthew Tancik, Angjoo Kanazawa

We optimize this field from a set of 2D masks provided by Segment Anything (SAM) in a way that respects coarse-to-fine hierarchy, using scale to consistently fuse conflicting masks from different viewpoints.

Scene Understanding

From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations

1 code implementation3 Jan 2024 Evonne Ng, Javier Romero, Timur Bagautdinov, Shaojie Bai, Trevor Darrell, Angjoo Kanazawa, Alexander Richard

We present a framework for generating full-bodied photorealistic avatars that gesture according to the conversational dynamics of a dyadic interaction.


Reconstructing Hands in 3D with Transformers

no code implementations8 Dec 2023 Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, Jitendra Malik

The key to HaMeR's success lies in scaling up both the data used for training and the capacity of the deep network for hand reconstruction.

NeRFiller: Completing Scenes via Generative 3D Inpainting

no code implementations7 Dec 2023 Ethan Weber, Aleksander Hołyński, Varun Jampani, Saurabh Saxena, Noah Snavely, Abhishek Kar, Angjoo Kanazawa

In contrast to related works, we focus on completing scenes rather than deleting foreground objects, and our approach does not require tight 2D object masks or text.

3D Inpainting

Mathematical Supplement for the $\texttt{gsplat}$ Library

1 code implementation4 Dec 2023 Vickie Ye, Angjoo Kanazawa

This report provides the mathematical details of the gsplat library, a modular toolbox for efficient differentiable Gaussian splatting, as proposed by Kerbl et al.

State of the Art on Diffusion Models for Visual Computing

no code implementations11 Oct 2023 Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T. Barron, Amit H. Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, C. Karen Liu, Lingjie Liu, Ben Mildenhall, Matthias Nießner, Björn Ommer, Christian Theobalt, Peter Wonka, Gordon Wetzstein

The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.

Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping

no code implementations14 Sep 2023 Adam Rashid, Satvik Sharma, Chung Min Kim, Justin Kerr, Lawrence Chen, Angjoo Kanazawa, Ken Goldberg

Instead, we propose LERF-TOGO, Language Embedded Radiance Fields for Task-Oriented Grasping of Objects, which uses vision-language models zero-shot to output a grasp distribution over an object given a natural language query.


Can Language Models Learn to Listen?

no code implementations ICCV 2023 Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, Shiry Ginosar

We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words.

Language Modelling Large Language Model

Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives

no code implementations NeurIPS 2023 Tom Monnier, Jake Austin, Angjoo Kanazawa, Alexei A. Efros, Mathieu Aubry

We compare our approach to the state of the art on diverse scenes from DTU, and demonstrate its robustness on real-life captures from BlendedMVS and Nerfstudio.

Physical Simulations

Generative Proxemics: A Prior for 3D Social Interaction from Images

1 code implementation15 Jun 2023 Lea Müller, Vickie Ye, Georgios Pavlakos, Michael Black, Angjoo Kanazawa

To address this, we present a novel approach that learns a prior over the 3D proxemics two people in close social interaction and demonstrate its use for single-view 3D reconstruction.

3D Reconstruction Denoising +1

NerfAcc: Efficient Sampling Accelerates NeRFs

no code implementations ICCV 2023 RuiLong Li, Hang Gao, Matthew Tancik, Angjoo Kanazawa

Optimizing and rendering Neural Radiance Fields is computationally expensive due to the vast number of samples required by volume rendering.

Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs

1 code implementation ICCV 2023 Frederik Warburg, Ethan Weber, Matthew Tancik, Aleksander Holynski, Angjoo Kanazawa

Casually captured Neural Radiance Fields (NeRFs) suffer from artifacts such as floaters or flawed geometry when rendered outside the camera trajectory.

Novel View Synthesis

Generating Continual Human Motion in Diverse 3D Scenes

no code implementations4 Apr 2023 Aymen Mir, Xavier Puig, Angjoo Kanazawa, Gerard Pons-Moll

We decompose the continual motion synthesis problem into walking along paths and transitioning in and out of the actions specified by the keypoints, which enables long generation of motions that satisfy scene constraints without explicitly incorporating scene information.

Motion Synthesis Navigate

LERF: Language Embedded Radiance Fields

5 code implementations ICCV 2023 Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, Matthew Tancik

Humans describe the physical world using natural language to refer to specific 3D locations based on a vast range of properties: visual appearance, semantics, abstract associations, or actionable affordances.

Decoupling Human and Camera Motion from Videos in the Wild

1 code implementation CVPR 2023 Vickie Ye, Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa

Our method robustly recovers the global 3D trajectories of people in challenging in-the-wild videos, such as PoseTrack.

Nerfstudio: A Modular Framework for Neural Radiance Field Development

2 code implementations8 Feb 2023 Matthew Tancik, Ethan Weber, Evonne Ng, RuiLong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, Angjoo Kanazawa

Neural Radiance Fields (NeRF) are a rapidly growing area of research with wide-ranging applications in computer vision, graphics, robotics, and more.

Monocular Dynamic View Synthesis: A Reality Check

1 code implementation24 Oct 2022 Hang Gao, RuiLong Li, Shubham Tulsiani, Bryan Russell, Angjoo Kanazawa

We study the recent progress on dynamic view synthesis (DVS) from monocular video.

NerfAcc: A General NeRF Acceleration Toolbox

1 code implementation10 Oct 2022 RuiLong Li, Matthew Tancik, Angjoo Kanazawa

We propose NerfAcc, a toolbox for efficient volumetric rendering of radiance fields.

Studying Bias in GANs through the Lens of Race

no code implementations6 Sep 2022 Vongani H. Maluleke, Neerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexei A. Efros, Angjoo Kanazawa, Devin Guillory

In this work, we study how the performance and evaluation of generative image models are impacted by the racial composition of their training datasets.

The One Where They Reconstructed 3D Humans and Environments in TV Shows

no code implementations28 Jul 2022 Georgios Pavlakos, Ethan Weber, Matthew Tancik, Angjoo Kanazawa

TV shows depict a wide variety of human behaviors and have been studied extensively for their potential to be a rich source of data for many applications.

3D Reconstruction Gaze Estimation

InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images

1 code implementation22 Jul 2022 Zhengqi Li, Qianqian Wang, Noah Snavely, Angjoo Kanazawa

We present a method for learning to generate unbounded flythrough videos of natural scenes starting from a single view, where this capability is learned from a collection of single photographs, without requiring camera poses or even multiple views of each scene.

Perpetual View Generation

Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery

1 code implementation21 Jun 2022 Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa, Serena Yeung

The ability to perceive 3D human bodies from a single image has a multitude of applications ranging from entertainment and robotics to neuroscience and healthcare.

Data Augmentation Domain Adaptation +1

TAVA: Template-free Animatable Volumetric Actors

1 code implementation17 Jun 2022 RuiLong Li, Julian Tanke, Minh Vo, Michael Zollhofer, Jurgen Gall, Angjoo Kanazawa, Christoph Lassner

Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals.

Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion

no code implementations CVPR 2022 Evonne Ng, Hanbyul Joo, Liwen Hu, Hao Li, Trevor Darrell, Angjoo Kanazawa, Shiry Ginosar

We present a framework for modeling interactional communication in dyadic conversations: given multimodal inputs of a speaker, we autoregressively output multiple possibilities of corresponding listener motion.

Tracking People by Predicting 3D Appearance, Location and Pose

no code implementations CVPR 2022 Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik

For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.

Tracking People by Predicting 3D Appearance, Location & Pose

no code implementations8 Dec 2021 Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik

For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.

Tracking People with 3D Representations

1 code implementation NeurIPS 2021 Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik

We find that 3D representations are more effective than 2D representations for tracking in these settings, and we obtain state-of-the-art performance.

Differentiable Gradient Sampling for Learning Implicit 3D Scene Reconstructions from a Single Image

no code implementations ICLR 2022 Shizhan Zhu, Sayna Ebrahimi, Angjoo Kanazawa, Trevor Darrell

Existing approaches for single object reconstruction impose supervision signals based on the loss of the signed distance value from all locations in a scene, posing difficulties when extending to real-world scenarios.

Indoor Scene Reconstruction Object Reconstruction +1

De-rendering the World's Revolutionary Artefacts

1 code implementation CVPR 2021 Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa

Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.

AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control

3 code implementations5 Apr 2021 Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, Angjoo Kanazawa

Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips.

Imitation Learning Reinforcement Learning (RL)

PlenOctrees for Real-time Rendering of Neural Radiance Fields

5 code implementations ICCV 2021 Alex Yu, RuiLong Li, Matthew Tancik, Hao Li, Ren Ng, Angjoo Kanazawa

We introduce a method to render Neural Radiance Fields (NeRFs) in real time using PlenOctrees, an octree-based 3D representation which supports view-dependent effects.

Neural Rendering Novel View Synthesis

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++

1 code implementation ICCV 2021 RuiLong Li, Shan Yang, David A. Ross, Angjoo Kanazawa

We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music.

Motion Synthesis Pose Estimation

Human Mesh Recovery from Multiple Shots

no code implementations CVPR 2022 Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa

The tools we develop open the door to processing and analyzing in 3D content from a large library of edited media, which could be helpful for many downstream applications.

3D Reconstruction Human Mesh Recovery

Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image

1 code implementation ICCV 2021 Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa

We introduce the problem of perpetual view generation - long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image.

Image Generation Perpetual View Generation +1

pixelNeRF: Neural Radiance Fields from One or Few Images

2 code implementations CVPR 2021 Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa

This allows the network to be trained across multiple scenes to learn a scene prior, enabling it to perform novel view synthesis in a feed-forward manner from a sparse set of views (as few as one).

3D Reconstruction Generalizable Novel View Synthesis +1

Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild

1 code implementation ECCV 2020 Jason Y. Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, Angjoo Kanazawa

We present a method that infers spatial arrangements and shapes of humans and objects in a globally consistent 3D scene, all from a single image in-the-wild captured in an uncontrolled environment.

3D Human Pose Estimation 3D Human Reconstruction +5

Shape and Viewpoint without Keypoints

no code implementations ECCV 2020 Shubham Goel, Angjoo Kanazawa, Jitendra Malik

We present a learning framework that learns to recover the 3D shape, pose and texture from a single image, trained on an image collection without any ground truth 3D shape, multi-view, camera viewpoints or keypoint supervision.

An Analysis of SVD for Deep Rotation Estimation

2 code implementations NeurIPS 2020 Jake Levinson, Carlos Esteves, Kefan Chen, Noah Snavely, Angjoo Kanazawa, Afshin Rostamizadeh, Ameesh Makadia

Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$.

3D Pose Estimation 3D Rotation Estimation

Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"

1 code implementation ICCV 2019 Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black

In contrast to research on human pose, shape and texture estimation, training data for endangered species is limited, the animals are in complex natural scenes with occlusion, they are naturally camouflaged, travel in herds, and look similar to each other.

Pose Estimation Texture Synthesis

Predicting 3D Human Dynamics from Video

1 code implementation ICCV 2019 Jason Y. Zhang, Panna Felsen, Angjoo Kanazawa, Jitendra Malik

In this work, we present perhaps the first approach for predicting a future 3D mesh model sequence of a person from past video input.

3D Human Dynamics 3D Human Pose Estimation +2

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

1 code implementation ICCV 2019 Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, Hao Li

We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object.

3D Human Pose Estimation 3D Human Reconstruction +3

Learning 3D Human Dynamics from Video

1 code implementation CVPR 2019 Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, Jitendra Malik

We present a framework that can similarly learn a representation of 3D dynamics of humans from video via a simple but effective temporal encoding of image features.

Ranked #15 on 3D Human Pose Estimation on 3DPW (Acceleration Error metric)

3D Human Dynamics 3D Human Pose Estimation

SFV: Reinforcement Learning of Physical Skills from Videos

1 code implementation8 Oct 2018 Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, Sergey Levine

In this paper, we propose a method that enables physically simulated characters to learn skills from videos (SFV).

Pose Estimation reinforcement-learning +1

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

5 code implementations ICLR 2019 Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, Sergey Levine

By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients.

Continuous Control Image Generation +1

Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape From Images

no code implementations CVPR 2018 Silvia Zuffi, Angjoo Kanazawa, Michael J. Black

Animals are widespread in nature and the analysis of their shape and motion is important in many fields and industries.

Learning Category-Specific Mesh Reconstruction from Image Collections

no code implementations ECCV 2018 Angjoo Kanazawa, Shubham Tulsiani, Alexei A. Efros, Jitendra Malik

The shape is represented as a deformable 3D mesh model of an object category where a shape is parameterized by a learned mean shape and per-instance predicted deformation.

End-to-end Recovery of Human Shape and Pose

9 code implementations CVPR 2018 Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik

The main objective is to minimize the reprojection loss of keypoints, which allow our model to be trained using images in-the-wild that only have ground truth 2D annotations.

3D Hand Pose Estimation 3D Human Shape Estimation +4

Towards Accurate Markerless Human Shape and Pose Estimation over Time

no code implementations24 Jul 2017 Yinghao Huang, Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V. Gehler, Ijaz Akhter, Michael J. Black

Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios.

Pose Estimation

3D Menagerie: Modeling the 3D shape and pose of animals

no code implementations CVPR 2017 Silvia Zuffi, Angjoo Kanazawa, David Jacobs, Michael J. Black

The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals.

WarpNet: Weakly Supervised Matching for Single-view Reconstruction

no code implementations CVPR 2016 Angjoo Kanazawa, David W. Jacobs, Manmohan Chandraker

This is in contrast to prior works that require part annotations, since matching objects across class and pose variations is challenging with appearance features alone.

Learning 3D Deformation of Animals from 2D Images

no code implementations28 Jul 2015 Angjoo Kanazawa, Shahar Kovalsky, Ronen Basri, David W. Jacobs

In this paper, we show that such information can be learned from user-clicked 2D images and a template 3D model of the target animal.

Locally Scale-Invariant Convolutional Neural Networks

no code implementations16 Dec 2014 Angjoo Kanazawa, Abhishek Sharma, David Jacobs

We show on a modified MNIST dataset that when faced with scale variation, building in scale-invariance allows ConvNets to learn more discriminative features with reduced chances of over-fitting.

Cannot find the paper you are looking for? You can Submit a new open access paper.