no code implementations • 22 Mar 2023 • Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, Angjoo Kanazawa
We propose a method for editing NeRF scenes with text-instructions.
no code implementations • 16 Mar 2023 • Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, Matthew Tancik
Humans describe the physical world using natural language to refer to specific 3D locations based on a vast range of properties: visual appearance, semantics, abstract associations, or actionable affordances.
no code implementations • 24 Feb 2023 • Vickie Ye, Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa
Our method robustly recovers the global 3D trajectories of people in challenging in-the-wild videos, such as PoseTrack.
1 code implementation • 8 Feb 2023 • Matthew Tancik, Ethan Weber, Evonne Ng, RuiLong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, Angjoo Kanazawa
Neural Radiance Fields (NeRF) are a rapidly growing area of research with wide-ranging applications in computer vision, graphics, robotics, and more.
1 code implementation • 24 Jan 2023 • Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, Angjoo Kanazawa
We introduce k-planes, a white-box model for radiance fields in arbitrary dimensions.
Ranked #1 on
Novel View Synthesis
on LLFF
1 code implementation • 24 Oct 2022 • Hang Gao, RuiLong Li, Shubham Tulsiani, Bryan Russell, Angjoo Kanazawa
We study the recent progress on dynamic view synthesis (DVS) from monocular video.
1 code implementation • 10 Oct 2022 • RuiLong Li, Matthew Tancik, Angjoo Kanazawa
We propose NerfAcc, a toolbox for efficient volumetric rendering of radiance fields.
no code implementations • 6 Sep 2022 • Vongani H. Maluleke, Neerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexei A. Efros, Angjoo Kanazawa, Devin Guillory
In this work, we study how the performance and evaluation of generative image models are impacted by the racial composition of their training datasets.
no code implementations • 28 Jul 2022 • Georgios Pavlakos, Ethan Weber, Matthew Tancik, Angjoo Kanazawa
TV shows depict a wide variety of human behaviors and have been studied extensively for their potential to be a rich source of data for many applications.
1 code implementation • 22 Jul 2022 • Zhengqi Li, Qianqian Wang, Noah Snavely, Angjoo Kanazawa
We present a method for learning to generate unbounded flythrough videos of natural scenes starting from a single view, where this capability is learned from a collection of single photographs, without requiring camera poses or even multiple views of each scene.
1 code implementation • 21 Jun 2022 • Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa, Serena Yeung
The ability to perceive 3D human bodies from a single image has a multitude of applications ranging from entertainment and robotics to neuroscience and healthcare.
1 code implementation • 17 Jun 2022 • RuiLong Li, Julian Tanke, Minh Vo, Michael Zollhofer, Jurgen Gall, Angjoo Kanazawa, Christoph Lassner
Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals.
no code implementations • CVPR 2022 • Evonne Ng, Hanbyul Joo, Liwen Hu, Hao Li, Trevor Darrell, Angjoo Kanazawa, Shiry Ginosar
We present a framework for modeling interactional communication in dyadic conversations: given multimodal inputs of a speaker, we autoregressively output multiple possibilities of corresponding listener motion.
no code implementations • CVPR 2022 • Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa, Noah Snavely
We describe a method to extract persistent elements of a dynamic scene from an input video.
no code implementations • CVPR 2022 • Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
3 code implementations • CVPR 2022 • Alex Yu, Sara Fridovich-Keil, Matthew Tancik, Qinhong Chen, Benjamin Recht, Angjoo Kanazawa
We introduce Plenoxels (plenoptic voxels), a system for photorealistic view synthesis.
no code implementations • 8 Dec 2021 • Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
1 code implementation • NeurIPS 2021 • Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Jitendra Malik
We find that 3D representations are more effective than 2D representations for tracking in these settings, and we obtain state-of-the-art performance.
no code implementations • ICLR 2022 • Shizhan Zhu, Sayna Ebrahimi, Angjoo Kanazawa, Trevor Darrell
Existing approaches for single object reconstruction impose supervision signals based on the loss of the signed distance value from all locations in a scene, posing difficulties when extending to real-world scenarios.
no code implementations • CVPR 2021 • Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa
We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category.
1 code implementation • CVPR 2021 • Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa
Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.
2 code implementations • 5 Apr 2021 • Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, Angjoo Kanazawa
Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips.
5 code implementations • ICCV 2021 • Alex Yu, RuiLong Li, Matthew Tancik, Hao Li, Ren Ng, Angjoo Kanazawa
We introduce a method to render Neural Radiance Fields (NeRFs) in real time using PlenOctrees, an octree-based 3D representation which supports view-dependent effects.
1 code implementation • ICCV 2021 • RuiLong Li, Shan Yang, David A. Ross, Angjoo Kanazawa
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music.
Ranked #2 on
Motion Synthesis
on BRACE
no code implementations • CVPR 2022 • Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa
The tools we develop open the door to processing and analyzing in 3D content from a large library of edited media, which could be helpful for many downstream applications.
1 code implementation • ICCV 2021 • Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa
We introduce the problem of perpetual view generation - long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image.
no code implementations • ICCV 2021 • Zhe Cao, Ilija Radosavovic, Angjoo Kanazawa, Jitendra Malik
In this work we explore reconstructing hand-object interactions in the wild.
2 code implementations • CVPR 2021 • Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa
This allows the network to be trained across multiple scenes to learn a scene prior, enabling it to perform novel view synthesis in a feed-forward manner from a sparse set of views (as few as one).
Ranked #3 on
Generalizable Novel View Synthesis
on ZJU-MoCap
1 code implementation • ECCV 2020 • Jason Y. Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, Angjoo Kanazawa
We present a method that infers spatial arrangements and shapes of humans and objects in a globally consistent 3D scene, all from a single image in-the-wild captured in an uncontrolled environment.
3D Human Pose Estimation
3D Shape Reconstruction From A Single 2D Image
+2
no code implementations • ECCV 2020 • Shubham Goel, Angjoo Kanazawa, Jitendra Malik
We present a learning framework that learns to recover the 3D shape, pose and texture from a single image, trained on an image collection without any ground truth 3D shape, multi-view, camera viewpoints or keypoint supervision.
2 code implementations • NeurIPS 2020 • Jake Levinson, Carlos Esteves, Kefan Chen, Noah Snavely, Angjoo Kanazawa, Afshin Rostamizadeh, Ameesh Makadia
Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$.
1 code implementation • ICCV 2019 • Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black
In contrast to research on human pose, shape and texture estimation, training data for endangered species is limited, the animals are in complex natural scenes with occlusion, they are naturally camouflaged, travel in herds, and look similar to each other.
1 code implementation • ICCV 2019 • Jason Y. Zhang, Panna Felsen, Angjoo Kanazawa, Jitendra Malik
In this work, we present perhaps the first approach for predicting a future 3D mesh model sequence of a person from past video input.
1 code implementation • ICCV 2019 • Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, Hao Li
We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object.
Ranked #1 on
3D Object Reconstruction
on RenderPeople
1 code implementation • CVPR 2019 • Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, Jitendra Malik
We present a framework that can similarly learn a representation of 3D dynamics of humans from video via a simple but effective temporal encoding of image features.
Ranked #8 on
3D Human Pose Estimation
on 3DPW
(Acceleration Error metric)
1 code implementation • 8 Oct 2018 • Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, Sergey Levine
In this paper, we propose a method that enables physically simulated characters to learn skills from videos (SFV).
5 code implementations • ICLR 2019 • Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, Sergey Levine
By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients.
no code implementations • CVPR 2018 • Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, David W. Jacobs
SfSNet learns from a mixture of labeled synthetic and unlabeled real world images.
no code implementations • CVPR 2018 • Silvia Zuffi, Angjoo Kanazawa, Michael J. Black
Animals are widespread in nature and the analysis of their shape and motion is important in many fields and industries.
no code implementations • ECCV 2018 • Angjoo Kanazawa, Shubham Tulsiani, Alexei A. Efros, Jitendra Malik
The shape is represented as a deformable 3D mesh model of an object category where a shape is parameterized by a learned mean shape and per-instance predicted deformation.
7 code implementations • CVPR 2018 • Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik
The main objective is to minimize the reprojection loss of keypoints, which allow our model to be trained using images in-the-wild that only have ground truth 2D annotations.
Ranked #1 on
Weakly-supervised 3D Human Pose Estimation
on Human3.6M
(3D Annotations metric)
1 code implementation • CVPR 2018 • Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, David Jacobs
SfSNet learns from a mixture of labeled synthetic and unlabeled real world images.
no code implementations • 24 Jul 2017 • Yinghao Huang, Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V. Gehler, Ijaz Akhter, Michael J. Black
Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios.
no code implementations • CVPR 2017 • Silvia Zuffi, Angjoo Kanazawa, David Jacobs, Michael J. Black
The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals.
2 code implementations • 27 Jul 2016 • Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, Michael J. Black
We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints.
Ranked #27 on
3D Human Pose Estimation
on HumanEva-I
no code implementations • CVPR 2016 • Angjoo Kanazawa, David W. Jacobs, Manmohan Chandraker
This is in contrast to prior works that require part annotations, since matching objects across class and pose variations is challenging with appearance features alone.
no code implementations • 28 Jul 2015 • Angjoo Kanazawa, Shahar Kovalsky, Ronen Basri, David W. Jacobs
In this paper, we show that such information can be learned from user-clicked 2D images and a template 3D model of the target animal.
no code implementations • 16 Dec 2014 • Angjoo Kanazawa, Abhishek Sharma, David Jacobs
We show on a modified MNIST dataset that when faced with scale variation, building in scale-invariance allows ConvNets to learn more discriminative features with reduced chances of over-fitting.