Search Results for author: Natalia Neverova

Found 38 papers, 17 papers with code

CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos

1 code implementation15 Oct 2024 Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, Christian Rupprecht

Most state-of-the-art point trackers are trained on synthetic data due to the difficulty of annotating real videos for this task.

Point Tracking

Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objects

no code implementations2 Jul 2024 Raphael Bensadoun, Yanir Kleiman, Idan Azuri, Omri Harosh, Andrea Vedaldi, Natalia Neverova, Oran Gafni

The recent availability and adaptability of text-to-image models has sparked a new era in many related domains that benefit from the learned text priors as well as high-quality and fast generation capabilities, one of which is texture generation for 3D objects.

4k Texture Synthesis

Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials

no code implementations2 Jul 2024 Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahendra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, David Novotny

We present Meta 3D AssetGen (AssetGen), a significant advancement in text-to-3D generation which produces faithful, high-quality meshes with texture and material control.

3D Generation Text to 3D

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

no code implementations13 Feb 2024 Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, Filippos Kokkinos

A mitigation is to fine-tune the 2D generator to be multi-view aware, which can help distillation or can be combined with reconstruction networks to output 3D objects directly.

3D Generation 3D Reconstruction +1

Mosaic-SDF for 3D Generative Models

no code implementations CVPR 2024 Lior Yariv, Omri Puny, Natalia Neverova, Oran Gafni, Yaron Lipman

Current diffusion or flow-based generative models for 3D shapes divide to two: distilling pre-trained 2D image diffusion models, and training directly on 3D shapes.

3D Generation 3D Shape Representation +1

DynamicStereo: Consistent Dynamic Depth from Stereo Videos

1 code implementation CVPR 2023 Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, Christian Rupprecht

The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions.

Real-time volumetric rendering of dynamic humans

1 code implementation21 Mar 2023 Ignacio Rocco, Iurii Makarov, Filippos Kokkinos, David Novotny, Benjamin Graham, Natalia Neverova, Andrea Vedaldi

We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits.

3D Reconstruction

Novel-View Acoustic Synthesis

no code implementations CVPR 2023 Changan Chen, Alexander Richard, Roman Shapovalov, Vamsi Krishna Ithapu, Natalia Neverova, Kristen Grauman, Andrea Vedaldi

We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen target viewpoint?

Neural Rendering Novel View Synthesis

Self-Supervised Correspondence Estimation via Multiview Registration

1 code implementation6 Dec 2022 Mohamed El Banani, Ignacio Rocco, David Novotny, Andrea Vedaldi, Natalia Neverova, Justin Johnson, Benjamin Graham

To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences.

Diversity

Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Space

no code implementations ICLR 2022 Steeven Janny, Fabien Baradel, Natalia Neverova, Madiha Nadri, Greg Mori, Christian Wolf

Learning causal relationships in high-dimensional data (images, videos) is a hard task, as they are often defined on low dimensional manifolds and must be extracted from complex signals dominated by appearance, lighting, textures and also spurious correlations in the data.

counterfactual Counterfactual Reasoning +1

KeyTr: Keypoint Transporter for 3D Reconstruction of Deformable Objects in Videos

no code implementations CVPR 2022 David Novotny, Ignacio Rocco, Samarth Sinha, Alexandre Carlier, Gael Kerchenbaum, Roman Shapovalov, Nikita Smetanin, Natalia Neverova, Benjamin Graham, Andrea Vedaldi

Compared to weaker deformation models, this significantly reduces the reconstruction ambiguity and, for dynamic objects, allows Keypoint Transporter to obtain reconstructions of the quality superior or at least comparable to prior approaches while being much faster and reliant on a pre-trained monocular depth estimator network.

3D Reconstruction Depth Estimation +2

BANMo: Building Animatable 3D Neural Models from Many Casual Videos

1 code implementation CVPR 2022 Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo

Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.

3D Shape Reconstruction from Videos Dynamic Reconstruction

NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go

no code implementations CVPR 2021 Marvin Eisenberger, David Novotny, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, Andrea Vedaldi

We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i. e. in a single feed forward pass, a smooth interpolation and point-to-point correspondences between them.

XCiT: Cross-Covariance Image Transformers

11 code implementations NeurIPS 2021 Alaaeldin El-Nouby, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, Hervé Jegou

We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries.

Instance Segmentation object-detection +3

Cross-Modal Retrieval Augmentation for Multi-Modal Classification

no code implementations Findings (EMNLP) 2021 Shir Gur, Natalia Neverova, Chris Stauffer, Ser-Nam Lim, Douwe Kiela, Austin Reiter

Recent advances in using retrieval components over external knowledge sources have shown impressive results for a variety of downstream tasks in natural language processing.

Cross-Modal Retrieval General Classification +4

Training Vision Transformers for Image Retrieval

1 code implementation10 Feb 2021 Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev, Hervé Jégou

Transformers have shown outstanding results for natural language understanding and, more recently, for image classification.

Image Classification Image Retrieval +3

Continuous Surface Embeddings

1 code implementation NeurIPS 2020 Natalia Neverova, David Novotny, Vasil Khalidov, Marc Szafraniec, Patrick Labatut, Andrea Vedaldi

In this work, we focus on the task of learning and representing dense correspondences in deformable object categories.

Object Pose Estimation

Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation

1 code implementation7 Apr 2020 Hanbyul Joo, Natalia Neverova, Andrea Vedaldi

Remarkably, the resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks such as 3DPW.

Ranked #23 on 3D Human Pose Estimation on MPI-INF-3DHP (PA-MPJPE metric)

3D Human Pose Estimation 3D Pose Estimation

Transferring Dense Pose to Proximal Animal Classes

1 code implementation CVPR 2020 Artsiom Sanakoyeu, Vasil Khalidov, Maureen S. McCarthy, Andrea Vedaldi, Natalia Neverova

Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail.

Transfer Learning

Correlated Uncertainty for Learning Dense Correspondences from Noisy Labels

no code implementations NeurIPS 2019 Natalia Neverova, David Novotny, Andrea Vedaldi

We show that these models, by understanding uncertainty better, can solve the original DensePose task more accurately, thus setting the new state-of-the-art accuracy in this benchmark.

CoPhy: Counterfactual Learning of Physical Dynamics

1 code implementation ICLR 2020 Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf

Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world.

counterfactual Video Prediction

C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion

2 code implementations ICCV 2019 David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, Andrea Vedaldi

We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images.

Dense Pose Transfer

no code implementations ECCV 2018 Natalia Neverova, Riza Alp Guler, Iasonas Kokkinos

In this work we integrate ideas from surface-based modeling with neural synthesis: we propose a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i. e. synthesize a new image of a person based on a single image of that person and the image of a pose donor.

Pose Estimation Pose Transfer

Object Level Visual Reasoning in Videos

1 code implementation ECCV 2018 Fabien Baradel, Natalia Neverova, Christian Wolf, Julien Mille, Greg Mori

Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context.

Human Activity Recognition Object +3

DensePose: Dense Human Pose Estimation In The Wild

22 code implementations CVPR 2018 Riza Alp Güler, Natalia Neverova, Iasonas Kokkinos

In this work, we establish dense correspondences between RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation.

Monocular 3D Human Pose Estimation

Mass Displacement Networks

no code implementations12 Aug 2017 Natalia Neverova, Iasonas Kokkinos

Despite the large improvements in performance attained by using deep learning in computer vision, one can often further improve results with some additional post-processing that exploits the geometric nature of the underlying task.

Pose Estimation

Houdini: Fooling Deep Structured Prediction Models

no code implementations17 Jul 2017 Moustapha Cisse, Yossi Adi, Natalia Neverova, Joseph Keshet

Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines.

General Classification Pose Estimation +4

Learning Human Identity from Motion Patterns

no code implementations12 Nov 2015 Natalia Neverova, Christian Wolf, Griffin Lacey, Lex Fridman, Deepak Chandra, Brandon Barbello, Graham Taylor

We present a large-scale study exploring the capability of temporal deep neural networks to interpret natural human kinematics and introduce the first method for active biometric authentication with mobile inertial sensors.

ModDrop: adaptive multi-modal gesture recognition

no code implementations31 Dec 2014 Natalia Neverova, Christian Wolf, Graham W. Taylor, Florian Nebout

We present a method for gesture detection and localisation based on multi-scale and multi-modal deep learning.

Gesture Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.