Search Results for author: Niloy J. Mitra

Found 40 papers, 20 papers with code

Points2Surf Learning Implicit Surfaces from Point Clouds

no code implementations ECCV 2020 Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Niloy J. Mitra, Michael Wimmer

Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans.

ReLU Fields: The Little Non-linearity That Could

no code implementations22 May 2022 Animesh Karnewar, Tobias Ritschel, Oliver Wang, Niloy J. Mitra

Although the MLPs are able to represent complex scenes with unprecedented quality and memory footprint, this expressive power of the MLPs, however, comes at the cost of long training and inference times.

Neural Convolutional Surfaces

no code implementations5 Apr 2022 Luca Morreale, Noam Aigerman, Paul Guerrero, Vladimir G. Kim, Niloy J. Mitra

Our pipeline and architecture are designed so that disentanglement of global geometry from local details is accomplished through optimization, in a completely unsupervised manner.


InsetGAN for Full-Body Image Generation

1 code implementation14 Mar 2022 Anna Frühstück, Krishna Kumar Singh, Eli Shechtman, Niloy J. Mitra, Peter Wonka, Jingwan Lu

Instead of modeling this complex domain with a single GAN, we propose a novel method to combine multiple pretrained GANs, where one GAN generates a global canvas (e. g., human body) and a set of specialized GANs, or insets, focus on different parts (e. g., faces, shoes) that can be seamlessly inserted onto the global canvas.

Image Generation

ShapeFormer: Transformer-based Shape Completion via Sparse Representation

no code implementations25 Jan 2022 Xingguang Yan, Liqiang Lin, Niloy J. Mitra, Dani Lischinski, Daniel Cohen-Or, Hui Huang

We present ShapeFormer, a transformer-based network that produces a distribution of object completions, conditioned on incomplete, and possibly noisy, point clouds.

CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions

no code implementations9 Dec 2021 Rameen Abdal, Peihao Zhu, John Femiani, Niloy J. Mitra, Peter Wonka

The success of StyleGAN has enabled unprecedented semantic editing capabilities, on both synthesized and real images.

Zero-Shot Learning

Dance In the Wild: Monocular Human Animation with Neural Dynamic Appearance Synthesis

no code implementations10 Nov 2021 Tuanfeng Y. Wang, Duygu Ceylan, Krishna Kumar Singh, Niloy J. Mitra

Synthesizing dynamic appearances of humans in motion plays a central role in applications such as AR/VR and video editing.

Frame motion retargeting

CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds

1 code implementation ICCV 2021 Eric-Tuan Lê, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, Niloy J. Mitra

We present Cascaded Primitive Fitting Networks (CPFN) that relies on an adaptive patch sampling network to assemble detection results of global and local primitive detection networks.

MoCo-Flow: Neural Motion Consensus Flow for Dynamic Humans in Stationary Monocular Cameras

no code implementations8 Jun 2021 Xuelin Chen, Weiyu Li, Daniel Cohen-Or, Niloy J. Mitra, Baoquan Chen

In this paper, we introduce Neural Motion Consensus Flow (MoCo-Flow), a representation that models dynamic humans in stationary monocular cameras using a 4D continuous time-variant function.

Consistent Two-Flow Network for Tele-Registration of Point Clouds

1 code implementation1 Jun 2021 Zihao Yan, Zimu Yi, Ruizhen Hu, Niloy J. Mitra, Daniel Cohen-Or, Hui Huang

In this paper, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration.

ShapeMOD: Macro Operation Discovery for 3D Shape Programs

1 code implementation13 Apr 2021 R. Kenny Jones, David Charatan, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie

In this paper, we present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs.

Neural Surface Maps

1 code implementation CVPR 2021 Luca Morreale, Noam Aigerman, Vladimir Kim, Niloy J. Mitra

Maps are arguably one of the most fundamental concepts used to define and operate on manifold surfaces in differentiable geometry.

Dynamic Neural Garments

no code implementations23 Feb 2021 Meng Zhang, Duygu Ceylan, Tuanfeng Wang, Niloy J. Mitra

A vital task of the wider digital human effort is the creation of realistic garments on digital avatars, both in the form of characteristic fold patterns and wrinkles in static frames as well as richness of garment dynamics under avatars' motion.

Neural Rendering

Generative Modelling of BRDF Textures from Flash Images

no code implementations23 Feb 2021 Philipp Henzler, Valentin Deschaintre, Niloy J. Mitra, Tobias Ritschel

We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance.

Im2Vec: Synthesizing Vector Graphics without Vector Supervision

1 code implementation CVPR 2021 Pradyumna Reddy, Michael Gharbi, Michal Lukac, Niloy J. Mitra

The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time.

Vector Graphics

Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

no code implementations CVPR 2021 Norman Müller, Yu-Shiang Wong, Niloy J. Mitra, Angela Dai, Matthias Nießner

From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space.

3D Multi-Object Tracking Frame

Points2Surf: Learning Implicit Surfaces from Point Cloud Patches

2 code implementations20 Jul 2020 Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Michael Wimmer, Niloy J. Mitra

Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans.

Towards a Neural Graphics Pipeline for Controllable Image Generation

no code implementations18 Jun 2020 Xuelin Chen, Daniel Cohen-Or, Baoquan Chen, Niloy J. Mitra

NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation.

Image Generation Neural Rendering

Learning a Neural 3D Texture Space from 2D Exemplars

1 code implementation CVPR 2020 Philipp Henzler, Niloy J. Mitra, Tobias Ritschel

We propose a generative model of 2D and 3D natural textures with diversity, visual fidelity and at high computational efficiency.

Going Deeper with Lean Point Networks

1 code implementation CVPR 2020 Eric-Tuan Le, Iasonas Kokkinos, Niloy J. Mitra

By combining these blocks, we design wider and deeper point-based architectures.

Unsupervised Intuitive Physics from Past Experiences

no code implementations26 May 2019 Sébastien Ehrhardt, Aron Monszpart, Niloy J. Mitra, Andrea Vedaldi

We are interested in learning models of intuitive physics similar to the ones that animals use for navigation, manipulation and planning.


Unpaired Point Cloud Completion on Real Scans using Adversarial Training

2 code implementations ICLR 2020 Xuelin Chen, Baoquan Chen, Niloy J. Mitra

As 3D scanning solutions become increasingly popular, several deep learning setups have been developed geared towards that task of scan completion, i. e., plausibly filling in regions there were missed in the raw scans.

Point Cloud Completion

PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

1 code implementation4 Jan 2019 Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J. Mitra, Maks Ovsjanikov

Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers.

Denoising Surface Reconstruction

Learning a Shared Shape Space for Multimodal Garment Design

no code implementations29 Jun 2018 Tuanfeng Y. Wang, Duygu Ceylan, Jovan Popovic, Niloy J. Mitra

Designing real and virtual garments is becoming extremely demanding with rapidly changing fashion trends and increasing need for synthesizing realistic dressed digital humans for various applications.


iMapper: Interaction-guided Joint Scene and Human Motion Mapping from Monocular Videos

no code implementations20 Jun 2018 Aron Monszpart, Paul Guerrero, Duygu Ceylan, Ersin Yumer, Niloy J. Mitra

A long-standing challenge in scene analysis is the recovery of scene arrangements under moderate to heavy occlusion, directly from monocular video.

Human-Object Interaction Detection

FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs

1 code implementation19 Jun 2018 Tom Kelly, Paul Guerrero, Anthony Steed, Peter Wonka, Niloy J. Mitra

The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods.


Learning on the Edge: Explicit Boundary Handling in CNNs

no code implementations8 May 2018 Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra

Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding.

Colorization Disparity Estimation +1

Joint Material and Illumination Estimation from Photo Sets in the Wild

1 code implementation23 Oct 2017 Tuanfeng Y. Wang, Tobias Ritschel, Niloy J. Mitra

To the other hand, methods that are automatic and work on 'in the wild' Internet images, often extract only low-frequency lighting or diffuse materials.


PCPNET: Learning Local Shape Properties from Raw Point Clouds

35 code implementations13 Oct 2017 Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, Niloy J. Mitra

In this paper, we propose PCPNet, a deep-learning based approach for estimating local 3D shape properties in point clouds.

Computational Geometry

DepthCut: Improved Depth Edge Estimation Using Multiple Unreliable Channels

no code implementations22 May 2017 Paul Guerrero, Holger Winnemöller, Wilmot Li, Niloy J. Mitra

In the context of scene understanding, a variety of methods exists to estimate different information channels from mono or stereo images, including disparity, depth, and normals.

Scene Understanding

Learning A Physical Long-term Predictor

no code implementations1 Mar 2017 Sebastien Ehrhardt, Aron Monszpart, Niloy J. Mitra, Andrea Vedaldi

Evolution has resulted in highly developed abilities in many natural intelligences to quickly and accurately predict mechanical phenomena.

SMASH: Physics-guided Reconstruction of Collisions from Videos

1 code implementation29 Mar 2016 Aron Monszpart, Nils Thuerey, Niloy J. Mitra

Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized.

3D Reconstruction

RAID: A Relation-Augmented Image Descriptor

no code implementations5 Oct 2015 Paul Guerrero, Niloy J. Mitra, Peter Wonka

As humans, we regularly interpret images based on the relations between image regions.

Super 4PCS Fast Global Pointcloud Registration via Smart Indexing

1 code implementation Computer Graphics Forum 2014 Nicolas Mellado, Dror Aiger, Niloy J. Mitra

We present Super 4PCS for global pointcloud registration that is optimal, i. e., runs in linear time (in the number of data points) and is also output sensitive in the complexity of the alignment problem based on the (unknown) overlap across scan pairs.

Point Cloud Registration

SmartAnnotator: An Interactive Tool for Annotating RGBD Indoor Images

no code implementations23 Mar 2014 Yu-Shiang Wong, Hung-Kuo Chu, Niloy J. Mitra

Further, as more scenes are annotated, the system makes better suggestions based on structural and geometric priors learns from the previous annotation sessions.

Image Manipulation

Cannot find the paper you are looking for? You can Submit a new open access paper.