Search Results for author: Matthias Zwicker

Found 46 papers, 18 papers with code

OmnimatteRF: Robust Omnimatte with 3D Background Modeling

no code implementations ICCV 2023 Geng Lin, Chen Gao, Jia-Bin Huang, Changil Kim, Yipeng Wang, Matthias Zwicker, Ayush Saraf

Video matting has broad applications, from adding interesting effects to casually captured movies to assisting video production professionals.

Image Matting Video Matting

Inverse Global Illumination using a Neural Radiometric Prior

no code implementations3 May 2023 Saeed Hadadan, Geng Lin, Jan Novák, Fabrice Rousselle, Matthias Zwicker

We train our radiance network and optimize scene parameters simultaneously using a loss consisting of both a photometric term between renderings and the multi-view input images, and our radiometric prior (the residual term).

Inverse Rendering

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

1 code implementation CVPR 2022 Baorui Ma, Yu-Shen Liu, Matthias Zwicker, Zhizhong Han

To reconstruct a surface at a specific query location at inference time, these methods then match the local reconstruction target by searching for the best match in the local prior space (by optimizing the parameters encoding the local context) at the given query location.

Surface Reconstruction

Differentiable Neural Radiosity

no code implementations31 Jan 2022 Saeed Hadadan, Matthias Zwicker

We introduce Differentiable Neural Radiosity, a novel method of representing the solution of the differential rendering equation using a neural network.

Inverse Rendering

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

no code implementations19 Dec 2021 Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, Matthias Zwicker

To handle complicated motions (e. g., self-occlusions), we then leverage the encoded information on the UV manifold to construct a 3D volumetric representation based on a dynamic pose-conditioned neural radiance field.

Neural Rendering

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images

no code implementations ICCV 2021 Tao Hu, Kripasindhu Sarkar, Lingjie Liu, Matthias Zwicker, Christian Theobalt

We next combine the target pose image and the textures into a combined feature image, which is transformed into the output color image using a neural image translation network.

Texture Synthesis Translation

Improving the Perceptual Quality of 2D Animation Interpolation

1 code implementation24 Nov 2021 Shuhong Chen, Matthias Zwicker

Traditional 2D animation is labor-intensive, often requiring animators to manually draw twelve illustrations per second of movement.


PatchGame: Learning to Signal Mid-level Patches in Referential Games

1 code implementation NeurIPS 2021 Kamal Gupta, Gowthami Somepalli, Anubhav Gupta, Vinoj Jayasundara, Matthias Zwicker, Abhinav Shrivastava

We study a referential game (a type of signaling game) where two agents communicate with each other via a discrete bottleneck to achieve a common goal.

Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views

no code implementations8 Aug 2021 Zhizhong Han, Xiyang Wang, Yu-Shen Liu, Matthias Zwicker

To mine highly discriminative information from unordered views, HVP performs a novel hierarchical view prediction over a view pair, and aggregates the knowledge learned from the predictions in all view pairs into a global feature.


Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching

1 code implementation8 Aug 2021 Chen Chao, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Our method pushes the neural network to generate a 3D point cloud whose 2D projections match the irregular point supervision from different view angles.

Point Cloud Generation

Transfer Learning for Pose Estimation of Illustrated Characters

1 code implementation4 Aug 2021 Shuhong Chen, Matthias Zwicker

Likewise, a pose estimator for the illustrated character domain would provide a valuable prior for assistive content creation tasks, such as reference pose retrieval and automatic character animation.

Activity Recognition Pose Estimation +3

Neural Radiosity

1 code implementation26 May 2021 Saeed Hadadan, Shuhong Chen, Matthias Zwicker

We introduce Neural Radiosity, an algorithm to solve the rendering equation by minimizing the norm of its residual similar as in traditional radiosity techniques.

Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projections Matching

1 code implementation ICCV 2021 Chao Chen, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Our method pushes the neural network to generate a 3D point cloud whose 2D projections match the irregular point supervision from different view angles.

Point Cloud Generation

Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces

1 code implementation26 Nov 2020 Baorui Ma, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Specifically, we train a neural network to pull query 3D locations to their closest points on the surface using the predicted signed distance values and the gradient at the query locations, both of which are computed by the network itself.

Image Reconstruction Surface Reconstruction

Improved Modeling of 3D Shapes with Multi-view Depth Maps

1 code implementation7 Sep 2020 Kamal Gupta, Susmija Jabbireddy, Ketul Shah, Abhinav Shrivastava, Matthias Zwicker

Our simple encoder-decoder framework, comprised of a novel identity encoder and class-conditional viewpoint generator, generates 3D consistent depth maps.

Image Generation

DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images

no code implementations ICML 2020 Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker

To optimize 3D shape parameters, current renderers rely on pixel-wise losses between rendered images of 3D reconstructions and ground truth images from corresponding viewpoints.

Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions

1 code implementation26 May 2020 Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

According to our experiments under this fine-grained dataset, we find that state-of-the-art methods are significantly limited by the small variance among subcategories in the same category.

3D Shape Classification General Classification +2

LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts

no code implementations18 Mar 2020 Xinhai Liu, Zhizhong Han, Fangzhou Hong, Yu-Shen Liu, Matthias Zwicker

However, due to the irregularity and sparsity in sampled point clouds, it is hard to encode the fine-grained geometry of local regions and their spatial relationships when only using the fixed-size filters and individual local feature integration, which limit the ability to learn discriminative features.

SeqXY2SeqZ: Structure Learning for 3D Shapes by Sequentially Predicting 1D Occupancy Segments From 2D Coordinates

no code implementations ECCV 2020 Zhizhong Han, Guanhui Qiao, Yu-Shen Liu, Matthias Zwicker

To avoid dense and irregular sampling in 3D, we propose to represent shapes using 2D functions, where the output of the function at each 2D location is a sequence of line segments inside the shape.

ORCSolver: An Efficient Solver for Adaptive GUI Layout with OR-Constraints

1 code implementation23 Feb 2020 Yue Jiang, Wolfgang Stuerzlinger, Matthias Zwicker, Christof Lutteroth

OR-constrained (ORC) graphical user interface layouts unify conventional constraint-based layouts with flow layouts, which enables the definition of flexible layouts that adapt to screens with different sizes, orientations, or aspect ratios with only a single layout specification.

Learning Generative Models using Denoising Density Estimators

2 code implementations8 Jan 2020 Siavash A. Bigdeli, Geng Lin, Tiziano Portenier, L. Andrea Dunbar, Matthias Zwicker

Learning probabilistic models that can estimate the density of a given set of samples, and generate samples from that density, is one of the fundamental challenges in unsupervised machine learning.

Denoising Density Estimation

Learning to Generate Dense Point Clouds with Textures on Multiple Categories

1 code implementation22 Dec 2019 Tao Hu, Geng Lin, Zhizhong Han, Matthias Zwicker

In this paper, we propose a novel approach for reconstructing point clouds from RGB images.

3D Reconstruction

SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization

1 code implementation CVPR 2020 Yue Jiang, Dantong Ji, Zhizhong Han, Matthias Zwicker

We propose SDFDiff, a novel approach for image-based shape optimization using differentiable rendering of 3D shapes represented by signed distance functions (SDFs).

 Ranked #1 on Single-View 3D Reconstruction on ShapeNet (using extra training data)

3D Reconstruction Multi-View 3D Reconstruction +1

3D Shape Completion with Multi-view Consistent Inference

no code implementations28 Nov 2019 Tao Hu, Zhizhong Han, Matthias Zwicker

We formulate the regularization term as a consistency loss that encourages geometric consistency among multiple views, while the data term guarantees that the optimized views do not drift away too much from a learned shape descriptor.

L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention

no code implementations2 Aug 2019 Xinhai Liu, Zhizhong Han, Xin Wen, Yu-Shen Liu, Matthias Zwicker

Specifically, L2G-AE employs an encoder to encode the geometry information of multiple scales in a local region at the same time.


ShapeCaptioner: Generative Caption Network for 3D Shapes by Learning a Mapping from Parts Detected in Multiple Views to Sentences

no code implementations31 Jul 2019 Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker

Specifically, ShapeCaptioner aggregates the parts detected in multiple colored views using our novel part class specific aggregation to represent a 3D shape, and then, employs a sequence to sequence model to generate the caption.

Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views

no code implementations18 May 2019 Zhizhong Han, Xinhai Liu, Yu-Shen Liu, Matthias Zwicker

In contrast, we propose a deep neural network, called Parts4Feature, to learn 3D global features from part-level information in multiple views.

Region Proposal

3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention

no code implementations17 May 2019 Zhizhong Han, Xiyang Wang, Chi-Man Vong, Yu-Shen Liu, Matthias Zwicker, C. L. Philip Chen

Then, the content and spatial information of each pair of view nodes are encoded by a novel spatial pattern correlation, where the correlation is computed among latent semantic patterns.

Render4Completion: Synthesizing Multi-View Depth Maps for 3D Shape Completion

no code implementations17 Apr 2019 Tao Hu, Zhizhong Han, Abhinav Shrivastava, Matthias Zwicker

Different from image-to-image translation network that completes each view separately, our novel network, multi-view completion net (MVCN), leverages information from all views of a 3D shape to help the completion of each single view.

Image-to-Image Translation Translation

Smart, Deep Copy-Paste

no code implementations15 Mar 2019 Tiziano Portenier, Qiyang Hu, Paolo Favaro, Matthias Zwicker

In this work, we propose a novel system for smart copy-paste, enabling the synthesis of high-quality results given a masked source image content and a target image context as input.

Understanding the (un)interpretability of natural image distributions using generative models

no code implementations6 Jan 2019 Ryen Krusinga, Sohil Shah, Matthias Zwicker, Tom Goldstein, David Jacobs

Probability density estimation is a classical and well studied problem, but standard density estimation methods have historically lacked the power to model complex and high-dimensional image distributions.

Density Estimation

Learning to Take Directions One Step at a Time

1 code implementation5 Dec 2018 Qiyang Hu, Adrian Wälchli, Tiziano Portenier, Matthias Zwicker, Paolo Favaro

Because items in an image can be animated in arbitrarily many different ways, we introduce as control signal a sequence of motion strokes.

Video Prediction

View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions

no code implementations7 Nov 2018 Zhizhong Han, Mingyang Shang, Yu-Shen Liu, Matthias Zwicker

Intuitively, this memory enables the system to aggregate information that is useful to better solve the view inter-prediction tasks for each shape, and to leverage the memory as a view-independent shape representation.

3D Point Cloud Linear Classification Representation Learning

Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences

no code implementations7 Nov 2018 Zhizhong Han, Mingyang Shang, Xiyang Wang, Yu-Shen Liu, Matthias Zwicker

A recent method employs 3D voxels to represent 3D shapes, but this limits the approach to low resolutions due to the computational cost caused by the cubic complexity of 3D voxels.

3D Shape Representation Cross-Modal Retrieval +2

Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

no code implementations6 Nov 2018 Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features.

3D Part Segmentation 3D Point Cloud Classification +1

Understanding Degeneracies and Ambiguities in Attribute Transfer

no code implementations ECCV 2018 Attila Szabo, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, Paolo Favaro

We study the problem of building models that can transfer selected attributes from one image to another without affecting the other attributes.

Learning to Importance Sample in Primary Sample Space

no code implementations23 Aug 2018 Quan Zheng, Matthias Zwicker

Importance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering.

Specular-to-Diffuse Translation for Multi-View Reconstruction

no code implementations ECCV 2018 Shihao Wu, Hui Huang, Tiziano Portenier, Matan Sela, Danny Cohen-Or, Ron Kimmel, Matthias Zwicker

To alleviate this restriction, we introduce S2Dnet, a generative adversarial network for transferring multiple views of objects with specular reflection into diffuse ones, so that multi-view reconstruction methods can be applied more effectively.

3D Reconstruction Lighting Estimation +3

FaceShop: Deep Sketch-based Face Image Editing

no code implementations24 Apr 2018 Tiziano Portenier, Qiyang Hu, Attila Szabó, Siavash Arjomand Bigdeli, Paolo Favaro, Matthias Zwicker

We present a novel system for sketch-based face image editing, enabling users to edit images intuitively by sketching a few strokes on a region of interest.

Image Manipulation

Disentangling Factors of Variation by Mixing Them

no code implementations CVPR 2018 Qiyang Hu, Attila Szabó, Tiziano Portenier, Matthias Zwicker, Paolo Favaro

We learn our representation without any labeling or knowledge of the data domain, using an autoencoder architecture with two novel training objectives: first, we propose an invariance objective to encourage that encoding of each attribute, and decoding of each chunk, are invariant to changes in other attributes and chunks, respectively; second, we include a classification objective, which ensures that each chunk corresponds to a consistently discernible attribute in the represented image, hence avoiding degenerate feature mappings where some chunks are completely ignored.

General Classification Retrieval

Challenges in Disentangling Independent Factors of Variation

2 code implementations ICLR 2018 Attila Szabó, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, Paolo Favaro

Such models could be used to encode features that can efficiently be used for classification and to transfer attributes between different images in image synthesis.

Image Generation

Image Restoration using Autoencoding Priors

no code implementations29 Mar 2017 Siavash Arjomand Bigdeli, Matthias Zwicker

We propose to leverage denoising autoencoder networks as priors to address image restoration problems.

Denoising Image Restoration +1

Temporally Consistent Motion Segmentation from RGB-D Video

no code implementations16 Aug 2016 Peter Bertholet, Alexandru-Eugen Ichim, Matthias Zwicker

We present a method for temporally consistent motion segmentation from RGB-D videos assuming a piecewise rigid motion model.

Motion Segmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.