Search Results for author:
Found 60 papers, 30 papers with code
Learning a Category-level Object Pose Estimator without Pose Annotations
Instead of using manually annotated images, we leverage diffusion models (e. g., Zero-1-to-3) to generate a set of images under controlled pose differences and propose to learn our object pose estimator with those images.
A Bayesian Approach to OOD Robustness in Image Classification
This enables us to learn a transitional dictionary of vMF kernels that are intermediate between the source and target domains and train the generative model on this dictionary using the annotations on the source domain, followed by iterative refinement.
Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation
We focus on individual locally robust mesh vertex features and iteratively update them based on their proximity to corresponding features in the target domain even when the global pose is not correct.
Ranked #1 on
Unsupervised Domain Adaptation
on OOD-CV
Relightable Neural Actor with Intrinsic Decomposition and Pose Control
Creating a digital human avatar that is relightable, drivable, and photorealistic is a challenging and important problem in Vision and Graphics.
ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering
Real-time rendering of photorealistic and controllable human avatars stands as a cornerstone in Computer Vision and Graphics.
Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Replay-based methods in class-incremental learning (CIL) have attained remarkable success, as replaying the exemplars of old classes can significantly mitigate catastrophic forgetting.
DatasetNeRF: Efficient 3D-aware Data Factory with Generative Radiance Fields
The generated data is applicable across various computer vision tasks, including video segmentation and 3D point cloud segmentation.
3D-Aware Visual Question Answering about Parts, Poses and Occlusions
In this work, we introduce the task of 3D-aware VQA, which focuses on challenging questions that require a compositional reasoning over the 3D structure of visual scenes.
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
Ranked #1 on
Animal Pose Estimation
on Animal3D
3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation
Motivated by the recent success of generative models in rigid object pose estimation, we propose 3D-aware Neural Body Fitting (3DNBF) - an approximate analysis-by-synthesis approach to 3D human pose estimation with SOTA performance and occlusion robustness.
Generating Images with 3D Annotations Using Diffusion Models
With explicit 3D geometry control, we can easily change the 3D structures of the objects in the generated images and obtain ground-truth 3D annotations automatically.
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search
(2) We find regions in the latent space that lead to distorted images independent of the text prompt, suggesting that parts of the latent space are not well-structured.
Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis
Human vision demonstrates higher robustness than current AI algorithms under out-of-distribution scenarios.
Robust Category-Level 3D Pose Estimation from Synthetic Data
In this work, we aim to narrow the performance gap between models trained on synthetic data and few real images and fully supervised models trained on large-scale data.
Robust 3D-aware Object Classification via Discriminative Render-and-Compare
Using differentiable rendering, we estimate the 3D object pose by minimizing the reconstruction error between the mesh and the feature representation of the target image.
General Neural Gauge Fields
In this work, we extend this problem to a general paradigm with a taxonomy of discrete \& continuous cases, and develop a learning framework to jointly optimize gauge transformations and neural fields.
OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
Enhancing the robustness of vision algorithms in real-world scenarios is challenging.
PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation
We introduce a learning-based testing method, termed PoseExaminer, that automatically diagnoses HPS algorithms by searching over the parameter space of human pose images to find the failure modes.
Scene-Aware 3D Multi-Human Motion Capture from a Single Camera
In this work, we consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera.
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning
Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization.
State of the Art in Dense Monocular Non-Rigid 3D Reconstruction
3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics.
HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow
Through this, we demonstrate the quality of our probabilistic reconstruction and show that explicit ambiguity modeling is better-suited for this challenging problem.
Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features
We consider the problem of category-level 6D pose estimation from a single RGB image.
VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis
The Gaussian reconstruction kernels have been proposed by Westover (1990) and studied by the computer graphics community back in the 90s, which gives an alternative representation of object 3D geometry from meshes and point clouds.
SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering
By swapping the context object features, the model reliance on context can be suppressed effectively.
Multimodal Image Synthesis and Editing: The Generative AI Era
With superb power in modeling the interaction among multimodal information, multimodal image synthesis and editing has become a hot research topic in recent years.
PartImageNet: A Large, High-Quality Dataset of Parts
To help address this problem, we propose PartImageNet, a large, high-quality dataset with part segmentation annotations.
OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors.
Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose
The model is initialized from a few labelled images and is subsequently used to synthesize feature representations of unseen 3D views.
A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation
It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset.
Locally Enhanced Self-Attention: Combining Self-Attention and Convolution as Local and Context Terms
Self-Attention has become prevalent in computer vision models.
Robust Model-based Face Reconstruction through Weakly-Supervised Outlier Segmentation
The main challenge is that the model fitting and the outlier segmentation are mutually dependent on each other, and need to be inferred jointly.
Ranked #4 on
3D Face Reconstruction
on NoW Benchmark
Simulated Adversarial Testing of Face Recognition Models
In this work, we propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner in order to find weaknesses in the model before deploying it in critical scenarios.
Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
In particular, we decouple the training of the representation and the classifier, and systematically investigate the effects of different data re-sampling techniques when training the whole network including a classifier as well as fine-tuning the feature extractor only.
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation
To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation.
Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles
We believe our dataset provides a rich testbed to study UDA for part segmentation and will help to significantly push forward research in this area.
TransFG: A Transformer Architecture for Fine-grained Recognition
Fine-grained visual classification (FGVC) which aims at recognizing objects from subcategories is a very challenging task due to the inherently subtle inter-class differences.
Ranked #4 on
Fine-Grained Image Classification
on CUB-200-2011
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Additionally, current approaches that deal with forgetting ignore the problem of catastrophic remembering, i. e. the worsening ability to discriminate between data from different tasks.
Ranked #1 on
Continual Learning
on ImageNet-50 (5 tasks)
NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation
Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image.
CORL: Compositional Representation Learning for Few-Shot Classification
In particular, during meta-learning, we train a knowledge base that consists of a dictionary of component representations and a dictionary of component activation maps that encode common spatial activation patterns of components.
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion
The improved segmentation masks are, in turn, integrated into the network in a top-down manner to improve the image classification.
Robustness Out of the Box: Compositional Representations Naturally Defend Against Black-Box Patch Attacks
Second, we find that compositional deep networks, which have part-based representations that lead to innate robustness to natural occlusion, are robust to patch attacks on PASCAL3D+ and the German Traffic Sign Recognition Benchmark, without adversarial training.
Unsupervised Part Discovery via Feature Alignment
Specifically, given a training image, we find a set of similar images that show instances of the same object category in the same pose, through an affine alignment of their corresponding feature maps.
Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model
Moreover, by leveraging an outlier process, Bayesian models can further generalize out-of-distribution to segment partially occluded objects and to predict their amodal object boundaries.
CoKe: Localized Contrastive Learning for Robust Keypoint Detection
In this paper, we introduce a contrastive learning framework for keypoint detection (CoKe).
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion
The structure of the compositional model enables CompositionalNets to decompose images into objects and context, as well as to further decompose object representations in terms of individual parts and the objects' pose.
Robust Object Detection under Occlusion with Context-Aware CompositionalNets
In this work, we propose to overcome two limitations of CompositionalNets which will enable them to detect partially occluded objects: 1) CompositionalNets, as well as other DCNN architectures, do not explicitly separate the representation of the context from the object itself.
PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
PatchAttack induces misclassifications by superimposing small textured patches on the input image.
Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion
Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion.
Localizing Occluders with Compositional Convolutional Networks
Our experimental results demonstrate that the proposed extensions increase the model's performance at localizing occluders as well as at classifying partially occluded objects.
TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
Despite deep convolutional neural networks' great success in object classification, it suffers from severe generalization performance drop under occlusion due to the inconsistency between training and testing data.
3D Morphable Face Models -- Past, Present and Future
In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed.
Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion
In this work, we combine DCNNs and compositional object models to retain the best of both approaches: a discriminative model that is robust to partial occlusion and mask attacks.
SkelNetOn 2019: Dataset and Challenge on Deep Learning for Geometric Shape Understanding
We present SkelNetOn 2019 Challenge and Deep Learning for Geometric Shape Understanding workshop to utilize existing and develop novel deep learning architectures for shape understanding.
Informed MCMC with Bayesian Neural Networks for Facial Image Analysis
Computer vision tasks are difficult because of the large variability in the data that is induced by changes in light, background, partial occlusion as well as the varying pose, texture, and shape of objects.
Can Synthetic Faces Undo the Damage of Dataset Bias to Face Recognition and Facial Landmark Detection?
We observe the following positive effects for face recognition and facial landmark detection tasks: 1) Priming with synthetic face images improves the performance consistently across all benchmarks because it reduces the negative effects of biases in the training data.
Training Deep Face Recognition Systems with Synthetic Data
In our experiments with an off-the-shelf face recognition software we observe the following phenomena: 1) The amount of real training data needed to train competitive deep face recognition systems can be reduced significantly.
Empirically Analyzing the Effect of Dataset Biases on Deep Face Recognition Systems
4) We uncover a main limitation of current DCNN architectures, which is the difficulty to generalize when different identities to not share the same pose variation.
Greedy Structure Learning of Hierarchical Compositional Models
In this work, we consider the problem of learning a hierarchical generative model of an object from a set of images which show examples of the object in the presence of variable background clutter.
