Search Results for author:
Found 198 papers, 83 papers with code
Towards Automated Testing and Robustification by Semantic Adversarial Data Generation
Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector.
HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentation
This work studies semantic segmentation under the domain generalization setting, where a model is trained only on the source domain and tested on the unseen target domain.
Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding
Masked signal modeling has greatly advanced self-supervised pre-training for language and 2D images.
Continual Detection Transformer for Incremental Object Detection
Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories.
Class-Incremental Exemplar Compression for Class-Incremental Learning
Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget.
Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data
Such a schedule results in a constant `task switching' between an emphasis on instance discrimination and group-wise discrimination and thereby ensures that the model learns both group-wise features, as well as instance-specific details.
Using Explanations to Guide Models
To gain a better understanding of which model-guiding approaches actually transfer to more challenging real-world datasets, in this work we conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets, and show that model guidance can sometimes even improve model performance.
Better Understanding Differences in Attribution Methods via Systematic Evaluations
Finally, we propose a post-processing smoothing step that significantly improves the performance of some attribution methods, and discuss its applicability.
Robustifying Token Attention for Vision Transformers
Interestingly, we observe that the attention mechanism of ViTs tends to rely on few important tokens, a phenomenon we call token overfocusing.
A Meta-Learning Approach to Predicting Performance and Data Requirements
The PPL improves the performance estimation on average by 37% across 16 classification and 33% across 10 detection datasets, compared to the power law.
No One Left Behind: Real-World Federated Class-Incremental Learning
To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model to address local and global catastrophic forgetting on old categories, which is a pioneering work to explore a global class-incremental model in the FL feld.
SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning
The critical challenge of Semi-Supervised Learning (SSL) is how to effectively leverage the limited labeled data and massive unlabeled data to improve the model's generalization performance.
Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences
The image sequences are aligned with a total of 12K stories which were collected via crowdsourcing given the image sequences and a set of grounded characters from the corresponding image sequence.
Holistically Explainable Vision Transformers
Transformers increasingly dominate the machine learning landscape across many tasks and domains, which increases the importance for understanding their outputs.
DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets
However, due to the sparse characteristics of point clouds, it is non-trivial to apply a standard transformer on sparse points.
Ranked #1 on
3D Object Detection
on waymo cyclist
RMM: Reinforced Memory Management for Class-Incremental Learning
Class-Incremental Learning (CIL) [40] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase.
Online Hyperparameter Optimization for Class-Incremental Learning
Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase.
Learning by Sorting: Self-supervised Learning with Group Ordering Constraints
Building on the recent success of differentiable sorting algorithms, group ordering constraints enforce that the distances of all positive samples (a positive group) are smaller than the distances of all negative images (a negative group); thus, enforcing positive samples to gather around an anchor.
Weakly-Supervised Domain Adaptive Semantic Segmentation With Prototypical Contrastive Learning
In this work, we propose a common framework to use different weak labels, e. g. image, point and coarse labels from target domain to reduce this performance gap.
Improving Robustness of Vision Transformers by Reducing Sensitivity To Patch Corruptions
Despite their success, vision transformers still remain vulnerable to image corruptions, such as noise or blur.
Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation
For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets.
Discovering Class-Specific GAN Controls for Semantic Image Synthesis
Prior work has extensively studied the latent space structure of GANs for unconditional image synthesis, enabling global editing of generated images by the unsupervised discovery of interpretable latent directions.
An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learning
While standard SSL assumes uniform data distribution, we consider a more realistic and challenging setting called imbalanced SSL, where imbalanced class distributions occur in both labeled and unlabeled data.
Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts
Thus, we propose to perturb the channel statistics of source domain features to synthesize various latent styles, so that the trained deep model can perceive diverse potential domains and generalizes well even without observations of target domain data in training.
Motion Transformer with Global Intention Localization and Local Movement Refinement
Predicting multimodal future behavior of traffic participants is essential for robotic vehicles to make safe decisions.
Leveraging Self-Supervised Training for Unintentional Action Recognition
To enhance representations via self-supervised training for the task of unintentional action recognition we propose temporal transformations, called Temporal Transformations of Inherent Biases of Unintentional Actions (T2IBUA).
TeST: Test-time Self-Training under Distribution Shift
We find that TeST sets the new state-of-the art for test-time domain adaptation algorithms.
MTR-A: 1st Place Solution for 2022 Waymo Open Dataset Challenge -- Motion Prediction
In this report, we present the 1st place solution for motion prediction track in 2022 Waymo Open Dataset Challenges.
ComplETR: Reducing the cost of annotations for object detection in dense scenes with vision transformers
Combined with a matching loss, it can effectively find objects that are similar to the input patch and complete the missing annotations.
MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes
3D human motion capture from monocular RGB images respecting interactions of a subject with complex and possibly deformable environments is a very challenging, ill-posed and under-explored problem.
USB: A Unified Semi-supervised Learning Benchmark for Classification
We further provide the pre-trained versions of the state-of-the-art neural models for CV tasks to make the cost affordable for further tuning.
Assaying Out-Of-Distribution Generalization in Transfer Learning
Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e. g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations.
Adversarial Robustness
Out-of-Distribution Generalization
+1
SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation
Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems.
Towards Better Understanding Attribution Methods
Finally, we propose a post-processing smoothing step that significantly improves the performance of some attribution methods, and discuss its applicability.
B-cos Networks: Alignment is All We Need for Interpretability
We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training.
FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning
Semi-supervised Learning (SSL) has witnessed great success owing to the impressive performances brought by various methods based on pseudo labeling and consistency regularization.
Bi-level Alignment for Cross-Domain Crowd Counting
The main challenge for this task is to achieve high-quality manual annotations on a large amount of training data.
ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Training
We present a method to train a classification system that achieves paragon performance in both error rate and NFR, at the inference cost of a single model.
On Fragile Features and Batch Normalization in Adversarial Training
In order to shed light on the role of BN in adversarial training, we investigate to what extent the expressiveness of BN can be used to robustify fragile features in comparison to random features.
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #8 on
3D Object Detection
on ScanNetV2
Attribute Prototype Network for Any-Shot Learning
While a visual-semantic embedding layer learns global features, local features are learned through an attribute prototype network that simultaneously regresses and decorrelates attributes from intermediate features.
Omni-DETR: Omni-Supervised Object Detection with Transformers
This is enabled by a unified architecture, Omni-DETR, based on the recent progress on student-teacher framework and end-to-end transformer based object detection.
Ranked #12 on
Semi-Supervised Object Detection
on COCO 2% labeled data
VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learning
Our model visually divides a set of images from seen classes into clusters of local image regions according to their visual similarity, and further imposes their class discrimination and semantic relatedness.
A Unified Query-based Paradigm for Point Cloud Understanding
This is achieved by introducing an intermediate representation, i. e., Q-representation, in the querying stage to serve as a bridge between the embedding stage and task heads.
Ranked #3 on
Semantic Segmentation
on S3DIS
Improving Robustness by Enhancing Weak Subnets
We show that EWS greatly improves both robustness against corrupted images as well as accuracy on clean data.
PoseTrack21: A Dataset for Person Search, Multi-Object Tracking and Multi-Person Pose Tracking
Current research evaluates person search, multi-object tracking and multi-person pose estimation as separate tasks and on different datasets although these tasks are very akin to each other and comprise similar sub-tasks, e. g. person detection or appearance-based association of detected persons.
Revisiting Consistency Regularization for Semi-Supervised Learning
Generally, the aim is to train a model that is invariant to various data augmentations.
CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning
In this paper, we propose a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL.
Keypoint Message Passing for Video-based Person Re-Identification
Video-based person re-identification (re-ID) is an important technique in visual surveillance systems which aims to match video snippets of people captured by different cameras.
Representation Learning
Video-Based Person Re-Identification
Seeking Similarities over Differences: Similarity-based Domain Alignment for Adaptive Object Detection
In order to robustly deploy object detectors across a wide range of scenarios, they should be adaptable to shifts in the input distribution without the need to constantly annotate new data.
Multi-Source Unsupervised Domain Adaptation
Object Detection
+1
Wakening Past Concepts without Past Data: Class-incremental Learning from Placebos
However, we empirically observe that this both harms learning of new classes and also underperforms to distil old class knowledge from the previous phase model.
Optimising for Interpretability: Convolutional Dynamic Alignment Networks
As a result, CoDA Nets model the classification prediction through a series of input-dependent linear transformations, allowing for linear decomposition of the output into individual input contributions.
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting
Both generalized and incremental few-shot learning have to deal with three major challenges: learning novel classes from only few samples per class, preventing catastrophic forgetting of base classes, and classifier calibration across novel and base classes.
Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centers
In this work, we propose Euro-PVI, a dataset of pedestrian and bicyclist trajectories.
Ranked #1 on
Pedestrian Trajectory Prediction
on Euro-PVI
Learning Spatially-Variant MAP Models for Non-Blind Image Deblurring
The classical maximum a-posteriori (MAP) framework for non-blind image deblurring requires defining suitable data and regularization terms, whose interplay yields the desired clear image through optimization.
Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators
Moreover, we present a novel adversarial bit error attack and are able to obtain robustness against both targeted and untargeted bit-level attacks.
Relating Adversarially Robust Generalization to Flat Minima
To this end, we propose average- and worst-case metrics to measure flatness in the robust loss landscape and show a correlation between good robust generalization and flatness.
Convolutional Dynamic Alignment Networks for Interpretable Classifications
Given the alignment of the DAUs, the resulting contribution maps align with discriminative input patterns.
Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring
We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning.
Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes
The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks.
You Only Need Adversarial Supervision for Semantic Image Synthesis
By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically-aware discriminator feedback, we are able to synthesize images of higher fidelity with better alignment to their input label maps, making the use of the perceptual loss superfluous.
PoseTrackReID: Dataset Description
For that reason, we present PoseTrackReID, a large-scale dataset for multi-person pose tracking and video-based person re-ID.
Diverse and Relevant Visual Storytelling with Scene Graph Embeddings
A problem in automatically generated stories for image sequences is that they use overly generic vocabulary and phrase structure and fail to match the distributional characteristics of human-generated text.
Adaptive Aggregation Networks for Class-Incremental Learning
Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase.
Haar Wavelet based Block Autoregressive Flows for Trajectories
This yields an exact inference method that models trajectories at different spatio-temporal resolutions in a hierarchical manner.
Synthetic Convolutional Features for Improved Semantic Segmentation
Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss.
Long-Term Anticipation of Activities with Cycle Consistency
With the success of deep learning methods in analyzing activities in videos, more attention has recently been focused towards anticipating future activities.
Attribute Prototype Network for Zero-Shot Learning
As an additional benefit, our model points to the visual evidence of the attributes in an image, e. g. for the CUB dataset, confirming the improved attribute localization ability of our image representation.
Kinematic 3D Object Detection in Monocular Video
In this work, we propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization.
Ranked #11 on
Monocular 3D Object Detection
on KITTI Cars Moderate
(using extra training data)
Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
Few-shot learning aims to recognize novel classes from a few examples.
Bit Error Robustness for Energy-Efficient DNN Accelerators
Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights.
Adversarial Training against Location-Optimized Adversarial Patches
Then, we apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB.
Normalizing Flows with Multi-Scale Autoregressive Priors
Flow-based generative models are an important class of exact inference models that admit efficient inference and sampling for image synthesis.
Long-Tailed Recognition Using Class-Balanced Experts
Deep learning enables impressive performance in image recognition using large-scale artificially-balanced datasets.
Ranked #17 on
Long-tail Learning
on Places-LT
A U-Net Based Discriminator for Generative Adversarial Networks
The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics, enabling the generator to synthesize images with varying structure, appearance and levels of detail, maintaining global and local realism.
Ranked #1 on
Image Generation
on CelebA 128x128
Mnemonics Training: Multi-Class Incremental Learning without Forgetting
However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones.
Analyzing the Dependency of ConvNets on Spatial Information
In this paper, we are pushing the envelope and aim to further investigate the reliance on spatial information.
Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation
Today's success of state of the art methods for semantic segmentation is driven by large datasets.
Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
Our confidence-calibrated adversarial training (CCAT) tackles this problem by biasing the model towards low confidence predictions on adversarial examples.
Meta-Transfer Learning through Hard Tasks
In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks.
"Best-of-Many-Samples" Distribution Matching
Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.
Confidence-Calibrated Adversarial Training: Towards Robust Models Generalizing Beyond the Attack Used During Training
Adversarial training is the standard to train models robust against adversarial examples.
``"Best-of-Many-Samples" Distribution Matching
Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.
Conditional Flow Variational Autoencoders for Structured Sequence Prediction
Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world.
Ranked #10 on
Trajectory Prediction
on Stanford Drone
Interpretability Beyond Classification Output: Semantic Bottleneck Networks
Additionally we show how the activations of the SB-Layer can be used for both the interpretation of failure cases of the network as well as for confidence prediction of the resulting output.
Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
We find such passive defenses ineffective against DNN stealing attacks.
Learning to Self-Train for Semi-Supervised Few-Shot Classification
On each task, we train a few-shot model to predict pseudo labels for unlabeled data, and then iterate the self-training steps on labeled and pseudo-labeled data with each step followed by fine-tuning.
A Novel BiLevel Paradigm for Image-to-Image Translation
Image-to-image (I2I) translation is a pixel-level mapping that requires a large number of paired training data and often suffers from the problems of high diversity and strong category bias in image scenes.
An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning
"Empirical" means that the hyperparameters, e. g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data.
f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
When labeled training data is scarce, a promising data augmentation approach is to generate visual features of unknown classes using their attributes.
Ranked #3 on
Generalized Zero-Shot Learning
on SUN Attribute
Learning Decision Trees Recurrently Through Communication
Integrated interpretability without sacrificing the prediction accuracy of decision making algorithms has the potential of greatly improving their value to the user.
Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation
We propose a method to quantify the sensitivity of black-box vision models to visual context by editing images to remove selected objects and measuring the response of the target models.
Meta-Transfer Learning for Few-Shot Learning
In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks.
Knockoff Nets: Stealing Functionality of Black-Box Models
We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" with queried image-prediction pairs.
Disentangling Adversarial Robustness and Generalization
A recent hypothesis even states that both robust and accurate models are impossible, i. e., adversarial robustness and generalization are conflicting goals.
Parameter-Free Spatial Attention Network for Person Re-Identification
We propose a novel architecture for Person Re-Identification, based on a novel parameter-free spatial attention layer introducing spatial relations among the feature map activations back to the model.
Ranked #20 on
Person Re-Identification
on DukeMTMC-reID
Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods
For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence.
Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation
Direct prediction of 3D body pose and shape remains a challenge even for highly parameterized deep learning models.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Diverse Conditional Image Generation by Stochastic Regression with Latent Drop-Out Codes
Recent advances in Deep Learning and probabilistic modeling have led to strong improvements in generative models for images.
Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Ranked #6 on
Human Pose Forecasting
on Human3.6M
(ADE metric)
Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Adversarial Scene Editing: Automatic Object Removal from Weak Supervision
While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.
Occluded Pedestrian Detection Through Guided Attention in CNNs
In this paper, we aim to propose a simple and compact method based on the FasterRCNN architecture for occluded pedestrian detection.
Accurate and Diverse Sampling of Sequences Based on a âBest of Manyâ Sample Objective
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning
At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data.
Learning to Refine Human Pose Estimation
Multi-person pose estimation in images and videos is an important yet challenging task with many applications.
A Hybrid Model for Identity Obfuscation by Face Replacement
As more and more personal photos are shared and tagged in social media, avoiding privacy risks such as unintended recognition becomes increasingly challenging.
Video Object Segmentation with Language Referring Expressions
We show that our language-supervised approach performs on par with the methods which have access to a pixel-level mask of the target object on DAVIS'16 and is competitive to methods using scribbles on the challenging DAVIS'17 dataset.
Ranked #1 on
Video Object Segmentation
on DAVIS 2017
(mIoU metric)
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
We propose a multimodal approach to explanation, and argue that the two modalities provide complementary explanatory strengths.
Disentangled Person Image Generation
Generating novel, yet realistic, images of persons is a challenging task due to the complex interplay between the different image factors, such as the foreground, background and pose information.
Ranked #2 on
Gesture-to-Gesture Translation
on Senz3D
Feature Generating Networks for Zero-Shot Learning
Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task.
Ranked #5 on
Generalized Zero-Shot Learning
on SUN Attribute
Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images
Images convey a broad spectrum of personal information.
Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
Our experimental results show that it is indeed possible to predict people trajectories at the desired time horizons and that our uncertainty estimates are informative of the prediction error.
Ranked #5 on
Trajectory Prediction
on JAAD
Natural and Effective Obfuscation by Head Inpainting
As more and more personal photos are shared online, being able to obfuscate identities in such photos is becoming a necessity for privacy protection.
Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
We also introduce a multimodal methodology for generating visual and textual explanations simultaneously.
$A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation
In this paper, we propose an automatic method, called Adversarial Author Attribute Anonymity Neural Translation ($A^4NT$), to combat such text-based adversaries.
Towards Reverse-Engineering Black-Box Neural Networks
On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model.
PoseTrack: A Benchmark for Human Pose Estimation and Tracking
In this work, we aim to further advance the state of the art by establishing "PoseTrack", a new large-scale benchmark for video-based human pose estimation and articulated tracking, and bringing together the community of researchers working on visual human analysis.
Ranked #3 on
Multi-Person Pose Estimation
on PoseTrack2017
Person Recognition in Personal Photo Collections
Person recognition in social media photos sets new challenges for computer vision, including non-cooperative subjects (e. g. backward viewpoints, unusual poses) and great changes in appearance.
Learning Dilation Factors for Semantic Segmentation of Street Scenes
In this paper, we present an approach for learning dilation parameters adaptively per channel, consistently improving semantic segmentation results on street-scene datasets like Cityscapes and Camvid.
Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
Due to the importance of zero-shot learning, i. e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily.
Multiple People Tracking by Lifted Multicut and Person Re-Identification
This allows us to reward tracks that assign detections of similar appearance to the same person in a way that does not introduce implausible solutions.
Joint Graph Decomposition & Node Labeling: Problem, Algorithms, Applications
In order to find feasible solutions efficiently, we define two local search algorithms that converge monotonously to a local optimum, offering a feasible solution at any time.
Pose Guided Person Image Generation
This paper proposes the novel Pose Guided Person Generation Network (PG$^2$) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose.
Ranked #3 on
Gesture-to-Gesture Translation
on Senz3D
Learning non-maximum suppression
Object detectors have hugely profited from moving towards an end-to-end learning paradigm: proposals, features, and the classifier becoming one neural network improved results two-fold on general object detection.
A Domain Based Approach to Social Relation Recognition
Social relations are the foundation of human daily life.
Generating Descriptions with Grounded and Co-Referenced People
At training time, we first learn how to localize characters by relating their visual appearance to mentions in the descriptions via a semi-supervised approach.
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
While strong progress has been made in image captioning over the last years, machine and human captions are still quite distinct.
Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images
Third, we propose models that predict user specific privacy score from images in order to enforce the users' privacy preferences.
Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective
We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.
Lucid Data Dreaming for Video Object Segmentation
Our approach is suitable for both single and multiple object segmentation.
Zero-Shot Learning -- The Good, the Bad and the Ugly
Due to the importance of zero-shot learning, the number of proposed approaches has increased steadily recently.
CityPersons: A Diverse Dataset for Pedestrian Detection
Convnets have enabled significant progress in pedestrian detection recently, but there are still open questions regarding suitable architectures and training data.
Ranked #14 on
Pedestrian Detection
on Caltech
Exploiting saliency for object segmentation from image level labels
We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.
Ranked #25 on
Semantic Segmentation
on PASCAL VOC 2012 val
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)
Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which led to their decisions.
Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification
In particular, we find that it is possible to obtain effective multilabel classifiers on Pascal VOC using a single label per image for training, while the gap between multiclass and multilabel methods on MS COCO is more significant.
Learning Video Object Segmentation from Static Images
Inspired by recent advances of deep learning in instance segmentation and object tracking, we introduce video object segmentation problem as a concept of guided instance segmentation.
Ranked #6 on
Semi-Supervised Video Object Segmentation
on YouTube
ArtTrack: Articulated Multi-person Tracking in the Wild
In this paper we propose an approach for articulated tracking of multiple people in unconstrained videos.
Ranked #7 on
Keypoint Detection
on MPII Multi-Person
Gaze Embeddings for Zero-Shot Image Classification
Zero-shot image classification using auxiliary information, such as attributes describing discriminative object properties, requires time-consuming annotation by domain experts.
Long-Term Image Boundary Prediction
Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception.
Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications
In order to find feasible solutions efficiently, we define two local search algorithms that converge monotonously to a local optimum, offering a feasible solution at any time.
Learning What and Where to Draw
Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers.
Ranked #11 on
Text-to-Image Generation
on CUB
(using extra training data)
3D Pictorial Structures Revisited: Multiple Human Pose Estimation
We address the problem of 3D pose estimation of multiple humans from multiple views.
Ranked #15 on
3D Multi-Person Pose Estimation
on Campus
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.
Multi-Person Tracking by Multicut and Deep Matching
In [1], we proposed a graph-based formulation that links and clusters person hypotheses over time by solving a minimum cost subgraph multicut problem.
Faceless Person Recognition; Privacy Implications in Social Media
As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy.
A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects
Recently, Minimum Cost Multicut Formulations have been proposed and proven to be successful in both motion trajectory segmentation and multi-target tracking scenarios.
Learning Deep Representations of Fine-grained Visual Descriptions
State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information.
Generative Adversarial Text to Image Synthesis
Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal.
Improved Image Boundaries for Better Video Segmentation
Graph-based video segmentation methods rely on superpixels as starting point.
Movie Description
In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions.
DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model
The goal of this paper is to advance the state-of-the-art of articulated pose estimation in scenes with multiple people.
Ranked #1 on
Multi-Person Pose Estimation
on WAF
The Cityscapes Dataset for Semantic Urban Scene Understanding
Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications.
Latent Embeddings for Zero-shot Classification
We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image.
Multi-Cue Zero-Shot Learning with Strong Supervision
A promising research direction is zero-shot learning, which does not require any training data to recognize new classes, but rather relies on some form of auxiliary information describing the new classes.
Generating Visual Explanations
Clearly explaining a rationale for a classification decision to an end-user can be as important as the decision itself.
Simple Does It: Weakly Supervised Instance and Semantic Segmentation
Semantic labelling and instance segmentation are two tasks that require particularly costly annotations.
Ranked #1 on
Semantic Segmentation
on PASCAL VOC 2012 val
(Mean IoU metric)
How Far are We from Solving Pedestrian Detection?
We enable our analysis by creating a human baseline for pedestrian detection (over the Caltech dataset), and by manually clustering the recurrent errors of a top detector.
Loss Functions for Top-k Error: Analysis and Insights
In the experiments, we compare on various datasets all of the proposed and established methods for top-k error optimization.
Weakly Supervised Object Boundaries
State-of-the-art learning based boundary detection methods require extensive training data.
Ranked #2 on
Edge Detection
on SBD
DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation
This paper considers the task of articulated human pose estimation of multiple people in real world images.
Ranked #2 on
Multi-Person Pose Estimation
on WAF
Top-k Multiclass SVM
Class ambiguity is typical in image classification problems with a large number of classes.
A convnet for non-maximum suppression
Non-maximum suppression (NMS) is used in virtually all state-of-the-art object detection pipelines.
Efficient Output Kernel Learning for Multiple Tasks
The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other.
Grounding of Textual Phrases in Images by Reconstruction
We propose a novel approach which learns grounding by reconstructing a given phrase using an attention mechanism, which can be either latent or optimized directly.
Ranked #12 on
Phrase Grounding
on Flickr30k Entities Test
