Search Results for author: Jan Hendrik Metzen

Found 30 papers, 9 papers with code

Identification of Fine-grained Systematic Errors via Controlled Scene Generation

no code implementations10 Apr 2024 Valentyn Boreiko, Matthias Hein, Jan Hendrik Metzen

Our approach, BEV2EGO, allows for a realistic generation of the complete scene with road-contingent control that maps 2D bird's-eye view (BEV) scene configurations to a first-person view (EGO).

Autonomous Driving Object +3

AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models

no code implementations28 Sep 2023 Jan Hendrik Metzen, Piyapat Saranrittichai, Chaithanya Kumar Mummadi

We show that AutoCLIP outperforms baselines across a broad range of vision-language models, datasets, and prompt templates consistently and by up to 3 percent point accuracy.

Image Classification Language Modelling +1

Identifying Systematic Errors in Object Detectors with the SCROD Pipeline

no code implementations23 Sep 2023 Valentyn Boreiko, Matthias Hein, Jan Hendrik Metzen

Moreover, our framework introduces an evaluation setting that can serve as a benchmark for similar pipelines.


Identification of Systematic Errors of Image Classifiers on Rare Subgroups

no code implementations ICCV 2023 Jan Hendrik Metzen, Robin Hutmacher, N. Grace Hua, Valentyn Boreiko, Dan Zhang

Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data.

Adversarial Attack Fairness

Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks

1 code implementation NeurIPS 2021 Maksym Yatsura, Jan Hendrik Metzen, Matthias Hein

We demonstrate that plugging the learned controller into the attack consistently improves its black-box robustness estimate in different query regimes by up to 20% for a wide range of different models with black-box access.


Bag of Tricks for Neural Architecture Search

no code implementations8 Jul 2021 Thomas Elsken, Benedikt Staffler, Arber Zela, Jan Hendrik Metzen, Frank Hutter

While neural architecture search methods have been successful in previous years and led to new state-of-the-art performance on various problems, they have also been criticized for being unstable, being highly sensitive with respect to their hyperparameters, and often not performing better than random search.

Neural Architecture Search

Does enhanced shape bias improve neural network robustness to common corruptions?

no code implementations ICLR 2021 Chaithanya Kumar Mummadi, Ranjitha Subramaniam, Robin Hutmacher, Julien Vitay, Volker Fischer, Jan Hendrik Metzen

We conclude that the data augmentation caused by style-variation accounts for the improved corruption robustness and increased shape bias is only a byproduct.

Data Augmentation

Efficient Certified Defenses Against Patch Attacks on Image Classifiers

no code implementations ICLR 2021 Jan Hendrik Metzen, Maksym Yatsura

Adversarial patches pose a realistic threat model for physical world attacks on autonomous systems via their perception component.

Meta Adversarial Training against Universal Patches

1 code implementation27 Jan 2021 Jan Hendrik Metzen, Nicole Finnie, Robin Hutmacher

However, tailoring adversarial training to universal patches is computationally expensive since the optimal universal patch depends on the model weights which change during training.

Autonomous Driving Image Classification +1

Adversarial and Natural Perturbations for General Robustness

no code implementations3 Oct 2020 Sadaf Gulshad, Jan Hendrik Metzen, Arnold Smeulders

In this paper we aim to explore the general robustness of neural network classifiers by utilizing adversarial as well as natural perturbations.

Meta-Learning of Neural Architectures for Few-Shot Learning

2 code implementations CVPR 2020 Thomas Elsken, Benedikt Staffler, Jan Hendrik Metzen, Frank Hutter

The recent progress in neural architecture search (NAS) has allowed scaling the automated design of neural architectures to real-world domains, such as object detection and semantic segmentation.

Few-Shot Learning Neural Architecture Search +3

Understanding Misclassifications by Attributes

1 code implementation15 Oct 2019 Sadaf Gulshad, Zeynep Akata, Jan Hendrik Metzen, Arnold Smeulders

We study the changes in attributes for clean as well as adversarial images in both standard and adversarially robust networks.

Interpreting Adversarial Examples with Attributes

1 code implementation17 Apr 2019 Sadaf Gulshad, Jan Hendrik Metzen, Arnold Smeulders, Zeynep Akata

Deep computer vision systems being vulnerable to imperceptible and carefully crafted noise have raised questions regarding the robustness of their decisions.

Attribute General Classification

Defending Against Universal Perturbations With Shared Adversarial Training

no code implementations ICCV 2019 Chaithanya Kumar Mummadi, Thomas Brox, Jan Hendrik Metzen

Classifiers such as deep neural networks have been shown to be vulnerable against adversarial perturbations on problems with high-dimensional input space.

Image Classification Semantic Segmentation

Neural Architecture Search: A Survey

1 code implementation16 Aug 2018 Thomas Elsken, Jan Hendrik Metzen, Frank Hutter

Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation.

Machine Translation Neural Architecture Search +3

Scaling provable adversarial defenses

4 code implementations NeurIPS 2018 Eric Wong, Frank R. Schmidt, Jan Hendrik Metzen, J. Zico Kolter

Recent work has developed methods for learning deep network classifiers that are provably robust to norm-bounded adversarial perturbation; however, these methods are currently only possible for relatively small feedforward networks.

Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution

no code implementations ICLR 2019 Thomas Elsken, Jan Hendrik Metzen, Frank Hutter

Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts.

Neural Architecture Search

Universality, Robustness, and Detectability of Adversarial Perturbations under Adversarial Training

no code implementations ICLR 2018 Jan Hendrik Metzen

While adversarial training improves the robustness of classifiers against such adversarial perturbations, it leaves classifiers sensitive to them on a non-negligible fraction of the inputs.

Universal Adversarial Perturbations Against Semantic Image Segmentation

no code implementations ICCV 2017 Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, Volker Fischer

We show empirically that there exist barely perceptible universal noise patterns which result in nearly the same predicted segmentation for arbitrary inputs.

Image Classification Image Segmentation +2

Adversarial Examples for Semantic Image Segmentation

no code implementations3 Mar 2017 Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, Thomas Brox

Machine learning methods in general and Deep Neural Networks in particular have shown to be vulnerable to adversarial perturbations.

BIG-bench Machine Learning General Classification +4

On Detecting Adversarial Perturbations

1 code implementation14 Feb 2017 Jan Hendrik Metzen, Tim Genewein, Volker Fischer, Bastian Bischoff

In this work, we propose to augment deep neural networks with a small "detector" subnetwork which is trained on the binary classification task of distinguishing genuine data from data containing adversarial perturbations.

Adversarial Attack Binary Classification +1

Minimum Regret Search for Single- and Multi-Task Optimization

1 code implementation2 Feb 2016 Jan Hendrik Metzen

We propose minimum regret search (MRS), a novel acquisition function for Bayesian optimization.

Bayesian Optimization

Active Contextual Entropy Search

no code implementations13 Nov 2015 Jan Hendrik Metzen

Contextual policy search allows adapting robotic movement primitives to different situations.

Bayesian Optimization

Cannot find the paper you are looking for? You can Submit a new open access paper.