Search Results for author:
Found 102 papers, 49 papers with code
Identification of Fine-grained Systematic Errors via Controlled Scene Generation
Our approach, BEV2EGO, allows for a realistic generation of the complete scene with road-contingent control that maps 2D bird's-eye view (BEV) scene configurations to a first-person view (EGO).
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models
The CLIP model, or one of its variants, is used as a frozen vision encoder in many vision-language models (VLMs), e. g. LLaVA and OpenFlamingo.
Analyzing and Explaining Image Classifiers via Diffusion Guidance
While deep learning has led to huge progress in complex image classification tasks like ImageNet, unexpected failure modes, e. g. via spurious features, call into question how reliably these classifiers work in the wild.
Segment (Almost) Nothing: Prompt-Agnostic Adversarial Attacks on Segmentation Models
General purpose segmentation models are able to generate (semantic) segmentation masks from a variety of prompts, including visual (points, boxed, etc.)
Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models
Counterfactual reasoning is often used in clinical settings to explain decisions or weigh alternatives.
Identifying Systematic Errors in Object Detectors with the SCROD Pipeline
Moreover, our framework introduces an evaluation setting that can serve as a benchmark for similar pipelines.
On the Adversarial Robustness of Multi-Modal Foundation Models
In this paper we show that imperceivable attacks on images in order to change the caption output of a multi-modal foundation model can be used by malicious content providers to harm honest users e. g. by guiding them to malicious websites or broadcast fake information.
Robust Semantic Segmentation: Strong Adversarial Attacks and Fast Training of Robust Models
While a large amount of work has focused on designing adversarial attacks against image classifiers, only a few methods exist to attack semantic segmentation models.
Normalization Layers Are All That Sharpness-Aware Minimization Needs
Sharpness-aware minimization (SAM) was proposed to reduce sharpness of minima and has been shown to enhance generalization performance in various settings.
In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation
The OOD detection performance when the in-distribution (ID) is ImageNet-1K is commonly being tested on a small range of test OOD datasets.
Ranked #1 on
Out-of-Distribution Detection
on ImageNet-1k vs NINCO
(using extra training data)
Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models
While adversarial training has been extensively studied for ResNet architectures and low resolution datasets like CIFAR, much less is known for ImageNet.
A Modern Look at the Relationship between Sharpness and Generalization
Overall, we observe that sharpness does not correlate well with generalization but rather with some training parameters like the learning rate that can be positively or negatively correlated with generalization depending on the setup.
Spurious Features Everywhere - Large-Scale Detection of Harmful Spurious Features in ImageNet
In contrast, we work with ImageNet and validate our results by showing that presence of the harmful spurious feature of a class alone is sufficient to trigger the prediction of that class.
Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet
In contrast, we work with ImageNet and validate our results by showing that presence of the harmful spurious feature of a class alone is sufficient to trigger the prediction of that class.
Diffusion Visual Counterfactual Explanations
Two modifications to the diffusion process are key for our DVCEs: first, an adaptive parameterization, whose hyperparameters generalize across images and models, together with distance regularization and late start of the diffusion process, allow us to generate images with minimal semantic changes to the original ones but different classification.
On the interplay of adversarial robustness and architecture components: patches, convolution and attention
In recent years novel architecture components for image classification have been developed, starting with attention and patches used in transformers.
Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation
Adversarial patch attacks are an emerging security threat for real world deep learning applications.
Adversarial Robustness of MR Image Reconstruction under Realistic Perturbations
Deep Learning (DL) methods have shown promising results for solving ill-posed inverse problems such as MR image reconstruction from undersampled $k$-space data.
Provably Adversarially Robust Nearest Prototype Classifiers
In particular we provide scalable algorithms for the \emph{exact} computation of the minimal adversarial perturbation when using $\ell_2$-distance and improved lower bounds in other cases.
Sound Randomized Smoothing in Floating-Point Arithmetics
Randomized smoothing is sound when using infinite precision.
Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities
Moreover, we show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function in the case where training and test out-distribution are the same, which again is similar to the one used when training an Energy-Based OOD detector or when adding a background class.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Sparse Visual Counterfactual Explanations in Image Space
Visual counterfactual explanations (VCEs) in image space are an important tool to understand decisions of image classifiers as they show under which changes of the image the decision of the classifier would change.
Evaluating the Adversarial Robustness of Adaptive Test-time Defenses
Adaptive defenses, which optimize at test time, promise to improve adversarial robustness.
Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
We demonstrate that plugging the learned controller into the attack consistently improves its black-box robustness estimate in different query regimes by up to 20% for a wide range of different models with black-box access.
Revisiting Out-of-Distribution Detection: A Simple Baseline is Surprisingly Effective
When trained in a shared fashion with a standard classifier, this binary discriminator reaches an OOD detection performance similar to that of Outlier Exposure.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
The Needle in the haystack: Out-distribution aware Self-training in an Open-World Setting
Traditional semi-supervised learning (SSL) has focused on the closed world assumption where all unlabeled samples are task-related.
Being a Bit Frequentist Improves Bayesian Neural Networks
Despite their compelling theoretical properties, Bayesian neural networks (BNNs) tend to perform worse than frequentist methods in classification-based uncertainty quantification (UQ) tasks such as out-of-distribution (OOD) detection.
Provably Robust Detection of Out-of-distribution Data (almost) for free
The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty.
Adversarial Robustness against Multiple and Single $l_p$-Threat Models via Quick Fine-Tuning of Robust Classifiers
In this way we get the first multiple-norm robust model for ImageNet and boost the state-of-the-art for multiple-norm robustness to more than $51\%$ on CIFAR-10.
Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators
Moreover, we present a novel adversarial bit error attack and are able to obtain robustness against both targeted and untargeted bit-level attacks.
Relating Adversarially Robust Generalization to Flat Minima
To this end, we propose average- and worst-case metrics to measure flatness in the robust loss landscape and show a correlation between good robust generalization and flatness.
Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
Finally, we combine $l_1$-APGD and an adaptation of the Square Attack to $l_1$ into $l_1$-AutoAttack, an ensemble of attacks which reliably assesses adversarial robustness for the threat model of $l_1$-ball intersected with $[0, 1]^d$.
Out-distribution aware Self-training in an Open World Setting
The goal of this paper is to leverage unlabeled data in an open world setting to further improve prediction performance.
RobustBench: a standardized adversarial robustness benchmark
As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models.
Learnable Uncertainty under Laplace Approximations
Laplace approximations are classic, computationally lightweight means for constructing Bayesian neural networks (BNNs).
An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
We extend finite ReLU BNNs with infinite ReLU features via the GP and show that the resulting model is asymptotically maximally uncertain far away from the data while the BNNs' predictive power is unaffected near the data.
Fixing Asymptotic Uncertainty of Bayesian Neural Networks with Infinite ReLU Features
However, far away from the training data, even Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be overconfident.
Certifiably Adversarially Robust Detection of Out-of-Distribution Data
Deep neural networks are known to be overconfident when applied to out-of-distribution (OOD) inputs which clearly do not belong to any class.
Bit Error Robustness for Energy-Efficient DNN Accelerators
Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights.
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting.
Adversarial Robustness on In- and Out-Distribution Improves Explainability
Neural networks have led to major improvements in image classification but suffer from being non-robust to adversarial changes, unreliable uncertainty estimates on out-distribution samples and their inscrutable black-box decisions.
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness.
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
These theoretical results validate the usage of last-layer Bayesian approximation and motivate a range of a fidelity-cost trade-off.
Square Attack: a query-efficient black-box adversarial attack via random search
We propose the Square Attack, a score-based black-box $l_2$- and $l_\infty$-adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking.
Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs
We study the task of semi-supervised learning on multilayer graphs by taking into account both labeled and unlabeled observations together with the information encoded by each individual graph layer.
Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
Our confidence-calibrated adversarial training (CCAT) tackles this problem by biasing the model towards low confidence predictions on adversarial examples.
Towards neural networks that provably know when they don't know
It has recently been shown that ReLU networks produce arbitrarily over-confident predictions far away from the training data.
Confidence-Calibrated Adversarial Training: Towards Robust Models Generalizing Beyond the Attack Used During Training
Adversarial training is the standard to train models robust against adversarial examples.
Sparse and Imperceivable Adversarial Attacks
On the other hand the pixelwise perturbations of sparse attacks are typically large and thus can be potentially detected.
Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
The evaluation of robustness against adversarial manipulation of neural networks-based classifiers is mainly tested with empirical attacks as methods for the exact computation, even when available, do not scale to large networks.
Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
The problem of adversarial robustness has been studied extensively for neural networks.
Provable robustness against all adversarial $l_p$-perturbations for $p\geq 1$
In recent years several adversarial attacks and defenses have been proposed.
Spectral Clustering of Signed Graphs via Matrix Power Means
Moreover, we prove that the eigenvalues and eigenvector of the signed power mean Laplacian concentrate around their expectation under reasonable conditions in the general Stochastic Block Model.
Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks
Modern neural networks are highly non-robust against adversarial manipulation.
Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem
We show that this technique is surprisingly effective in reducing the confidence of predictions far away from the training data while maintaining high confidence predictions and test error on the original classification task compared to standard training.
Disentangling Adversarial Robustness and Generalization
A recent hypothesis even states that both robust and accurate models are impossible, i. e., adversarial robustness and generalization are conflicting goals.
A randomized gradient-free attack on ReLU networks
Relatively fast heuristics have been proposed to produce these adversarial inputs but the problem of finding the optimal adversarial input, that is with the minimal change of the input, is NP-hard.
Logit Pairing Methods Can Fool Gradient-Based Attacks
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.
Provable Robustness of ReLU networks via Maximization of Linear Regions
It has been shown that neural network classifiers are not robust.
On the loss landscape of a class of deep neural networks with no bad local valleys
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero.
The Power Mean Laplacian for Multilayer Graph Clustering
Multilayer graphs encode different kind of interactions between the same set of entities.
Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions
In the recent literature the important role of depth in deep learning has been emphasized.
Error estimates for spectral convergence of the graph Laplacian on random geometric graphs towards the Laplace--Beltrami operator
sample from a $m$-dimensional submanifold $M$ in $R^d$ as the sample size $n$ increases and the neighborhood size $h$ tends to zero.
The loss surface and expressivity of deep convolutional neural networks
We show that such CNNs produce linearly independent features at a “wide” layer which has more neurons than the number of training samples.
Optimization Landscape and Expressivity of Deep CNNs
We show that such CNNs produce linearly independent features at a "wide" layer which has more neurons than the number of training samples.
Community detection in networks via nonlinear modularity eigenvectors
In this work we propose a nonlinear relaxation which is instead based on the spectrum of a nonlinear modularity operator $\mathcal M$.
Variants of RMSProp and Adagrad with Logarithmic Regret Bounds
Adaptive gradient methods have become recently very popular, in particular as they have been shown to be useful in the training of deep neural networks.
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
We show in this paper for the first time formal guarantees on the robustness of a classifier by giving instance-specific lower bounds on the norm of the input manipulation required to change the classifier decision.
The loss surface of deep and wide neural networks
While the optimization problem behind deep neural networks is highly non-convex, it is frequently observed in practice that training deep networks seems possible without getting stuck in suboptimal points.
Clustering Signed Networks with the Geometric Mean of Laplacians
As a solution we propose to use the geometric mean of the Laplacians of positive and negative part and show that it outperforms the existing approaches.
Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification
In particular, we find that it is possible to obtain effective multilabel classifiers on Pascal VOC using a single label per image for training, while the gap between multiclass and multilabel methods on MS COCO is more significant.
Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
The optimization problem behind neural networks is highly non-convex.
Improved Image Boundaries for Better Video Segmentation
Graph-based video segmentation methods rely on superpixels as starting point.
Latent Embeddings for Zero-shot Classification
We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image.
Simple Does It: Weakly Supervised Instance and Semantic Segmentation
Semantic labelling and instance segmentation are two tasks that require particularly costly annotations.
Ranked #1 on
Semantic Segmentation
on PASCAL VOC 2012 val
(Mean IoU metric)
Loss Functions for Top-k Error: Analysis and Insights
In the experiments, we compare on various datasets all of the proposed and established methods for top-k error optimization.
Weakly Supervised Object Boundaries
State-of-the-art learning based boundary detection methods require extensive training data.
Ranked #2 on
Edge Detection
on SBD
Top-k Multiclass SVM
Class ambiguity is typical in image classification problems with a large number of classes.
Efficient Output Kernel Learning for Multiple Tasks
The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other.
An Efficient Multilinear Optimization Framework for Hypergraph Matching
Hypergraph matching has recently become a popular approach for solving correspondence problems in computer vision as it allows to integrate higher-order geometric information.
Robust PCA: Optimization of the Robust Reconstruction Error over the Stiefel Manifold
In distinction to other methods for robust PCA, our method has no free parameter and is computationally very efficient.
Classifier Based Graph Construction for Video Segmentation
Video segmentation has become an important and active research area with a large diversity of proposed approaches.
Constrained 1-Spectral Clustering
Opposite to all other methods which have been suggested for constrained spectral clustering, we can always guarantee to satisfy all constraints.
Tight Continuous Relaxation of the Balanced $k$-Cut Problem
Spectral Clustering as a relaxation of the normalized/ratio cut has become one of the standard graph-based clustering methods.
A Flexible Tensor Block Coordinate Ascent Scheme for Hypergraph Matching
We propose two algorithms which both come along with the guarantee of monotonic ascent in the matching score on the set of discrete assignment matrices.
Regularization-free estimation in trace regression with symmetric positive semidefinite matrices
Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing.
Tight Continuous Relaxation of the Balanced k-Cut Problem
Spectral Clustering as a relaxation of the normalized/ratio cut has become one of the standard graph-based clustering methods.
Scalable Multitask Representation Learning for Scene Classification
The underlying idea of multitask learning is that learning tasks jointly is better than learning each task individually.
Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov Random fields
Consider a random vector with finite second moments.
Matrix factorization with Binary Components
Motivated by an application in computational biology, we consider low-rank matrix factorization with $\{0, 1\}$-constraints on one of the factors and optionally convex constraints on the second one.
The Total Variation on Hypergraphs - Learning on Hypergraphs Revisited
Hypergraphs allow one to encode higher-order relationships in data and are thus a very flexible modeling tool.
Nonlinear Eigenproblems in Data Analysis - Balanced Graph Cuts and the RatioDCA-Prox
It has been recently shown that a large class of balanced graph cuts allows for an exact relaxation into a nonlinear eigenproblem.
Constrained fractional set programs and their application in local clustering and community detection
While a globally optimal solution for the resulting non-convex problem cannot be guaranteed, we outperform the loose convex or spectral relaxations by a large margin on constrained local clustering problems.
Learning Using Privileged Information: SVM+ and Weighted SVM
Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training.
Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
We show that for these designs, the performance of NNLS with regard to prediction and estimation is comparable to that of the lasso.
Beyond Spectral Clustering - Tight Relaxations of Balanced Graph Cuts
Spectral clustering is based on the spectral relaxation of the normalized/ratio graph cut criterion.
Sparse recovery by thresholded non-negative least squares
Non-negative data are commonly encountered in numerous fields, making non-negative least squares regression (NNLS) a frequently used tool.
An Inverse Power Method for Nonlinear Eigenproblems with Applications in 1-Spectral Clustering and Sparse PCA
Many problems in machine learning and statistics can be formulated as (generalized) eigenproblems.
Getting lost in space: Large sample analysis of the resistance distance
As an alternative we introduce the amplified commute distance that corrects for the undesired large sample effects.
Robust Nonparametric Regression with Metric-Space Valued Output
Motivated by recent developments in manifold-valued regression we propose a family of nonparametric kernel-smoothing estimators with metric-space valued output including a robust median type estimator and the classical Frechet mean.
Semi-supervised Regression using Hessian energy with an application to semi-supervised dimensionality reduction
Semi-supervised regression based on the graph Laplacian suffers from the fact that the solution is biased towards a constant and the lack of extrapolating power.
Non-parametric Regression Between Manifolds
This paper discusses non-parametric regression between Riemannian manifolds.
