Search Results for author:
Found 62 papers, 32 papers with code
VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis
Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content.
Are Vision Language Models Texture or Shape Biased and Can We Steer Them?
If text does indeed influence visual biases, this suggests that we may be able to steer visual biases not just through visual input but also through language: a hypothesis that we confirm through extensive experiments.
Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive
Current L2I models either suffer from poor editability via text or weak alignment between the generated image and the input layout.
Improving Stability during Upsampling -- on the Importance of Spatial Context
While during downsampling, aliases and artifacts can be reduced by blurring feature maps, the emergence of fine details is crucial during upsampling.
Classification robustness to common optical aberrations
Experiments on ImageNet show that for a variety of different pre-trained DNNs, the performance varies strongly compared to disk-shaped kernels, indicating the necessity of considering realistic image degradations.
Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition
We propose a method specifically designed for hand action recognition which uses relative angular embeddings and local Spherical Harmonics to create novel hand representations.
On the unreasonable vulnerability of transformers for image restoration -- and an easy fix
We consider the recently proposed Restormer model, as well as NAFNet and the "Baseline network" which are both simplified versions of a Restormer.
Divide & Bind Your Attention for Improved Generative Semantic Nursing
To address the challenges posed by complex prompts or scenarios involving multiple entities and to achieve improved attribute binding, we propose Divide & Bind.
Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling
Convolutional neural networks encode images through a sequence of convolutions, normalizations and non-linearities as well as downsampling operations into potentially strong semantic embeddings.
As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domain
Motivated by the recent trend towards the usage of larger receptive fields for more context-aware neural networks in vision applications, we aim to investigate how large these receptive fields really need to be.
An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustness
We are interested in the single prediction task for robustness and the joint multi-objective of clean and robust accuracy.
Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization
Using the proposed masked noise encoder to randomize style and content combinations in the training set, i. e., intra-source style augmentation (ISSA) effectively increases the diversity of training data and reduces spurious correlation.
Neural Architecture Design and Robustness: A Dataset
We evaluate all these networks on a range of common adversarial attacks and corruption types and introduce a database on neural architecture design and robustness evaluations.
Happy People -- Image Synthesis as Black-Box Optimization Problem in the Discrete Latent Space of Deep Generative Models
In recent years, optimization in the learned latent space of deep generative models has been successfully applied to black-box optimization problems such as drug design, image generation or neural architecture search.
Differentiable Sensor Layouts for End-to-End Learning of Task-Specific Camera Parameters
The success of deep learning is frequently described as the ability to train all parameters of a network on a specific application in an end-to-end fashion.
An Extended Study of Human-like Behavior under Adversarial Training
Adversarial training poses a partial solution to address this issue by training models on worst-case perturbations.
FullFormer: Generating Shapes Inside Shapes
Implicit generative models have been widely employed to model 3D data and have recently proven to be successful in encoding and generating high-quality 3D shapes.
CosPGD: a unified white-box adversarial attack for pixel-wise prediction tasks
Further, we set a new benchmark for adversarial attacks on optical flow, and image restoration displaying the ability to extend to any pixel-wise prediction task.
Unfolding Local Growth Rate Estimates for (Almost) Perfect Adversarial Detection
Convolutional neural networks (CNN) define the state-of-the-art solution on many perceptual tasks.
Hypergraph Transformer for Skeleton-based Action Recognition
Skeleton-based action recognition aims to recognize human actions given human joint coordinates with skeletal interconnections.
Ranked #7 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Intra-Source Style Augmentation for Improved Domain Generalization
Using the proposed masked noise encoder to randomize style and content combinations in the training set, ISSA effectively increases the diversity of training data and reduces spurious correlation.
Robust Models are less Over-Confident
Further, our analysis of robust models shows that not only AT but also the model's building blocks (like activation functions and pooling) have a strong influence on the models' prediction confidences.
SP-ViT: Learning 2D Spatial Priors for Vision Transformers
Unlike convolutional inductive biases, which are forced to focus exclusively on hard-coded local regions, our proposed SPs are learned by the model itself and take a variety of spatial relations into account.
Ranked #153 on
Image Classification
on ImageNet
Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks
The minimum cost multicut problem is the NP-hard/APX-hard combinatorial optimization problem of partitioning a real-valued edge-weighted graph such as to minimize the total cost of the partition.
FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting
Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks.
Learning Where To Look -- Generative NAS is Surprisingly Efficient
the optimization of architectures for highest classification accuracy but also in the context of hardware constraints and outperform state-of-the-art methods on several NAS benchmarks for single and multiple objectives.
Optimizing Edge Detection for Image Segmentation with Multicut Penalties
The Minimum Cost Multicut Problem (MP) is a popular way for obtaining a graph decomposition by optimizing binary edge labels over edge costs.
Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?
In its most commonly reported sub-task, RobustBench evaluates and ranks the adversarial robustness of trained neural networks on CIFAR10 under AutoAttack (Croce and Hein 2020b) with l-inf perturbations limited to eps = 8/255.
Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders
In this paper, we propose a simple and end-to-end trainable deterministic autoencoding framework, that efficiently shapes the latent space of the model during training and utilizes the capacity of expressive multi-modal latent distributions.
Aliasing coincides with CNNs vulnerability towards adversarial attacks
Many commonly well-performing convolutional neural network models have shown to be susceptible to input data perturbations, indicating a low model robustness.
Detecting AutoAttack Perturbations in the Frequency Domain
Recently, adversarial attacks on image classification networks by the AutoAttack (Croce and Hein, 2020b) framework have drawn a lot of attention.
DARTS for Inverse Problems: a Study on Stability
Differentiable architecture search (DARTS) is a widely researched tool for neural architecture search, due to its promising results for image classification.
Is Differentiable Architecture Search truly a One-Shot Method?
In this work, we investigate DAS in a systematic case study of inverse problems, which allows us to analyze these potential benefits in a controlled manner.
Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space
Over the last decade, the development of deep image classification networks has mostly been driven by the search for the best performance in terms of classification accuracy on standardized benchmarks like ImageNet.
Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis
The rapid advances in deep generative models over the past years have led to highly {realistic media, known as deepfakes,} that are commonly indistinguishable from real to human eyes.
Uncertainty in Minimum Cost Multicuts for Image and Motion Segmentation
The minimum cost lifted multicut approach has proven practically good performance in a wide range of applications such as image decomposition, mesh segmentation, multiple object tracking, and motion segmentation.
Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation
Image generation has rapidly evolved in recent years.
SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain
Despite the success of convolutional neural networks (CNNs) in many computer vision and image analysis tasks, they remain vulnerable against so-called adversarial attacks: Small, crafted perturbations in the input images can lead to false predictions.
Spectral Distribution Aware Image Generation
In this paper, we propose to generate images according to the frequency distribution of the real data by employing a spectral discriminator.
Neural Architecture Performance Prediction Using Graph Neural Networks
In computer vision research, the process of automating architecture engineering, Neural Architecture Search (NAS), has gained substantial interest.
Smooth Variational Graph Embeddings for Efficient Neural Architecture Search
We evaluate the proposed approach on neural architectures defined by the ENAS approach, the NAS-Bench-101 and the NAS-Bench-201 search space and show that our smooth embedding space allows to directly extrapolate the performance prediction to architectures outside the seen domain (e. g. with more operations).
Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks
We show that surrogate NAS benchmarks can model the true performance of architectures better than tabular benchmarks (at a small fraction of the cost), that they lead to faithful estimates of how well different NAS methods work on the original non-surrogate benchmark, and that they can generate new scientific insight.
Self-supervised Sparse to Dense Motion Segmentation
We evaluate our method on the well-known motion segmentation datasets FBMS59 and DAVIS16.
Learning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches
In this work, we evaluate two different image clustering objectives, k-means clustering and correlation clustering, in the context of Triplet Loss induced feature space embeddings.
Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions
Generative convolutional deep neural networks, e. g. popular GAN architectures, are relying on convolution based up-sampling methods to produce non-scalar outputs like images or video sequences.
Unsupervised Multiple Person Tracking using AutoEncoder-Based Lifted Multicuts
Our method is based on straight-forward spatio-temporal cues that can be extracted from neighboring frames in an image sequences without superivison.
A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction
In computer vision research, the process of automating architecture engineering, Neural Architecture Search (NAS), has gained substantial interest.
Unmasking DeepFakes with simple Features
In this work, we present a simple way to detect such fake face images - so-called DeepFakes.
Object Segmentation Tracking from Generic Video Cues
Our simple method can provide competitive results compared to the costly CNN-based methods with parameter tuning.
Massively Parallel Benders Decomposition for Correlation Clustering
We tackle the problem of graph partitioning for image segmentation using correlation clustering (CC), which we treat as an integer linear program (ILP).
Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation
Making use of the estimated occlusions, we also show improved results on motion segmentation and scene flow estimation.
Learning Dilation Factors for Semantic Segmentation of Street Scenes
In this paper, we present an approach for learning dilation parameters adaptively per channel, consistently improving semantic segmentation results on street-scene datasets like Cityscapes and Camvid.
ProbFlow: Joint Optical Flow and Uncertainty Estimation
We overcome the artificial separation of optical flow and confidence estimation by introducing a method that jointly predicts optical flow and its underlying uncertainty.
Higher-Order Minimum Cost Lifted Multicuts for Motion Segmentation
Most state-of-the-art motion segmentation algorithms draw their potential from modeling motion differences of local entities such as point trajectories in terms of pairwise potentials in graphical models.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods.
Dense Pixel Correspondence Estimation
Optical Flow Estimation
+1
A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects
Recently, Minimum Cost Multicut Formulations have been proposed and proven to be successful in both motion trajectory segmentation and multi-target tracking scenarios.
Point-wise mutual information-based video segmentation with high temporal consistency
In this paper, we tackle the problem of temporally consistent boundary detection and hierarchical segmentation in videos.
STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling
The proposed network produces a high quality segmentation of a single image by leveraging information from additional views of the same scene.
Ranked #96 on
Semantic Segmentation
on NYU Depth v2
Motion Trajectory Segmentation via Minimum Cost Multicuts
For the segmentation of moving objects in videos, the analysis of long-term point trajectories has been very popular recently.
Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts
a pixel grid graph have received little attention, firstly, because the MP is NP-hard and instances w. r. t.
Spectral Graph Reduction for Efficient Image and Streaming Video Segmentation
In contrast to previous work, the reduced graph is reweighted such that the resulting segmentation is equivalent, under certain assumptions, to that of the full graph.
Blind Deconvolution of Widefield Fluorescence Microscopic Data by Regularization of the Optical Transfer Function (OTF)
With volumetric data from widefield fluorescence microscopy, many emerging questions in biological and biomedical research are being investigated.
