Search Results for author:
Found 97 papers, 52 papers with code
360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View
In this work, mapping from 360{\deg} panoramas to BEV semantics, the 360BEV task, is established for the first time to achieve holistic representations of indoor scenes in a top-down view.
Guiding the Guidance: A Comparative Analysis of User Guidance Signals for Interactive Segmentation of Volumetric Images
To address this, we conduct a comparative study of existing guidance signals by training interactive models with different signals and parameter settings to identify crucial parameters for the model's design.
Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging
Positron Emission Tomography (PET) and Computer Tomography (CT) are routinely used together to detect tumors.
MuscleMap: Towards Video-based Activated Muscle Group Estimation
To make the AMGE model applicable in real-life situations, it is crucial to ensure that the model can generalize well to types of physical activities not present during training and involving new combinations of activated muscles.
Delivering Arbitrary-Modal Semantic Segmentation
To make this possible, we present the arbitrary cross-modal segmentation model CMNeXt.
Ranked #1 on
Semantic Segmentation
on UrbanLF
MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments
To fill this gap, in this work, a wearable robotic system, MateRobot, is established for PVI to recognize materials before hand.
Multimodal Interactive Lung Lesion Segmentation: A Framework for Annotating PET/CT Images based on Physiological and Anatomical Cues
Recently, deep learning enabled the accurate segmentation of various diseases in medical imaging.
Uncertainty-aware Vision-based Metric Cross-view Geolocalization
This paper proposes a novel method for vision-based metric cross-view geolocalization (CVGL) that matches the camera images captured from a ground-based vehicle with an aerial image to determine the vehicle's geo-pose.
Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation
Although human action anticipation is a task which is inherently multi-modal, state-of-the-art methods on well known action anticipation datasets leverage this data by applying ensemble methods and averaging scores of unimodal anticipation networks.
Ranked #1 on
Action Anticipation
on EPIC-KITCHENS-100
Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding
To exploit anatomical structures in this scenario, we present a sophisticated automatic pipeline to gather and integrate human bodily structures from computed tomography datasets, which we incorporate in our PAXRay: A Projected dataset for the segmentation of Anatomical structures in X-Ray data.
AutoPET Challenge: Combining nn-Unet with Swin UNETR Augmented by Maximum Intensity Projection Classifier
Tumor volume and changes in tumor characteristics over time are important biomarkers for cancer therapy.
ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalization
Modality selection is an important step when designing multimodal systems, especially in the case of cross-domain activity recognition as certain modalities are more robust to domain shift than others.
Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living
We tackle this challenge and introduce an activity domain generation framework which creates novel ADL appearances (novel domains) from different existing activity modalities (source domains) inferred from video training data.
Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation
Panoramic segmentation is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of annotations for training panoramic segmenters.
Ranked #1 on
Semantic Segmentation
on Stanford2D3D Panoramic
Multi-modal Depression Estimation based on Sub-attentional Fusion
Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide.
Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
Humans have an innate ability to sense their surroundings, as they can extract the spatial representation from the egocentric perception and form an allocentric semantic map via spatial transformation and memory updating.
Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning
In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to the panoramic domain in a cost-minimizing way.
Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training
We show that despite using unstructured medical report supervision, we perform on par with direct label supervision through a sophisticated inference setting.
Ranked #1 on
Thoracic Disease Classification
on ChestX-ray14
Towards Automatic Parsing of Structured Visual Content through the Use of Synthetic Data
In this work, we propose a synthetic dataset, containing SVCs in the form of images as well as ground truths.
Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates
In this work, we for the first time examine how well the confidence values of modern driver observation models indeed match the probability of the correct outcome and show that raw neural network-based approaches tend to significantly overestimate their prediction quality.
A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding
Visual recognition inside the vehicle cabin leads to safer driving and more intuitive human-vehicle interaction but such systems face substantial obstacles as they need to capture different granularities of driver behaviour while dealing with highly limited body visibility and changing illumination.
Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor
Then, poses and speed of tracked dynamic objects can be estimated, which are passed to the users through acoustic feedback.
Hierarchical Nearest Neighbor Graph Embedding for Efficient Dimensionality Reduction
Dimensionality reduction is crucial both for visualization and preprocessing high dimensional data for machine learning.
Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning
Autonomous vehicles utilize urban scene segmentation to understand the real world like a human and react accordingly.
Ranked #1 on
Semantic Segmentation
on DADA-seg
(using extra training data)
MatchFormer: Interleaving Attention in Transformers for Feature Matching
While detector-based methods coupled with feature descriptors struggle in low-texture scenes, CNN-based methods with a sequential extract-to-match pipeline, fail to make use of the matching capacity of the encoder and tend to overburden the decoder for matching.
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers
In this work, we propose CMX, a transformer-based cross-modal fusion framework for RGB-X semantic segmentation.
Ranked #1 on
Thermal Image Segmentation
on RT-5K
Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors
Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos.
Revisiting Click-based Interactive Video Object Segmentation
While current methods for interactive Video Object Segmentation (iVOS) rely on scribble-based interactions to generate precise object masks, we propose a Click-based interactive Video Object Segmentation (CiVOS) framework to simplify the required user workload as much as possible.
Interactive Video Object Segmentation
Semantic Segmentation
+1
TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration
This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e. g., sensor changes) and general feature quality.
Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation
To get around this domain difference and bring together semantic annotations from pinhole- and 360-degree surround-visuals, we propose to learn object deformations and panoramic image distortions in the Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) components which blend into our Transformer for PAnoramic Semantic Segmentation (Trans4PASS) model.
Ranked #2 on
Semantic Segmentation
on Stanford2D3D Panoramic
TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation
Large pre-trained transformers are on top of contemporary semantic segmentation benchmarks, but come with high computational cost and a lengthy training.
Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions
Yet, the research of data-scarce recognition from skeleton sequences, such as one-shot action recognition, does not explicitly consider occlusions despite their everyday pervasiveness.
Ranked #1 on
Action Classification
on Toyota Smarthome dataset
(Accuracy metric)
Should I take a walk? Estimating Energy Expenditure from Video Data
To study this underresearched task, we introduce Vid2Burn -- an omni-source benchmark for estimating caloric expenditure from video data featuring both, high- and low-intensity activities for which we derive energy expenditure annotations based on models established in medical literature.
Exploring Event-driven Dynamic Context for Accident Scene Segmentation
Moreover, in order to evaluate the segmentation performance in traffic accidents, we provide a pixel-wise annotated accident dataset, namely DADA-seg, which contains a variety of critical scenarios from traffic accidents.
Ranked #3 on
Semantic Segmentation
on DADA-seg
(using extra training data)
Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation
Following this thought, we use a small number of labeled images as reference material and match pixels in an unlabeled image to the semantics of the best fitting pixel in a reference set.
Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning
Human affect recognition is a well-established research area with numerous applications, e. g., in psychological care, but existing methods assume that all emotions-of-interest are given a priori as annotated training examples.
Certainty Volume Prediction for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) deals with the problem of classifying unlabeled target domain data while labeled data is only available for a different source domain.
UBR$^2$S: Uncertainty-Based Resampling and Reweighting Strategy for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) deals with the adaptation process of a model to an unlabeled target domain while annotated data is only available for a given source domain.
Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
We look at this problem from the perspective of domain adaptation and bring panoramic semantic segmentation to a setting, where labelled training data originates from a different distribution of conventional pinhole camera images.
Ranked #7 on
Semantic Segmentation
on DensePASS
(using extra training data)
Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects.
Ranked #2 on
Semantic Segmentation
on DADA-seg
(using extra training data)
Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation
By recognizing the color of pedestrian traffic lights, our prototype can help the user to cross a street safely.
DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
First, we formalize the task of unsupervised domain adaptation for panoramic semantic segmentation, where a network trained on labelled examples from the source domain of pinhole camera data is deployed in a different target domain of panoramic images, for which no labels are available.
Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games
Recognizing Activities of Daily Living (ADL) is a vital process for intelligent assistive robots, but collecting large annotated datasets requires time-consuming temporal labeling and raises privacy concerns, e. g., if the data is collected in a real household.
HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
To tackle these issues, we propose HIDA, a lightweight assistive system based on 3D point cloud instance segmentation with a solid-state LiDAR sensor, for holistic indoor detection and avoidance.
Ranked #9 on
3D Instance Segmentation
on ScanNet(v2)
Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World
Common fully glazed facades and transparent objects present architectural barriers and impede the mobility of people with low vision or blindness, for instance, a path detected behind a glass door is inaccessible unless it is correctly perceived and reacted.
Ranked #1 on
Semantic Segmentation
on Trans10K
MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding
At the heart of all automated driving systems is the ability to sense the surroundings, e. g., through semantic segmentation of LiDAR sequences, which experienced a remarkable progress due to the release of large datasets such as SemanticKITTI and nuScenes-LidarSeg.
Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
Our framework provides the functionality to control the movement of the drone with simple arm gestures and to follow the user while keeping a safe distance.
Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentation
Pixel-wise segmentation is one of the most data and annotation hungry tasks in our field.
Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation
Action segmentation refers to inferring boundaries of semantically consistent visual concepts in videos and is an important requirement for many video understanding tasks.
Ranked #1 on
Action Segmentation
on Breakfast
(mIoU metric)
Capturing Omni-Range Context for Omnidirectional Segmentation
Convolutional Networks (ConvNets) excel at semantic segmentation and have become a vital component for perception in autonomous driving.
Ranked #10 on
Semantic Segmentation
on DensePASS
(using extra training data)
Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired
Based on Lintention, we then devise a novel panoptic segmentation model which we term Panoptic Lintention Net.
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired
As the scene information, including objectness and scene type, are important for people with visual impairment, in this work we present a multi-task efficient perception system for the scene parsing and recognition tasks.
DR-TANet: Dynamic Receptive Temporal Attention Network for Street Scene Change Detection
Street scene change detection continues to capture researchers' interests in the computer vision community.
Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning
In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
Prediction of low-keV monochromatic images from polyenergetic CT scans for improved automatic detection of pulmonary embolism
Detector-based spectral computed tomography is a recent dual-energy CT (DECT) technology that offers the possibility of obtaining spectral information.
Adaptiope: A Modern Benchmark for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) deals with the adaptation process of a given source domain with labeled training data to a target domain for which only unannotated data is available.
Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models
While temperature scaling alone drastically improves the reliability of the confidence values, our CARING method consistently leads to the best uncertainty estimates in all benchmark settings.
Vi2CLR: Video and Image for Visual Contrastive Learning of Representation
In this paper, we introduce a novel self-supervised visual representation learning method which understands both images and videos in a joint learning fashion.
Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information
Our experiments demonstrate clear benefits of multi-task learning for calorie estimation, surpassing the single-task calorie regression by 9. 9%.
Self-Guided Multiple Instance Learning for Weakly Supervised Disease Classification and Localization in Chest Radiographs
In this paper, we address the problem of weakly supervised identification and localization of abnormalities in chest radiographs.
Unsupervised Domain Adaptation by Uncertain Feature Alignment
Unsupervised domain adaptation (UDA) deals with the adaptation of models from a given source domain with labeled data to an unlabeled target domain.
ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications.
Ranked #6 on
Semantic Segmentation
on DADA-seg
Anchor-free Small-scale Multispectral Pedestrian Detection
In this way, we can both simplify the network architecture and achieve higher detection performance, especially for pedestrians under occlusion or at low object resolution.
Can we cover navigational perception needs of the visually impaired by panoptic segmentation?
Navigational perception for visually impaired people has been substantially promoted by both classic and deep learning based segmentation methods.
Detective: An Attentive Recurrent Model for Sparse Object Detection
At each iteration, our decoder focuses on the relevant parts of the image using an attention mechanism, and then estimates the object's class and the bounding box coordinates.
Clustering based Contrastive Learning for Improving Face Representations
We demonstrate our method on the challenging task of learning representations for video face clustering.
Deep Multimodal Feature Encoding for Video Ordering
True understanding of videos comes from a joint analysis of all its modalities: the video frames, the audio track, and any accompanying text such as closed captions.
DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing
Semantically interpreting the traffic scene is crucial for autonomous transportation and robotics systems.
Ranked #35 on
Semantic Segmentation
on DensePASS
Content and Colour Distillation for Learning Image Translations with the Spatial Profile Loss
In this paper, we propose a novel method of computing the loss directly between the source and target images that enable proper distillation of shape/content and colour/style.
DynamoNet: Dynamic Action and Motion Network
With these overall objectives, to this end, we introduce a novel unified spatio-temporal 3D-CNN architecture (DynamoNet) that jointly optimizes the video classification and learning motion representation by predicting future frames as a multi-task learning problem.
Large Scale Holistic Video Understanding
HVU is organized hierarchically in a semantic taxonomy that focuses on multi-label and multi-task video understanding as a comprehensive problem that encompasses the recognition of multiple semantic aspects in the dynamic scene.
Ranked #7 on
Action Recognition
on UCF101
Self-Supervised Learning of Face Representations for Video Face Clustering
In this paper, we address video face clustering using unsupervised methods.
Efficient Parameter-free Clustering Using First Neighbor Relations
We present a new clustering method in the form of a single clustering equation that is able to directly discover groupings in the data.
Can Image Enhancement be Beneficial to Find Smoke Images in Laparoscopic Surgery?
For feature extraction, we use statistical features based on bivariate histogram distribution of gradient magnitude~(GM) and Laplacian of Gaussian~(LoG).
Informed Democracy: Voting-based Novelty Detection for Action Recognition
While it is common in activity recognition to assume a closed-set setting, i. e. test samples are always of training categories, this assumption is impractical in a real-world scenario.
Taming the Cross Entropy Loss
We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks.
3D Vehicle Trajectory Reconstruction in Monocular Video Data Using Environment Structure Constraints
We apply Structure from Motion techniques to vehicle and background images to determine for each frame camera poses relative to vehicle instances and background structures.
Stereo 3D Object Trajectory Reconstruction
We compute the object trajectory by combining object and background camera pose information.
Classification-Driven Dynamic Image Enhancement
In this paper, we are interested in learning CNNs that can emulate image enhancement and restoration, but with the overall goal to improve image classification and not necessarily human perception.
A Pose-Sensitive Embedding for Person Re-Identification with Expanded Cross Neighborhood Re-Ranking
In contrast to the recent direction of explicitly modeling body parts or correcting for misalignment based on these, we show that a rather straightforward inclusion of acquired camera view and/or the detected joint locations into a convolutional neural network helps to learn a very effective representation.
Ranked #17 on
Person Re-Identification
on MARS
Weakly Supervised Object Discovery by Generative Adversarial & Ranking Networks
We approach GANs with a novel training method and learning objective, to discover multiple object instances for three cases: 1) synthesizing a picture of a specific object within a cluttered scene; 2) localizing different categories in images for weakly supervised object detection; and 3) improving object discov- ery in object detection pipelines.
Ranked #2 on
Weakly Supervised Object Detection
on COCO test-dev
3D Trajectory Reconstruction of Dynamic Objects Using Planarity Constraints
We apply Structure from Motion techniques to object and background images to determine for each frame camera poses relative to object instances and background structures.
Classification Driven Dynamic Image Enhancement
In this paper, we are interested in learning CNNs that can emulate image enhancement and restoration, but with the overall goal to improve image classification and not necessarily human perception.
Deep View-Sensitive Pedestrian Attribute Inference in an end-to-end Model
The visual cues hinting at attributes can be strongly localized and inference of person attributes such as hair, backpack, shorts, etc., are highly dependent on the acquired view of the pedestrian.
Automatic Discovery, Association Estimation and Learning of Semantic Attributes for a Thousand Categories
Furthermore, we demonstrate that our model outperforms the state-of-the-art in zero-shot learning on three data sets: ImageNet, Animals with Attributes and aPascal/aYahoo.
Relaxed Earth Mover's Distances for Chain- and Tree-connected Spaces and their use as a Loss Function in Deep Learning
The Earth Mover's Distance (EMD) computes the optimal cost of transforming one distribution into another, given a known transport metric between them.
Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learning
In this work, we aim to carry out attribute-based zero-shot classification in an unsupervised manner.
How to Transfer? Zero-Shot Object Recognition via Hierarchical Transfer of Semantic Attributes
We propose to capture these variations in a hierarchical model that expands the knowledge source with additional abstraction levels of attributes.
Deep Perceptual Mapping for Cross-Modal Face Recognition
Our method bridges the drop in performance due to the modality gap by more than 40\%.
Ranked #2 on
Face Recognition
on UND-X1
MovieQA: Understanding Stories in Movies through Question-Answering
We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text.
What's the point? Frame-wise Pointing Gesture Recognition with Latent-Dynamic Conditional Random Fields
We use Latent-Dynamic Conditional Random Fields to perform skeleton-based pointing gesture classification at each time instance of a video sequence, where we achieve a frame-wise pointing accuracy of roughly 83%.
Deep Perceptual Mapping for Thermal to Visible Face Recognition
Cross modal face matching between the thermal and visible spectrum is a much de- sired capability for night-time surveillance and security applications.
Book2Movie: Aligning Video Scenes With Book Chapters
Such an alignment facilitates finding differences between the adaptation and the original source, and also acts as a basis for deriving rich descriptions from the novel for the video clips.
On the Distribution of Salient Objects in Web Images and its Influence on Salient Object Detection
Tseng et al. have shown that the photographer's tendency to place interesting objects in the center is a likely cause for the center bias of eye fixations.
StoryGraphs: Visualizing Character Interactions as a Timeline
We present a novel way to automatically summarize and represent the storyline of a TV episode by visualizing character interactions as a chart.
Semi-supervised Learning with Constraints for Person Identification in Multimedia Data
We address the problem of person identification in TV series.
