Search Results for author:
Found 79 papers, 39 papers with code
AGS-Mesh: Adaptive Gaussian Splatting and Meshing with Geometric Priors for Indoor Room Reconstruction Using Smartphones
We develop a scale-aware meshing strategy inspired by TSDF and octree-based isosurface extraction, which recovers finer details from Gaussian models compared to other commonly used open-source meshing tools.
Temporally Aligned Audio for Video with Autoregression
We introduce V-AURA, the first autoregressive model to achieve high temporal alignment and relevance in video-to-audio generation.
UDGS-SLAM : UniDepth Assisted Gaussian Splatting for Monocular SLAM
Recent advancements in monocular neural depth estimation, particularly those achieved by the UniDepth network, have prompted the investigation of integrating UniDepth within a Gaussian splatting framework for monocular SLAM. This study presents UDGS-SLAM, a novel approach that eliminates the necessity of RGB-D sensors for depth estimation within Gaussian splatting framework.
DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing
In this work, we explore the use of readily accessible geometric cues to enhance Gaussian splatting optimization in challenging, ill-posed, and textureless scenes.
Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion
High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras.
GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splatting
At inference, GS-Pose operates sequentially by locating the object in the input image, estimating its initial 6D pose using a retrieval approach, and refining the pose with a render-and-compare method.
Synchformer: Efficient Synchronization from Sparse Cues
Our objective is audio-visual synchronization with a focus on 'in-the-wild' videos, such as those on YouTube, where synchronization cues can be sparse.
Bridging the gap between image coding for machines and humans
Image coding for machines (ICM) aims at reducing the bitrate required to represent an image while minimizing the drop in machine vision analysis accuracy.
NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines
To the best of our knowledge, this is the first research paper showing a hybrid video codec that outperforms VVC on multiple datasets and multiple machine vision tasks.
MuSHRoom: Multi-Sensor Hybrid Room Dataset for Joint 3D Reconstruction and Novel View Synthesis
Metaverse technologies demand accurate, real-time, and immersive modeling on consumer-grade hardware for both non-human perception (e. g., drone/robot/autonomous car navigation) and immersive technologies like AR/VR, requiring both structural accuracy and photorealism.
Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation
Human evaluation is critical for validating the performance of text-to-image generative models, as this highly cognitive process requires deep comprehension of text and images.
FinnWoodlands Dataset
Besides tree trunks, we also annotated "Obstacles" objects as instances as well as the semantic stuff classes "Lake", "Ground", and "Track".
MSDA: Monocular Self-supervised Domain Adaptation for 6D Object Pose Estimation
Though massive amounts of synthetic RGB images are easy to obtain, the models trained on them suffer from noticeable performance degradation due to the synthetic-to-real domain gap.
BS3D: Building-scale 3D Reconstruction from RGB-D Images
Various datasets have been proposed for simultaneous localization and mapping (SLAM) and related problems.
PanDepth: Joint Panoptic Segmentation and Depth Completion
Understanding 3D environments semantically is pivotal in autonomous driving applications where multiple computer vision tasks are involved.
Supervised Fine-tuning Evaluation for Long-term Visual Place Recognition
In this paper, we present a comprehensive study on the utility of deep convolutional neural networks with two state-of-the-art pooling layers which are placed after convolutional layers and fine-tuned in an end-to-end manner for visual place recognition task in challenging conditions, including seasonal and illumination variations.
Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors
This contrasts with the case of synchronising videos of talking heads, where audio-visual correspondence is dense in both time and space.
SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion
Holistic scene understanding is pivotal for the performance of autonomous machines.
The Weighting Game: Evaluating Quality of Explainability Methods
The objective of this paper is to assess the quality of explanation heatmaps for image classification tasks.
Cascaded and Generalizable Neural Radiance Fields for Fast View Synthesis
Moreover, our method can leverage a denser set of reference images of a single scene to produce accurate novel views without relying on additional explicit representations and still maintains the high-speed rendering of the pre-trained model.
SC6D: Symmetry-agnostic and Correspondence-free 6D Object Pose Estimation
The pose estimation is decomposed into three sub-tasks: a) object 3D rotation representation learning and matching; b) estimation of the 2D location of the object center; and c) scale-invariant distance estimation (the translation along the z-axis) via classification.
Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision
This paper focuses on perceiving and navigating 3D environments using echoes and RGB image.
Online panoptic 3D reconstruction as a Linear Assignment Problem
Real-time holistic scene understanding would allow machines to interpret their surrounding in a much more detailed manner than is currently possible.
AxIoU: An Axiomatically Justified Measure for Video Moment Retrieval
First, it is rank-insensitive: It ignores the rank positions of successfully localised moments in the top-$K$ ranked list by treating the list as a set.
Optimal Correction Cost for Object Detection Evaluation
OC-cost computes the cost of correcting detections to ground truths as a measure of accuracy.
Fast Neural Architecture Search for Lightweight Dense Prediction Networks
We present LDP, a lightweight dense prediction neural architecture search (NAS) framework.
OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimation
This paper proposes a universal framework, called OVE6D, for model-based 6D object pose estimation from a single depth image and a target object mask.
Adaptation and Attention for Neural Video Coding
In this work, we propose an end-to-end learned video codec that introduces several architectural novelties as well as training novelties, revolving around the concepts of adaptation and attention.
Taming Visually Guided Sound Generation
In this work, we propose a single model capable of generating visually relevant, high-fidelity sounds prompted with a set of frames from open-domain videos in less time than it takes to play it on a single GPU.
Towards a Real-Time Facial Analysis System
Facial analysis is an active research area in computer vision, with many practical applications.
V-SlowFast Network for Efficient Visual Sound Separation
The objective of this paper is to perform visual sound separation: i) we study visual sound separation on spectrograms of different temporal resolutions; ii) we propose a new light yet efficient three-stream framework V-SlowFast that operates on Visual frame, Slow spectrogram, and Fast spectrogram.
Monocular Depth Estimation Primed by Salient Point Detection and Normalized Hessian Loss
In addition, we introduce a normalized Hessian loss term invariant to scaling and shear along the depth direction, which is shown to substantially improve the accuracy.
Lightweight Monocular Depth with a Novel Neural Architecture Search Method
This paper presents a novel neural architecture search method, called LiDNAS, for generating lightweight monocular depth estimation models.
Learned Image Coding for Machines: A Content-Adaptive Approach
One possible solution approach consists of adapting current human-targeted image and video coding standards to the use case of machine consumption.
Image coding for machines: an end-to-end learned approach
Over recent years, deep learning-based computer vision systems have been applied to images at an ever-increasing pace, oftentimes representing the only type of consumption for those images.
On the Importance of Encrypting Deep Features
On the other hand, it is advisable to encrypt feature vectors, especially for a machine learning model in production.
HybVIO: Pushing the Limits of Real-time Visual-inertial Odometry
We present HybVIO, a novel hybrid approach for combining filtering-based visual-inertial odometry (VIO) with optimization-based SLAM.
FlipReID: Closing the Gap between Training and Inference in Person Re-Identification
More specifically, models using the FlipReID structure are trained on the original images and the flipped images simultaneously, and incorporating the flipping loss minimizes the mean squared error between feature vectors of corresponding image pairs.
Ranked #3 on
Person Re-Identification
on MSMT17
Sample selection for efficient image annotation
Supervised object detection has been proven to be successful in many benchmark datasets achieving human-level performances.
Selective Probabilistic Classifier Based on Hypothesis Testing
In this paper, we propose a simple yet effective method to deal with the violation of the Closed-World Assumption for a classifier.
Visually Guided Sound Source Separation and Localization using Self-Supervised Motion Representations
The objective of this paper is to perform audio-visual sound source separation, i. e.~to separate component audios from a mixture based on the videos of sound sources.
Optical Flow Estimation
Visually Guided Sound Source Separation
Single Source One Shot Reenactment using Weighted motion From Paired Feature Points
Image reenactment is a task where the target object in the source image imitates the motion represented in the driving image.
Boosting Monocular Depth Estimation with Lightweight 3D Point Fusion
In this paper, we propose enhancing monocular depth estimation by adding 3D points as depth guidance.
RGBD-Net: Predicting color and depth images for novel views synthesis
We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network.
FACEGAN: Facial Attribute Controllable rEenactment GAN
However, if the identity differs, the driving facial structures leak to the output distorting the reenactment result.
Uncovering Hidden Challenges in Query-Based Video Moment Retrieval
In this paper, we present a series of experiments assessing how well the benchmark results reflect the true progress in solving the moment retrieval task.
Learning to Learn to Compress
In a second phase, the Model-Agnostic Meta-learning approach is adapted to the specific case of image compression, where the inner-loop performs latent tensor overfitting, and the outer loop updates both encoder and decoder neural networks based on the overfitting performance.
Leveraging Category Information for Single-Frame Visual Sound Source Separation
Furthermore, our models are able to exploit the information of the sound source category in the separation process.
Visually Guided Sound Source Separation using Cascaded Opponent Filter Network
A key element in COF is a novel opponent filter module that identifies and relocates residual components between sources.
A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer
We show the effectiveness of the proposed model with audio and visual modalities on the dense video captioning task, yet the module is capable of digesting any two modalities in a sequence-to-sequence task.
Sequential View Synthesis with Transformer
This paper addresses the problem of novel view synthesis by means of neural rendering, where we are interested in predicting the novel view at an arbitrary camera pose based on a given set of input images from other viewpoints.
Guiding Monocular Depth Estimation Using Depth-Attention Volume
Recovering the scene depth from a single image is an ill-posed problem that requires additional priors, often referred to as monocular depth cues, to disambiguate different 3D interpretations.
Multi-modal Dense Video Captioning
We apply automatic speech recognition (ASR) system to obtain a temporally aligned textual description of the speech (similar to subtitles) and treat it as a separate input alongside video frames and the corresponding audio track.
Ranked #11 on
Dense Video Captioning
on ActivityNet Captions
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction
Our results suggest that (1) audio is a strong contributing cue for saliency prediction, (2) salient visible sound-source is the natural cause of the superiority of our Audio-Visual model, (3) richer feature representations for the input space leads to more powerful predictions even in absence of more sophisticated saliency decoders, and (4) Audio-Visual model improves over 53. 54\% of the frames predicted by the best Visual model (our baseline).
Digging Deeper into Egocentric Gaze Prediction
Extensive experiments over multiple datasets reveal that (1) spatial biases are strong in egocentric videos, (2) bottom-up saliency models perform poorly in predicting gaze and underperform spatial biases, (3) deep features perform better compared to traditional features, (4) as opposed to hand regions, the manipulation point is a strong influential cue for gaze prediction, (5) combining the proposed recurrent model with bottom-up cues, vanishing points and, in particular, manipulation point results in the best gaze prediction accuracy over egocentric videos, (6) the knowledge transfer works best for cases where the tasks or sequences are similar, and (7) task and activity recognition can benefit from gaze prediction.
Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data
Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief.
Predicting Novel Views Using Generative Adversarial Query Network
The problem of predicting a novel view of the scene using an arbitrary number of observations is a challenging problem for computers as well as for humans.
ICface: Interpretable and Controllable Face Reenactment Using GANs
This paper presents a generic face animator that is able to control the pose and expressions of a given face image.
Rethinking the Evaluation of Video Summaries
Video summarization is a technique to create a short skim of the original video while preserving the main stories/content.
DGC-Net: Dense Geometric Correspondence Network
This paper addresses the challenge of dense pixel correspondence estimation between two images.
Ranked #2 on
Dense Pixel Correspondence Estimation
on HPatches
Camera Pose Estimation
Dense Pixel Correspondence Estimation
+2
ADVIO: An authentic dataset for visual-inertial odometry
The lack of realistic and open benchmarking datasets for pedestrian visual-inertial odometry has made it hard to pinpoint differences in published methods.
Learning image-to-image translation using paired and unpaired training samples
In this paper, we propose a new general purpose image-to-image translation model that is able to utilize both paired and unpaired training data simultaneously.
Image Patch Matching Using Convolutional Descriptors with Euclidean Distance
In this work we propose a neural network based image descriptor suitable for image patch matching, which is an important task in many computer vision applications.
Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach
Here, we also report a radiological OA diagnosis area under the ROC curve of 0. 93.
Summarization of User-Generated Sports Video by Using Deep Action Recognition Features
The labels are provided by annotators possessing different experience with respect to Kendo to demonstrate how the proposed method adapts to different needs.
PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
This paper presents a novel method for visual-inertial odometry.
Investigating Natural Image Pleasantness Recognition using Deep Features and Eye Tracking for Loosely Controlled Human-computer Interaction
To investigate the current status in regard to affective image tagging, we (1) introduce a new eye movement dataset using an affordable eye tracker, (2) study the use of deep neural networks for pleasantness recognition, (3) investigate the gap between deep features and eye movements.
Image-based Localization using Hourglass Networks
In this paper, we propose an encoder-decoder convolutional neural network (CNN) architecture for estimating camera pose (orientation and location) from a single RGB-image.
Inertial Odometry on Handheld Smartphones
Building a complete inertial navigation system using the limited quality data provided by current smartphones has been regarded challenging, if not impossible.
Relative Camera Pose Estimation Using Convolutional Neural Networks
This paper presents a convolutional neural network based approach for estimating the relative pose between two cameras.
A novel method for automatic localization of joint area on knee plain radiographs
The obtained results for the used datasets show the mean intersection over the union equal to: 0. 84, 0. 79 and 0. 78.
Exploiting inter-image similarity and ensemble of extreme learners for fixation prediction using deep features
This paper presents a novel fixation prediction and saliency modeling framework based on inter-image similarities and ensemble of Extreme Learning Machines (ELM).
Video Summarization using Deep Semantic Features
For this, we design a deep neural network that maps videos as well as descriptions to a common semantic space and jointly trained it with associated pairs of videos and descriptions.
Real-time Human Pose Estimation from Video with Convolutional Neural Networks
In this paper, we present a method for real-time multi-person human pose estimation from video by utilizing convolutional neural networks.
Learning Joint Representations of Videos and Sentences with Web Image Search
In description generation, the performance level is comparable to the current state-of-the-art, although our embeddings were trained for the retrieval tasks.
Understanding Objects in Detail with Fine-Grained Attributes
We show that the collected data can be used to study the relation between part detection and attribute prediction by diagnosing the performance of classifiers that pool information from different parts of an object.
Generating Object Segmentation Proposals using Global and Local Search
The parameters of the graph cut problems are learnt in such a manner that they provide complementary sets of regions.
Fine-Grained Visual Classification of Aircraft
This paper introduces FGVC-Aircraft, a new dataset containing 10, 000 images of aircraft spanning 100 aircraft models, organised in a three-level hierarchy.
