Search Results for author:
Found 50 papers, 19 papers with code
Learn Your Scales: Towards Scale-Consistent Generative Novel View Synthesis
We study the effect of scene scale ambiguity in GNVS when sampled from a single image by isolating its effect on the resulting models and, based on these intuitions, define new metrics that measure the scale inconsistency of generated views.
Geometry-Aware Diffusion Models for Multiview Scene Inpainting
In this paper, we focus on 3D scene inpainting, where parts of an input image set, captured from different viewpoints, are masked out.
Visual Concept Connectome (VCC): Open World Concept Discovery and their Interlayer Connections in Deep Models
Understanding what deep network models capture in their learned representations is a fundamental challenge in computer vision.
PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesis
This paper considers the problem of generative novel view synthesis (GNVS), generating novel, plausible views of a scene given a limited number of known views.
Understanding Video Transformers via Universal Concept Discovery
Concretely, we seek to explain the decision-making process of video transformers based on high-level, spatiotemporal concepts that are automatically discovered.
Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations
The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster.
GePSAn: Generative Procedure Step Anticipation in Cooking Videos
While most previous work focus on the problem of data scarcity in procedural video datasets, another core challenge of future anticipation is how to account for multiple plausible future realizations in natural settings.
Dual-Camera Joint Deblurring-Denoising
Recent image enhancement methods have shown the advantages of using a pair of long and short-exposure images for low-light photography.
Watch Your Steps: Local Image and Scene Editing by Text Instructions
A field is trained on relevance maps of training views, denoted as the relevance field, defining the 3D region within which modifications should be made.
StepFormer: Self-supervised Step Discovery and Localization in Instructional Videos
This motivates the need to temporally localize the instruction steps in such videos, i. e. the task called key-step localization.
Long-Term Photometric Consistent Novel View Synthesis with Diffusion Models
In this paper, we propose a novel generative model capable of producing a sequence of photorealistic images consistent with a specified camera trajectory, and a single starting image.
Reference-guided Controllable Inpainting of Neural Radiance Fields
The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools.
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields
We refer to this task as 3D inpainting.
Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networks
(ii) Some datasets that are assumed to be biased toward dynamics are actually biased toward static information.
SAGE: Saliency-Guided Mixup with Optimal Rearrangements
For image classification, the most popular data augmentation techniques range from simple photometric and geometrical transformations, to more complex methods that use visual saliency to craft new training examples.
A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Information
To show the efficacy of our approach, we analyse two widely studied tasks, action recognition and video object segmentation.
P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision
Our model is based on a transformer equipped with a memory module, which maps the start and goal observations to a sequence of plausible actions.
Uncertainty-based Cross-Modal Retrieval with Probabilistic Representations
Probabilistic embeddings have proven useful for capturing polysemous word meanings, as well as ambiguity in image matching.
Semantic keypoint-based pose estimation from single RGB frames
Empirically, we show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios even against a cluttered background.
Temporal Transductive Inference for Few-Shot Video Object Segmentation
In this paper, we present a simple but effective temporal transductive inference (TTI) approach that leverages temporal consistency in the unlabelled video frames during few-shot inference.
Simpler Does It: Generating Semantic Labels with Objectness Guidance
Extensive experiments demonstrate the high quality of our generated pseudo-labels and effectiveness of the proposed framework in a variety of domains.
Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers
In our experiments, we show that Drop-DTW is a robust similarity measure for sequence retrieval and demonstrate its effectiveness as a training loss on diverse applications.
SegMix: Co-occurrence Driven Mixup for Semantic Segmentation and Adversarial Robustness
The premise is based on the notion of feature binding, which is defined as the process by which activations spread across space and layers in the network are successfully integrated to arrive at a correct inference decision.
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs
In this paper, we challenge the common assumption that collapsing the spatial dimensions of a 3D (spatial-channel) tensor in a convolutional neural network (CNN) into a vector via global pooling removes all spatial information.
Representation Learning via Global Temporal Alignment and Cycle-Consistency
We introduce a weakly supervised method for representation learning based on aligning temporal sequences (e. g., videos) of the same process (e. g., human action).
Stochastic Image-to-Video Synthesis using cINNs
Video understanding calls for a model to learn the characteristic interplay between static scene content and its dynamics: Given an image, the model must be able to predict a future progression of the portrayed scene and, conversely, a video should be explained in terms of its static image content and all the remaining characteristics not present in the initial frame.
Position, Padding and Predictions: A Deeper Look at Position Information in CNNs
; (ii) Does position encoding affect the learning of semantic representations?
Shape or Texture: Understanding Discriminative Features in CNNs
Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs actually exhibit a `texture bias': given an image with both texture and shape cues (e. g., a stylized image), a CNN is biased towards predicting the category corresponding to the texture.
Shape or Texture: Disentangling Discriminative Features in CNNs
Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs actually exhibit a 'texture bias': given an image with both texture and shape cues (e. g., a stylized image), a CNN is biased towards predicting the category corresponding to the texture.
Boundary Effects in CNNs: Feature or Bug?
Finally, we demonstrate the implications of these findings on a number of real-world tasks to show that position information can act as a feature or a bug.
Cycle-Consistent Generative Rendering for 2D-3D Modality Translation
In the context of computer vision, this corresponds to a learnable module that serves two purposes: (i) generate a realistic rendering of a 3D object (shape-to-image translation) and (ii) infer a realistic 3D shape from an image (image-to-shape translation).
Wavelet Flow: Fast Training of High Resolution Normalizing Flows
Normalizing flows are a class of probabilistic generative models which allow for both fast density computation and efficient sampling and are effective at modelling complex distributions like images.
Feature Binding with Category-Dependant MixUp for Semantic Segmentation and Adversarial Robustness
In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical information throughout the network.
Learning Multi-Scale Photo Exposure Correction
In contrast, our proposed method targets both over- and underexposure errors in photographs.
Ranked #4 on
Image Enhancement
on Exposure-Errors
Keyframing the Future: Discovering Temporal Hierarchy with Keyframe-Inpainter Prediction
To flexibly and efficiently reason about temporal sequences, abstract representations that compactly represent the important information in the sequence are needed.
End-to-End Learning of Representations for Asynchronous Event-Based Data
Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events".
Ranked #3 on
Classification
on N-CARS
(using extra training data)
Keyframing the Future: Keyframe Discovery for Visual Prediction and Planning
We propose a model that learns to discover these important events and the times when they occur and uses them to represent the full sequence.
Joint Spatial and Layer Attention for Convolutional Networks
In this paper, we propose a novel approach that learns to sequentially attend to different Convolutional Neural Networks (CNN) layers (i. e., ``what'' feature abstraction to attend to) and different spatial locations of the selected feature map (i. e., ``where'') to perform the task at hand.
Learning what you can do before doing anything
We introduce a loss term that encourages the network to capture the composability of visual sequences and show that it leads to representations that disentangle the structure of actions.
SIPs: Succinct Interest Points from Unsupervised Inlierness Probability Learning
In certain cases, our detector is able to obtain an equivalent amount of inliers with as little as 60% of the amount of points of other detectors.
Predicting the Future with Transformational States
We couple this latent state with a recurrent neural network (RNN) core that predicts future frames by transforming past states into future states by applying the accumulated state transformation with a learned operator.
Segmentation-Aware Convolutional Networks Using Local Attention Masks
We introduce an approach to integrate segmentation information within a convolutional neural network (CNN).
Two-Stream Convolutional Networks for Dynamic Texture Synthesis
Given an input dynamic texture, statistics of filter responses from the object recognition ConvNet encapsulate the per-frame appearance of the input texture, while statistics of filter responses from the optical flow ConvNet model its dynamics.
Harvesting Multiple Views for Marker-less 3D Human Pose Annotations
In this paper, we present a geometry-driven approach to automatically collect annotations for human pose prediction tasks.
Ranked #28 on
Weakly-supervised 3D Human Pose Estimation
on Human3.6M
6-DoF Object Pose from Semantic Keypoints
This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image.
Ranked #1 on
Keypoint Detection
on Pascal3D+
Building Usage Profiles Using Deep Neural Nets
Our goal is to construct an automatic approach to extract information about user actions from instructional videos.
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
This paper addresses the challenge of 3D human pose estimation from a single color image.
Ranked #18 on
3D Human Pose Estimation
on HumanEva-I
Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
Recently, convolutional networks (convnets) have proven useful for predicting optical flow.
Learning Dense Convolutional Embeddings for Semantic Segmentation
That is, for any two pixels on the same object, the embeddings are trained to be similar; for any pair that straddles an object boundary, the embeddings are trained to be dissimilar.
Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval
This paper presents a new state-of-the-art for document image classification and retrieval, using features learned by deep convolutional neural networks (CNNs).
