Search Results for author:
Found 49 papers, 23 papers with code
Navigating Hallucinations for Reasoning of Unintentional Activities
In this work we present a novel task of understanding unintentional human activities in videos.
DreamDistribution: Prompt Distribution Learning for Text-to-Image Diffusion Models
We introduce a solution that allows a pretrained T2I diffusion model to learn a set of soft prompts, enabling the generation of novel images by sampling prompts from the learned distribution.
PEEKABOO: Interactive Video Generation via Masked-Diffusion
Recently there has been a lot of progress in text-to-video generation, with state-of-the-art models being capable of generating high quality, realistic videos.
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Construction of a universal detector poses a crucial question: How can we most effectively train a model on a large mixture of datasets?
Efficiently Robustify Pre-trained Models
A recent trend in deep learning algorithms has been towards training large scale models, having high parameter count and trained on big dataset.
Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation
A foreground-background segmentation algorithm is then used to generate foreground object masks.
Robustness Analysis on Foundational Segmentation Models
Due to the increase in computational resources and accessibility of data, an increase in large, deep learning models trained on copious amounts of data using self-supervised or semi-supervised learning have emerged.
A Large-Scale Analysis on Self-Supervised Video Representation Learning
In this work, we first provide a benchmark that enables a comparison of existing approaches on the same ground.
Ranked #3 on
Self-Supervised Action Recognition
on UCF101
Controllable Text-to-Image Generation with GPT-4
Control-GPT works by querying GPT-4 to write TikZ code, and the generated sketches are used as references alongside the text instructions for diffusion models (e. g., ControlNet) to generate photo-realistic images.
PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
A rich representation is key to general robotic manipulation, but existing approaches to representation learning require large amounts of multimodal demonstrations.
Exploring the Sim2Real Gap Using Digital Twins
An emerging alternative is to use synthetic data, but if the synthetic data is not similar enough to the real data, the performance is typically below that of training with real data.
A Large-Scale Robustness Analysis of Video Action Recognition Models
In this work, we perform a large-scale robustness analysis of these existing models for video action recognition.
Benchmarking Spatial Relationships in Text-to-Image Generation
We investigate the ability of T2I models to generate correct spatial relationships among objects and present VISOR, an evaluation metric that captures how accurately the spatial relationship described in text is generated in the image.
EM-Paste: EM-guided Cut-Paste with DALL-E Augmentation for Image-level Weakly Supervised Instance Segmentation
Finally, the third component creates a large-scale pseudo-labeled instance segmentation training dataset by compositing the foreground object masks onto the original and generated background images.
Instance-Aware Image Completion
However, existing image completion methods tend to fill in the missing region with the surrounding texture instead of hallucinating a visual instance that is suitable in accordance with the context of the scene.
Learning to Simulate Realistic LiDARs
Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling.
Neural-Sim: Learning to Generate Training Data with NeRF
However, existing approaches either require human experts to manually tune each scene property or use automatic methods that provide little to no control; this requires rendering large amounts of random data variations, which is slow and is often suboptimal for the target domain.
TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
3D scene graphs (3DSGs) are an emerging description; unifying symbolic, topological, and metric scene representations.
Scaling Novel Object Detection with Weakly Supervised Detection Transformers
A critical object detection task is finetuning an existing model to detect novel objects, but the standard workflow requires bounding box annotations which are time-consuming and expensive to collect.
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Joint visual and language modeling on large-scale datasets has recently shown good progress in multi-modal tasks when compared to single modal learning.
Large-scale Robustness Analysis of Video Action Recognition Models
In this work, we perform a large-scale robustness analysis of these existing models for video action recognition.
DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection
For foreground object mask generation, we use a simple textual template with object class name as input to DALL-E to generate a diverse set of foreground images.
Missingness Bias in Model Debugging
Missingness, or the absence of features from an input, is a concept fundamental to many model debugging tools.
Image Retrieval from Contextual Descriptions
In particular, models are tasked with retrieving the correct image from a set of 10 minimally contrastive candidates based on a contextual description.
Ranked #1 on
Image Retrieval
on ImageCoDe
Inferring Articulated Rigid Body Dynamics from RGBD Video
Being able to reproduce physical phenomena ranging from light interaction to contact mechanics, simulators are becoming increasingly useful in more and more application domains where real-world interaction or labeled data are difficult to obtain.
One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learning
We conduct a large-scale study with over 100 variants of ResNet and MobileNet architectures and evaluate them across 11 downstream scenarios in the SSL setting.
Robust Contrastive Learning against Noisy Views
Contrastive learning relies on an assumption that positive pairs contain related views, e. g., patches of an image or co-occurring multimodal signals of a video, that share certain underlying information about an instance.
Learning to Align Sequential Actions in the Wild
To this end, we propose an approach to enforce temporal priors on the optimal transport matrix, which leverages temporal consistency, while allowing for variations in the order of actions.
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
The ability to perform causal and counterfactual reasoning are central properties of human intelligence.
3DB: A Framework for Debugging Computer Vision Models
We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation.
RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs
In this work, we introduce a Resource Aware Neuron Pruning (RANP) algorithm that prunes 3D CNNs at initialization to high sparsity levels.
Prediction of Object Geometry from Acoustic Scattering Using Convolutional Neural Networks
The present work proposes a method to infer object geometry from scattering features by training convolutional neural networks.
RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs
Specifically, the core idea is to obtain an importance score for each neuron based on their sensitivity to the loss function.
AutoSimulate: (Quickly) Learning Synthetic Data Generation
Simulation is increasingly being used for generating large labelled datasets in many machine learning problems.
Depth Completion Using a View-constrained Deep Prior
Given color images and noisy and incomplete target depth maps, we optimize a randomly-initialized CNN model to reconstruct a depth map restored by virtue of using the CNN network structure as a prior combined with a view-constrained photo-consistency loss.
Fast acoustic scattering using convolutional neural networks
Diffracted scattering and occlusion are important acoustic effects in interactive auralization and noise control applications, typically requiring expensive numerical simulation.
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors.
Live Reconstruction of Large-Scale Dynamic Outdoor Worlds
For each time step, our dynamic map maintains a relative pose of each volume with respect to the stationary background.
Privacy-Preserving Action Recognition using Coded Aperture Videos
We propose a lens-free coded aperture camera system for human action recognition that is privacy-preserving.
Photorealistic Image Synthesis for Object Instance Detection
We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images.
Playing for Data: Ground Truth from Computer Games
Recent progress in computer vision has been driven by high-capacity models trained on large datasets.
Feature Space Optimization for Semantic Video Segmentation
We present an approach to long-range spatio-temporal regularization in semantic video segmentation.
Dense Monocular Depth Estimation in Complex Dynamic Scenes
We present an approach to dense depth estimation from a single monocular camera that is moving through a dynamic scene.
SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenes
We present an open-source, real-time implementation of SemanticPaint, a system for geometric reconstruction, object-class segmentation and learning of 3D scenes.
Conditional Random Fields as Recurrent Neural Networks
Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding.
Ranked #36 on
Semantic Segmentation
on PASCAL VOC 2012 test
Dense Semantic Image Segmentation with Objects and Attributes
The concepts of objects and attributes are both important for describing images precisely, since verbal descriptions often contain both adjectives and nouns (e. g. "I see a shiny red chair').
A Tiered Move-making Algorithm for General Non-submodular Pairwise Energies
The algorithm converges to a local minimum for any general pairwise potential, and we give a theoretical analysis of the properties of the algorithm, characterizing the situations in which we can expect good performance.
Higher Order Priors for Joint Intrinsic Image, Objects, and Attributes Estimation
Many methods have been proposed to recover the intrinsic scene properties such as shape, reflectance and illumination from a single image.
ImageSpirit: Verbal Guided Image Parsing
This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images.
