Search Results for author:
Found 85 papers, 25 papers with code
Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models
Beyond high-fidelity image synthesis, diffusion models have recently exhibited promising results in dense visual perception tasks.
ReferEverything: Towards Segmenting Everything We Can Speak of in Videos
We present REM, a framework for segmenting a wide range of concepts in video that can be described through natural language.
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding
To address this issue, we present a comprehensive study that probes various visual encoding models for 3D scene understanding, identifying the strengths and limitations of each model across different scenarios.
Ranked #3 on
Question Answering
on SQA3D
Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models
Despite recent significant strides achieved by diffusion-based Text-to-Image (T2I) models, current systems are still less capable of ensuring decent compositional generation aligned with text prompts, particularly for the multi-object generation.
Multi-task View Synthesis with Neural Radiance Fields
To tackle the MTVS problem, we propose MuvieNeRF, a framework that incorporates both multi-task and cross-view knowledge to simultaneously synthesize multiple scene properties.
Flexible Techniques for Differentiable Rendering with 3D Gaussians
Fast, reliable shape reconstruction is an essential ingredient in many computer vision applications.
Optimizing Algorithms From Pairwise User Preferences
Typical black-box optimization approaches in robotics focus on learning from metric scores.
Thinking Like an Annotator: Generation of Dataset Labeling Instructions
In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples.
Discovering Multiple Algorithm Configurations
In this work, we extend algorithm configuration to automatically discover multiple modes in the tuning dataset.
Deep Projective Rotation Estimation through Relative Supervision
While self-supervised learning has been used successfully for translational object keypoints, in this work, we show that naively applying relative supervision to the rotational group $SO(3)$ will often fail to converge due to the non-convexity of the rotational space.
Learning Continuous Implicit Representation for Near-Periodic Patterns
To address these challenges, we learn a neural implicit representation using a coordinate-based MLP with single image optimization.
Approximate Differentiable Rendering with Algebraic Surfaces
In this work, we develop an approximate differentiable renderer for a compact, interpretable representation, which we call Fuzzy Metaballs.
Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields
Comprehensive 3D scene understanding, both geometrically and semantically, is important for real-world applications such as robot perception.
Discovering Objects that Can Move
Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects.
Generative Modeling for Multitask Visual Learning
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
Generative Modeling for Multi-task Visual Learning
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
Learning to Track Object Position through Occlusion
Occlusion is one of the most significant challenges encountered by object detectors and trackers.
ZePHyR: Zero-shot Pose Hypothesis Rating
We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation.
Learning To Hallucinate Examples From Extrinsic and Intrinsic Supervision
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks.
PanoNet3D: Combining Semantic and Geometric Understanding for LiDARPoint Cloud Detection
Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure.
Constrained Model-based Reinforcement Learning with Robust Cross-Entropy Method
We propose a model-based approach to enable RL agents to effectively explore the environment with unknown system dynamics and environment constraints given a significantly small number of violation budgets.
Model-based Reinforcement Learning
Model Predictive Control
+4
Few-Shot Learning with Intra-Class Knowledge Transfer
Recent works have proposed to solve this task by augmenting the training data of the few-shot classes using generative models with the few-shot training samples as the seeds.
AlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes.
Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis
We propose a novel task of joint few-shot recognition and novel-view synthesis: given only one or few images of a novel object from arbitrary views with only category annotation, we aim to simultaneously learn an object classifier and generate images of that type of object from new viewpoints.
PanoNet: Real-time Panoptic Segmentation through Position-Sensitive Feature Embedding
We propose a simple, fast, and flexible framework to generate simultaneously semantic and instance masks for panoptic segmentation.
MAPPER: Multi-Agent Path Planning with Evolutionary Reinforcement Learning in Mixed Dynamic Environments
Multi-agent navigation in dynamic environments is of great industrial value when deploying a large scale fleet of robot to real-world applications.
Learning Orientation Distributions for Object Pose Estimation
Our first method, which regresses from deep learned features to an isotropic Bingham distribution, gives the best performance for orientation distribution estimation for non-symmetric objects.
Unsupervised Learning of Video Representations via Dense Trajectory Clustering
This paper addresses the task of unsupervised learning of representations for action recognition in videos.
Unlocking the Full Potential of Small Data with Diverse Supervision
Indeed, even the majority of few-shot learning methods rely on a large set of "base classes" for pretraining.
Explainable Semantic Mapping for First Responders
One of the key challenges in the semantic mapping problem in postdisaster environments is how to analyze a large amount of data efficiently with minimal supervision.
Meta-Learning by Hallucinating Useful Examples
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks, which aim to learn novel concepts from very few examples.
Quadtree Generating Networks: Efficient Hierarchical Scene Parsing with Sparse Convolutions
Semantic segmentation with Convolutional Neural Networks is a memory-intensive task due to the high spatial resolution of feature maps and output predictions.
Growing a Brain: Fine-Tuning by Increasing Model Capacity
One of their remarkable properties is the ability to transfer knowledge from a large source dataset to a (typically smaller) target dataset.
Edge-Direct Visual Odometry
In contrast our method builds on direct visual odometry methods naturally with minimal added computation.
Image Deformation Meta-Networks for One-Shot Learning
Humans can robustly learn novel visual concepts even when images undergo various deformations and lose certain information.
Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency
We demonstrate our ability to learn MVS without 3D supervision using a real dataset, and show that each component of our proposed robust loss results in a significant improvement.
A Study on Action Detection in the Wild
In this work we study the problem of action detection in a highly-imbalanced dataset.
Direct Fitting of Gaussian Mixture Models
Part of this work analyzes a general formulation for evaluating likelihood of geometric objects.
Learning Compositional Representations for Few-Shot Recognition
One of the key limitations of modern deep learning approaches lies in the amount of data required to train them.
A Structured Model For Action Detection
A dominant paradigm for learning-based approaches in computer vision is training generic models, such as ResNet for image recognition, or I3D for video understanding, on large datasets and allowing them to discover the optimal representation for the problem at hand.
Iterative Transformer Network for 3D Point Cloud
Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation.
Adaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels
With our design, the network progressively learns features specific to the target domain using annotation from only the source domain.
PCN: Point Completion Network
Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications.
Ranked #1 on
Point Cloud Completion
on Completion3D
Deep Material-Aware Cross-Spectral Stereo Matching
Often, multiple cameras are used for cross-spectral imaging, thus requiring image alignment, or disparity estimation in a stereo setting.
Low-Shot Learning from Imaginary Data
Humans can quickly learn new visual concepts, perhaps because they can easily visualize or imagine what novel objects look like from different views.
Anytime Neural Network: a Versatile Trade-off Between Computation and Accuracy
We present an approach for anytime predictions in deep neural networks (DNNs).
Learning by Asking Questions
We also show that our model asks questions that generalize to state-of-the-art VQA models and to novel test time distributions.
Learning to Model the Tail
We cast this problem as transfer learning, where knowledge from the data-rich classes in the head of the distribution is transferred to the data-poor classes in the tail.
Log-DenseNet: How to Sparsify a DenseNet
Skip connections are increasingly utilized by deep neural networks to improve accuracy and cost-efficiency.
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
Ignoring Distractors in the Absence of Labels: Optimal Linear Projection to Remove False Positives During Anomaly Detection
We demonstrate that this method is able to remove uninformative parts of the feature space for the anomaly detection setting.
Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing
Experimentally, the adaptive weights induce more competitive anytime predictions on multiple recognition data-sets and models than non-adaptive approaches including weighing all losses equally.
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort.
From Red Wine to Red Tomato: Composition With Context
In this paper, we present a simple method that respects contextuality in order to compose classifiers of known visual concepts.
The Pose Knows: Video Forecasting by Generating Pose Futures
First we explicitly model the high level structure of active objects in the scene---humans---and use a VAE to model the possible future movements of humans in the pose space.
Ranked #2 on
Human Pose Forecasting
on Human3.6M
(CMD metric)
Gradient Boosting on Stochastic Data Streams
To generalize from batch to online, we first introduce the definition of online weak learning edge with which for strongly convex and smooth loss functions, we present an algorithm, Streaming Gradient Boosting (SGB) with exponential shrinkage guarantees in the number of weak learners.
Contextual Visual Similarity
Given a query image, a second positive image and a third negative image, dissimilar to the first two images, we define a contextualized similarity search criteria.
General models for rational cameras and the case of two-slit projections
The rational camera model recently introduced in [19] provides a general methodology for studying abstract nonlinear imaging systems and their multi-view geometry.
Learning from Small Sample Sets by Combining Unsupervised Meta-Training with CNNs
Inspired by the transferability properties of CNNs, we introduce an additional unsupervised meta-training stage that exposes multiple top layer units to a large amount of unlabeled real-world images.
A Discriminative Framework for Anomaly Detection in Large Videos
We address an anomaly detection setting in which training sequences are unavailable and anomalies are scored independently of temporal ordering.
Learning Transferable Policies for Monocular Reactive MAV Control
The ability to transfer knowledge gained in previous tasks into new contexts is one of the most important mechanisms of human learning.
Introspective Perception: Learning to Predict Failures in Vision Systems
As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical.
An Uncertain Future: Forecasting from Static Images using Variational Autoencoders
We show that our method is able to successfully predict events in a wide variety of scenes and can produce multiple different predictions when the future is ambiguous.
Consistency of Silhouettes and Their Duals
Silhouettes provide rich information on three-dimensional shape, since the intersection of the associated visual cones generates the "visual hull", which encloses and approximates the original shape.
Cross-stitch Networks for Multi-task Learning
In this paper, we propose a principled approach to learn shared representations in ConvNets using multi-task learning.
Ranked #118 on
Semantic Segmentation
on NYU Depth v2
Shuffle and Learn: Unsupervised Learning using Temporal Order Verification
With this simple task and no semantic labels, we learn a powerful visual representation using a Convolutional Neural Network (CNN).
Ranked #48 on
Self-Supervised Action Recognition
on HMDB51
Learning to Extract Motion from Videos in Convolutional Neural Networks
Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation.
Predicting Multiple Structured Visual Interpretations
We present a simple approach for producing a small number of structured visual outputs which have high recall, for a variety of tasks including monocular pose estimation and semantic scene segmentation.
Single Image 3D Without a Single 3D Image
Do we really need 3D labels in order to learn how to predict 3D?
The Joint Image Handbook
Given multiple perspective photographs, point correspondences form the "joint image", effectively a replica of three dimensional space distributed across its two-dimensional projections.
Inferring 3D Layout of Building Facades From a Single Image
In this paper, we propose a novel algorithm that infers the 3D layout of building facades from a single 2D image of an urban scene.
Model Recommendation: Generating Object Detectors From Few Samples
In this paper, we explore an approach to generating detectors that is radically different from the conventional way of learning a detector from a large corpus of annotated positive and negative data samples.
Dense Optical Flow Prediction from a Static Image
Because our CNN model makes no assumptions about the underlying scene, it can predict future optical flow on a diverse set of scenarios.
Vision and Learning for Deliberative Monocular Cluttered Flight
Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs).
Visual Chunking: A List Prediction Framework for Region-Based Object Detection
We present an efficient algorithm with provable performance for building a high-quality list of detections from any candidate set of region-based proposals.
Efficient Feature Group Sequencing for Anytime Linear Prediction
We theoretically guarantee that our algorithms achieve near-optimal linear predictions at each budget when a feature group is chosen.
Patch to the Future: Unsupervised Visual Prediction
In this paper we present a conceptually simple but surprisingly powerful method for visual prediction which combines the effectiveness of mid-level visual elements with temporal modeling.
Predicting Failures of Vision Systems
We show that a surprisingly straightforward and general approach, that we call ALERT, can predict the likely accuracy (or failure) of a variety of computer vision systems semantic segmentation, vanishing point and camera parameter estimation, and image memorability prediction on individual input images.
Trinocular Geometry Revisited
When do the visual rays associated with triplets of point correspondences converge, that is, intersect in a common point?
SpeedMachines: Anytime Structured Prediction
Structured prediction plays a central role in machine learning applications from computational biology to computer vision.
Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces
There has been a recent push in extraction of 3D spatial layout of scenes.
An Integer Projected Fixed Point Method for Graph Matching and MAP Inference
When applied to MAP inference, the algorithm is a parallel extension of Iterated Conditional Modes (ICM) with climbing and convergence properties that make it a compelling alternative to the sequential ICM.
