Search Results for author:
Found 157 papers, 80 papers with code
DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction
We target the challenge of online 2D and 3D point tracking from unposed monocular camera input introducing Dynamic Online Monocular Reconstruction (DynOMo).
Fast and Modular Autonomy Software for Autonomous Racing Vehicles
While far from challenging what a human racecar driver can do, the IAC is pushing the state of the art by facilitating full-sized ARV competitions.
GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
In this work, we conduct an extensive human study on GenAI-Bench to evaluate the performance of leading image and video generation models in various aspects of compositional text-to-visual generation.
Simultaneous Map and Object Reconstruction
To achieve this, we take inspiration from recent novel view synthesis methods and pose the reconstruction problem as a global optimization, minimizing the distance between our predicted surface and the input LiDAR scans.
Planning with Adaptive World Models for Autonomous Driving
Motion planning is crucial for safe navigation in complex urban environments.
Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detection
Specifically, we propose a shelf-supervised approach (e. g. supervised with off-the-shelf image foundation models) for generating zero-shot 3D bounding boxes from paired RGB and LiDAR data.
Neural Representations of Dynamic Visual Stimuli
Humans experience the world through constantly changing visual stimuli, where scenes can shift and move, change in appearance, and vary in distance.
Predicting Long-horizon Futures by Conditioning on Geometry and Time
To address both challenges, our key insight is to leverage the large-scale pretraining of image diffusion models which can handle multi-modality.
Evaluating Text-to-Visual Generation with Image-to-Text Generation
For instance, the widely-used CLIPScore measures the alignment between a (generated) image and text prompt, but it fails to produce reliable scores for complex prompts involving compositions of objects, attributes, and relations.
Better Call SAL: Towards Learning to Segment Anything in Lidar
We propose the SAL (Segment Anything in Lidar) method consisting of a text-promptable zero-shot model for segmenting and classifying any object in Lidar, and a pseudo-labeling engine that facilitates model training without manual supervision.
I Can't Believe It's Not Scene Flow!
Current scene flow methods broadly fail to describe motion on small objects, and current scene flow evaluation protocols hide this failure by averaging over many points, with most drawn larger objects.
Cameras as Rays: Pose Estimation via Ray Diffusion
Estimating camera poses is a fundamental task for 3D reconstruction and remains challenging given sparsely sampled views (<10).
FlashTex: Fast Relightable Mesh Texturing with LightControlNet
We introduce LightControlNet, a new text-to-image model based on the ControlNet architecture, which allows the specification of the desired lighting as a conditioning image to the model.
The Neglected Tails in Vision-Language Models
We address this by using large language models (LLMs) to count the number of pretraining texts that contain synonyms of these concepts.
SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM
Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications.
Revisiting Few-Shot Object Detection with Vision-Language Models
Concretely, we propose Foundational FSOD, a new benchmark protocol that evaluates detectors pre-trained on any external datasets and fine-tuned on multi-modal (text and visual) K-shot examples per target class.
TAO-Amodal: A Benchmark for Tracking Any Object Amodally
Amodal perception, the ability to comprehend complete object structures from partial visibility, is a fundamental skill, even for infants.
Long-Tailed 3D Detection via 2D Late Fusion
To address this limitation, we delve into a simple late-fusion framework that ensembles independently trained uni-modal LiDAR and RGB detectors.
HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces
Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render.
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications.
DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation
Just as object pose estimation and tracking have aided robots for rigid manipulation, dense 3D tracking (scene flow) of highly deformable objects will enable new applications in robotics while aiding existing approaches, such as imitation learning or creating digital twins with real2sim transfer.
PyNeRF: Pyramidal Neural Radiance Fields
Compared to Mip-NeRF, we reduce error rates by 20% while training over 60x faster.
Lidar Panoptic Segmentation and Tracking without Bells and Whistles
Our network is modular by design and optimized for all aspects of both the panoptic segmentation and tracking task.
Streaming Motion Forecasting for Autonomous Driving
Our benchmark inherently captures the disappearance and re-appearance of agents, presenting the emergent challenge of forecasting for occluded agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks.
Language Models as Black-Box Optimizers for Vision-Language Models
We highlight the advantage of conversational feedback that incorporates both positive and negative prompts, suggesting that LLMs can utilize the implicit gradient direction in textual feedback for a more efficient search.
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements.
Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
To distill knowledge from a highly accurate but complex teacher model, we construct a sequence of teachers to help the student gradually adapt.
An Empirical Analysis of Range for 3D Object Detection
LiDAR-based 3D detection plays a vital role in autonomous navigation.
Thinking Like an Annotator: Generation of Dataset Labeling Instructions
In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples.
Revisiting the Role of Language Priors in Vision-Language Models
Our first observation is that they can be repurposed for discriminative tasks (such as image-text retrieval) by simply computing the match score of generating a particular text string given an image.
Ranked #45 on
Visual Reasoning
on Winoground
ZeroFlow: Scalable Scene Flow via Distillation
Scene flow estimation is the task of describing the 3D motion field between temporally successive point clouds.
Ranked #2 on
Self-supervised Scene Flow Estimation
on Argoverse 2
(using extra training data)
WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models
WEDGE consists of 3360 images in 16 extreme weather conditions manually annotated with 16513 bounding boxes, supporting research in the tasks of weather classification and 2D object detection.
Joint Metrics Matter: A Better Standard for Trajectory Forecasting
In response to the limitations of marginal metrics, we present the first comprehensive evaluation of state-of-the-art (SOTA) trajectory forecasting methods with respect to multi-agent metrics (joint metrics): JADE, JFDE, and collision rate.
Reconstructing Animatable Categories from Videos
Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and manual rigging, which are difficult to scale to arbitrary categories.
3D Shape Reconstruction from Videos
Dynamic Reconstruction
+1
RelPose++: Recovering 6D Poses from Sparse-view Observations
We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images).
Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis
Given a minute-long RGBD video of people interacting with their pets, we render the scene from novel camera trajectories derived from the in-scene motion of actors: (1) egocentric cameras that simulate the point of view of a target actor and (2) 3rd-person cameras that follow the actor.
Re-Evaluating LiDAR Scene Flow for Autonomous Driving
Popular benchmarks for self-supervised LiDAR scene flow (stereoKITTI, and FlyingThings3D) have unrealistic rates of dynamic motion, unrealistic correspondences, and unrealistic sampling patterns.
Learning to Zoom and Unzoom
In this work (LZU), we "learn to zoom" in on the input image, compute spatial features, and then "unzoom" to revert any deformations.
SUDS: Scalable Urban Dynamic Scenes
We extend neural radiance fields (NeRFs) to dynamic large-scale urban scenes.
Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting
One promising self-supervised task is 3D point cloud forecasting from unannotated LiDAR sequences.
3D-aware Conditional Image Synthesis
We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis.
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models
The ability to quickly learn a new task with minimal instruction - known as few-shot learning - is a central aspect of intelligent agents.
Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images
Moreover, we show that our retrieved maps can be used to update or expand existing maps and even show proof-of-concept results for visual localization and image retrieval from spatial graphs.
TarViS: A Unified Approach for Target-based Video Segmentation
A single TarViS model can be trained jointly on a collection of datasets spanning different tasks, and can hot-swap between tasks during inference without any task-specific retraining.
Ranked #2 on
Video Panoptic Segmentation
on KITTI-STEP
(using extra training data)
Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting
Models are tasked with the prediction of future motion for "scored actors" in each scenario and are provided with track histories that capture object location, heading, velocity, and category.
PPR: Physically Plausible Reconstruction from Monocular Videos
Given monocular videos, we build 3D models of articulated objects and environments whose 3D configurations satisfy dynamics and contact constraints.
Distilling Neural Fields for Real-Time Articulated Shape Reconstruction
We present a method for reconstructing articulated 3D models from videos in real-time, without test-time optimization or manual 3D supervision at training time.
Far3Det: Towards Far-Field 3D Detection
We focus on the task of far-field 3D detection (Far3Det) of objects beyond a certain distance from an observer, e. g., $>$50m.
Towards Long-Tailed 3D Detection
Moreover, semantic classes are often organized within a hierarchy, e. g., tail classes such as child and construction-worker are arguably subclasses of pedestrian.
Soft Augmentation for Image Classification
We draw inspiration from human visual classification studies and propose generalizing augmentation with invariant transforms to soft augmentation where the learning target softens non-linearly as a function of the degree of the transform applied to the sample: e. g., more aggressive image crop augmentations produce less confident learning targets.
Learning to Discover and Detect Objects
We then train our network to learn to classify each RoI, either as one of the known classes, seen in the source dataset, or one of the novel classes, with a long-tail distribution constraint on the class assignments, reflecting the natural frequency of classes in the real world.
Ranked #2 on
Novel Object Detection
on LVIS v1.0 val
Continual Learning with Evolving Class Ontologies
LECO requires learning classifiers in distinct time periods (TPs); each TP introduces a new ontology of "fine" labels that refines old ontologies of "coarse" labels (e. g., dog breeds that refine the previous ${\tt dog}$).
Differentiable Raycasting for Self-supervised Occupancy Forecasting
Self-supervised representations proposed for large-scale planning, such as ego-centric freespace, confound these two motions, making the representation difficult to use for downstream motion planners.
BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video
Multiple existing benchmarks involve tracking and segmenting objects in video e. g., Video Object Segmentation (VOS) and Multi-Object Tracking and Segmentation (MOTS), but there is little interaction between them due to the use of disparate benchmark datasets and metrics (e. g. J&F, mAP, sMOTSA).
Ranked #4 on
Long-tail Video Object Segmentation
on BURST-val
(using extra training data)
Long-tail Video Object Segmentation
Multi-Object Tracking
+6
RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild
We describe a data-driven method for inferring the camera viewpoints given multiple images of an arbitrary object.
Differentiable Soft-Masked Attention
Recently, "Masked Attention" was proposed in which a given object representation only attends to those image pixel features for which the segmentation mask of that object is active.
Forecasting from LiDAR via Future Object Detection
Object detection and forecasting are fundamental components of embodied perception.
Long-Tailed Recognition via Weight Balancing
In contrast, weight decay penalizes larger weights more heavily and so learns small balanced weights; the MaxNorm constraint encourages growing small weights within a norm ball but caps all the weights by the radius.
Ranked #9 on
Long-tail Learning
on CIFAR-100-LT (ρ=10)
The CLEAR Benchmark: Continual LEArning on Real-World Imagery
The major strength of CLEAR over prior CL benchmarks is the smooth temporal evolution of visual concepts with real-world imagery, including both high-quality labeled data along with abundant unlabeled samples per time period for continual semi-supervised learning.
Opening Up Open World Tracking
A benchmark that would allow us to perform an apple-to-apple comparison of existing efforts is a crucial first step towards advancing this important research field.
Ranked #3 on
Open-World Video Segmentation
on BURST-val
(using extra training data)
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs
We use neural radiance fields (NeRFs) to build interactive 3D environments from large-scale visual captures spanning buildings or even multiple city blocks collected primarily from drones.
HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Images
Existing state-of-the-art methods for Video Object Segmentation (VOS) learn low-level pixel-to-pixel correspondences between frames to propagate object masks across video.
ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction
The surface embeddings are implemented as coordinate-based MLPs that are fit to each video via consistency and contrastive reconstruction losses. Experimental results show that ViSER compares favorably against prior work on challenging videos of humans with loose clothing and unusual poses as well as animals videos from DAVIS and YTVOS.
NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild
NeRS learns a neural shape representation of a closed surface that is diffeomorphic to a sphere, guaranteeing water-tight reconstructions.
Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories
For machine learning models trained with limited labeled training data, validation stands to become the main bottleneck to reducing overall annotation costs.
FOVEA: Foveated Image Magnification for Autonomous Navigation
Efficient processing of high-res video streams is safety-critical for many robotics applications such as autonomous driving.
Depth-supervised NeRF: Fewer Views and Faster Training for Free
Crucially, SFM also produces sparse 3D points that can be used as "free" depth supervision during training: we add a loss to encourage the distribution of a ray's terminating depth matches a given 3D keypoint, incorporating depth uncertainty.
Safe Local Motion Planning With Self-Supervised Freespace Forecasting
Finally, we propose future freespace as an additional source of annotation-free supervision.
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Remarkable progress has been made in 3D reconstruction of rigid structures from a video or a collection of images.
Opening up Open-World Tracking
We hope to open a new front in multi-object tracking research that will hopefully bring us a step closer to intelligent systems that can operate safely in the real world.
Multimodal Object Detection via Probabilistic Ensembling
Object detection with multimodal inputs can improve many safety-critical systems such as autonomous vehicles (AVs).
Ranked #2 on
Object Detection
on InOutDoor
OpenGAN: Open-Set Recognition via Open Data Generation
However, the former generalizes poorly to diverse open test data due to overfitting to the training outliers, which are unlikely to exhaustively span the open-world.
Streaming Self-Training via Domain-Agnostic Unlabeled Images
We present streaming self-training (SST) that aims to democratize the process of learning visual recognition models such that a non-expert user can define a new task depending on their needs via a few labeled examples and minimal domain knowledge.
Video-Specific Autoencoders for Exploring, Editing and Transmitting Videos
Associating latent codes of a video and manifold projection enables users to make desired edits.
Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details
On one hand, this is desirable as it treats all classes equally.
Learning to Segment Rigid Motions from Two Frames
Geometric motion segmentation algorithms, however, generalize to novel scenes, but have yet to achieve comparable performance to appearance-based ones, due to noisy motion estimations and degenerate motion configurations.
An Empirical Exploration of Open-Set Recognition via Lightweight Statistical Pipelines
Machine-learned safety-critical systems need to be self-aware and reliably know their unknowns in the open-world.
Learning Rare Category Classifiers on a Tight Labeling Budget
In this paper, we consider the scenario where we start with as-little-as five labeled positives of a rare category and a large amount of unlabeled data of which 99. 9% of it is negatives.
Detecting Invisible People
We demonstrate that current detection and tracking systems perform dramatically worse on this task.
Background Splitting: Finding Rare Classes in a Sea of Background
We focus on the real-world problem of training accurate deep models for image classification of a small number of rare categories.
What-If Motion Prediction for Autonomous Driving
Forecasting the long-term future motion of road actors is a core challenge to the deployment of safe autonomous vehicles (AVs).
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild
We present a method that infers spatial arrangements and shapes of humans and objects in a globally consistent 3D scene, all from a single image in-the-wild captured in an uncontrolled environment.
Ranked #3 on
3D Object Reconstruction
on BEHAVE
4D Visualization of Dynamic Events from Unconstrained Multi-View Videos
We present a data-driven approach for 4D space-time visualization of dynamic events from videos captured by hand-held multiple cameras.
Towards Streaming Perception
While past work has studied the algorithmic trade-off between latency and accuracy, there has not been a clear metric to compare different methods along the Pareto optimal latency-accuracy curve.
Ranked #2 on
Real-Time Object Detection
on Argoverse-HD (Detection-Only, Val)
(using extra training data)
TAO: A Large-Scale Benchmark for Tracking Any Object
To this end, we ask annotators to label objects that move at any point in the video, and give names to them post factum.
CATER: A diagnostic dataset for Compositional Actions & TEmporal Reasoning
In this work, we build a video dataset with fully observable and controllable object and scene bias, and which truly requires spatiotemporal understanding in order to be solved.
Learning Generative Models of Tissue Organization with Supervised GANs
A key step in understanding the spatial organization of cells and tissues is the ability to construct generative models that accurately reflect that organization.
Unsupervised Audiovisual Synthesis via Exemplar Autoencoders
We present an unsupervised approach that converts the input speech of any individual into audiovisual streams of potentially-infinitely many output speakers.
Learning to Move with Affordance Maps
In this paper, we combine the best of both worlds with a modular approach that learns a spatial representation of a scene that is trained to be effective when coupled with traditional geometric planners.
Hierarchical Deep Stereo Matching on High-resolution Images
We explore the problem of real-time stereo matching on high-res imagery.
Inferring Distributions Over Depth from a Single Image
Such approaches cannot diagnose when failures might occur.
Learning to Optimally Segment Point Clouds
We prove that if we score a segmentation by the worst objectness among its individual segments, there is an efficient algorithm that finds the optimal worst-case segmentation among an exponentially large number of candidate segmentations.
What You See is What You Get: Exploiting Visibility for 3D Object Detection
On the NuScenes 3D detection benchmark, we show that, by adding an additional stream for visibility input, we can significantly improve the overall detection accuracy of a state-of-the-art 3D detector.
Volumetric Correspondence Networks for Optical Flow
As a result, SOTA networks also employ various heuristics designed to limit volumetric processing, leading to limited accuracy and overfitting.
Ranked #14 on
Optical Flow Estimation
on KITTI 2015 (train)
Are we asking the right questions in MovieQA?
Joint vision and language tasks like visual question answering are fascinating because they explore high-level understanding, but at the same time, can be more prone to language biases.
Argoverse: 3D Tracking and Forecasting with Rich Maps
In our baseline experiments, we illustrate how detailed map information such as lane direction, driveable area, and ground height improves the accuracy of 3D object tracking and motion forecasting.
Learning to Track Any Object
Moreover, at test time the same network can be applied to detection and tracking, resulting in a unified approach for the two tasks.
MetaPix: Few-Shot Video Retargeting
We address the task of unsupervised retargeting of human actions from one video to another.
CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning
In this work, we build a video dataset with fully observable and controllable object and scene bias, and which truly requires spatiotemporal understanding in order to be solved.
Weakly-supervised Action Localization with Background Modeling
Our approach makes use of two innovations to attention-modeling in weakly-supervised learning.
Action Localization
Weakly Supervised Action Localization
+1
Growing a Brain: Fine-Tuning by Increasing Model Capacity
One of their remarkable properties is the ability to transfer knowledge from a large source dataset to a (typically smaller) target dataset.
Shapes and Context: In-the-Wild Image Synthesis & Manipulation
We introduce a data-driven approach for interactively synthesizing in-the-wild images from semantic label maps.
Do Image Classifiers Generalize Across Time?
Additionally, we evaluate three detection models and show that natural perturbations induce both classification as well as localization errors, leading to a median drop in detection mAP of 14 points.
A Systematic Framework for Natural Perturbations from Videos
We introduce a systematic framework for quantifying the robustness of classifiers to naturally occurring perturbations of images found in videos.
Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints
We also revisit existing approaches for fast convergence and show that budget-aware learning schedules readily outperform such approaches under (the practical but under-explored) budgeted training setting.
Towards Segmenting Anything That Moves
To address this concern, we propose two new benchmarks for generic, moving object detection, and show that our model matches top-down methods on common categories, while significantly out-performing both top-down and bottom-up methods on never-before-seen categories.
DistInit: Learning Video Representations Without a Single Labeled Video
In this work, we propose an alternative approach to learning video representations that require no semantically labeled videos and instead leverages the years of effort in collecting and labeling large and clean still-image datasets.
Ranked #71 on
Action Recognition
on HMDB-51
(using extra training data)
Photo-Sketching: Inferring Contour Drawings from Images
Edges, boundaries and contours are important subjects of study in both computer graphics and computer vision.
Online Model Distillation for Efficient Video Inference
Rather than learn a specialized student model on offline data from the video stream, we train the student in an online fashion on the live video, intermittently running the teacher to provide a target for learning.
Few-Shot Human Motion Prediction via Meta-Learning
This paper addresses the problem of few-shot human motion prediction, in the spirit of the recent progress on few-shot learning and meta-learning.
Recycle-GAN: Unsupervised Video Retargeting
We introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i. e., if contents of John Oliver's speech were to be transferred to Stephen Colbert, then the generated content/speech should be in Stephen Colbert's style.
Active Testing: An Efficient and Robust Framework for Estimating Accuracy
Much recent work on visual recognition aims to scale up learning to massive, noisily-annotated datasets.
Cross-Domain Image Matching with Deep Feature Maps
We investigate the problem of automatically determining what type of shoe left an impression found at a crime scene.
Active Learning with Partial Feedback
While many active learning papers assume that the learner can simply ask for a label and receive it, real annotation often presents a mismatch between the form of a label (say, one among many classes), and the form of an annotation (typically yes/no binary feedback).
Brute-Force Facial Landmark Analysis With A 140,000-Way Classifier
While most prior work treats this as a regression problem, we instead formulate it as a discrete $K$-way classification task, where a classifier is trained to return one of $K$ discrete alignments.
Learning to Model the Tail
We cast this problem as transfer learning, where knowledge from the data-rich classes in the head of the distribution is transferred to the data-poor classes in the tail.
Patch Correspondences for Interpreting Pixel-level CNNs
We present compositional nearest neighbors (CompNN), a simple approach to visually interpreting distributed representations learned by a convolutional neural network (CNN) for pixel-level tasks (e. g., image synthesis and segmentation).
Attentional Pooling for Action Recognition
We introduce a simple yet surprisingly powerful model to incorporate attention in action recognition and human object interaction tasks.
Ranked #7 on
Human-Object Interaction Detection
on HICO
PixelNN: Example-based Image Synthesis
We present a simple nearest-neighbor (NN) approach that synthesizes high-frequency photorealistic images from an "incomplete" signal such as a low-resolution image, a surface normal map, or edges.
Learning Policies for Adaptive Tracking with Deep Feature Cascades
Our fundamental insight is to take an adaptive approach, where easy frames are processed with cheap features (such as pixel values), while challenging frames are processed with invariant but expensive deep features.
Unconstrained Face Detection and Open-Set Face Recognition Challenge
Face detection and recognition benchmarks have shifted toward more difficult environments.
Comparing Apples and Oranges: Off-Road Pedestrian Detection on the NREC Agricultural Person-Detection Dataset
Person detection from vehicles has made rapid progress recently with the advent of multiple highquality datasets of urban and highway driving, yet no large-scale benchmark is available for the same problem in off-road or agricultural environments.
Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learning
We formulate tracking as an online decision-making process, where a tracking agent must follow an object despite ambiguous image frames and a limited computational budget.
Predictive-Corrective Networks for Action Detection
While deep feature learning has revolutionized techniques for static-image understanding, the same does not quite hold for video processing.
ActionVLAD: Learning spatio-temporal aggregation for action classification
In this work, we introduce a new video representation for action classification that aggregates local convolutional features across the entire spatio-temporal extent of the video.
Ranked #8 on
Long-video Activity Recognition
on Breakfast
Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters
Such "in-the-tail" data is notoriously hard to observe, making both training and testing difficult.
Need for Speed: A Benchmark for Higher Frame Rate Object Tracking
In this paper, we propose the first higher frame rate video dataset (called Need for Speed - NfS) and benchmark for visual object tracking.
PixelNet: Representation of the pixels, by the pixels, and for the pixels
We explore design principles for general pixel-level prediction problems, from low-level edge detection to mid-level surface normal estimation to high-level semantic segmentation.
3D Human Pose Estimation = 2D Pose Estimation + Matching
While many approaches try to directly predict 3D pose from image measurements, we explore a simple architecture that reasons through intermediate 2D pose predictions.
Ranked #310 on
3D Human Pose Estimation
on Human3.6M
Tinkering Under the Hood: Interactive Zero-Shot Learning with Net Surgery
We consider the task of visual net surgery, in which a CNN can be reconfigured without extra data to recognize novel concepts that may be omitted from the training set.
Finding Tiny Faces
We explore three aspects of the problem in the context of finding small faces: the role of scale invariance, image resolution, and contextual reasoning.
Ranked #24 on
Face Detection
on WIDER Face (Medium)
PixelNet: Towards a General Pixel-level Architecture
We explore architectures for general pixel-level prediction problems, from low-level edge detection to mid-level surface normal estimation to high-level semantic segmentation.
Depth-Based Hand Pose Estimation: Data, Methods, and Challenges
To spur further progress we introduce a challenging new dataset with diverse, cluttered scenes.
Understanding Everyday Hands in Action From RGB-D Images
We analyze functional manipulations of handheld objects, formalizing the problem as one of fine-grained grasp classification.
Look and Think Twice: Capturing Top-Down Visual Attention With Feedback Convolutional Neural Networks
While feedforward deep convolutional neural networks (CNNs) have been a great success in computer vision, it is important to remember that the human visual contex contains generally more feedback connections than foward connections.
Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians
We show that RGs can be optimized with a quadratic program (QP), that can in turn be optimized with a recurrent neural network (with rectified linear units).
Ranked #40 on
Pose Estimation
on MPII Human Pose
First-Person Pose Recognition Using Egocentric Workspaces
In egocentric views, hands and arms are observable within a well defined volume in front of the camera.
Multi-scale recognition with DAG-CNNs
We explore multi-scale convolutional neural nets (CNNs) for image classification.
Depth-based hand pose estimation: methods, data, and challenges
To spur further progress we introduce a challenging new dataset with diverse, cluttered scenes.
Do We Need More Training Data?
Datasets for training object recognition systems are steadily increasing in size.
Egocentric Pose Recognition in Four Lines of Code
We tackle the problem of estimating the 3D pose of an individual's upper limbs (arms+hands) from a chest mounted depth-camera.
3D Hand Pose Detection in Egocentric RGB-D Images
We focus on the task of everyday hand pose estimation from egocentric viewpoints.
Parsing Occluded People
Occlusion poses a significant difficulty for object recognition due to the combinatorial diversity of possible occlusion patterns.
Parsing Videos of Actions with Segmental Grammars
Real-world videos of human activities exhibit temporal structure at various scales; long videos are typically composed out of multiple action instances, where each instance is itself composed of sub-actions with variable durations and orderings.
Capturing Long-tail Distributions of Object Subcategories
We argue that object subcategories follow a long-tail distribution: a few subcategories are common, while many are rare.
Analysis by Synthesis: 3D Object Recognition by Object Reconstruction
We introduce an efficient "brute-force" approach to inference that searches through a large number of candidate reconstructions, returning the optimal one.
Microsoft COCO: Common Objects in Context
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding.
Dual coordinate solvers for large-scale structural SVMs
This manuscript describes a method for training linear SVMs (including binary SVMs, SVM regression, and structural SVMs) from large, out-of-core training datasets.
Self-Paced Learning for Long-Term Tracking
We address the problem of long-term object tracking, where the object may become occluded or leave-the-view.
Exploring Weak Stabilization for Motion Feature Extraction
We describe novel but simple motion features for the problem of detecting objects in video sequences.
Histograms of Sparse Codes for Object Detection
Object detection has seen huge progress in recent years, much thanks to the heavily-engineered Histograms of Oriented Gradients (HOG) features.
Analyzing 3D Objects in Cluttered Images
We use a morphable model to capture 3D within-class variation, and use a weak-perspective camera model to capture viewpoint.
Video Annotation and Tracking with Active Learning
We introduce a novel active learning framework for video annotation.
Statistical Tests for Optimization Efficiency
Learning problems such as logistic regression are typically formulated as pure optimization problems defined on some loss function.
Bilinear classifiers for visual recognition
Bilinear classifiers are a discriminative variant of bilinear models, which capture the dependence of data on multiple factors.
