Search Results for author:
Found 62 papers, 33 papers with code
Offline Imitation from Observation via Primal Wasserstein State Occupancy Matching
To address this problem, we propose Primal Wasserstein DICE (PW-DICE), which minimizes the primal Wasserstein distance between the expert and learner state occupancies with a pessimistic regularizer and leverages a contrastively learned distance as the underlying metric for the Wasserstein distance.
Frozen Transformers in Language Models Are Effective Visual Encoder Layers
This paper reveals that large language models (LLMs), despite being trained solely on textual data, are surprisingly strong encoders for purely visual tasks in the absence of language.
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
While large language models (LLMs) have demonstrated impressive performance on a range of decision-making tasks, they rely on simple acting processes and fall short of broad deployment as autonomous agents.
Ranked #1 on
Code Generation
on MBPP
Streaming Motion Forecasting for Autonomous Driving
Our benchmark inherently captures the disappearance and re-appearance of agents, presenting the emergent challenge of forecasting for occluded agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks.
Multi-task View Synthesis with Neural Radiance Fields
To tackle the MTVS problem, we propose MuvieNeRF, a framework that incorporates both multi-task and cross-view knowledge to simultaneously synthesize multiple scene properties.
Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictors
Dense depth and surface normal predictors should possess the equivariant property to cropping-and-resizing -- cropping the input image should result in cropping the same output image.
Aligning Large Multimodal Models with Factually Augmented RLHF
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounded by the multimodal information in context.
InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion
This paper addresses a novel task of anticipating 3D human-object interactions (HOIs).
An Empirical Analysis of Range for 3D Object Detection
LiDAR-based 3D detection plays a vital role in autonomous navigation.
Revisiting Deformable Convolution for Depth Completion
Our study reveals that, different from prior work, deformable convolution needs to be applied on an estimated depth map with a relatively high density for better performance.
Semi-Supervised Object Detection in the Open World
Existing approaches for semi-supervised object detection assume a fixed set of classes present in training and unlabeled datasets, i. e., in-distribution (ID) data.
Is Pre-training Truly Better Than Meta-Learning?
Using this analysis, we demonstrate the following: 1. when the formal diversity of a data set is low, PT beats MAML on average and 2. when the formal diversity is high, MAML beats PT on average.
Stochastic Multi-Person 3D Motion Forecasting
This paper aims to deal with the ignored real-world complexities in prior work on human motion forecasting, emphasizing the social properties of multi-person motion, the diversity of motion and social interactions, and the complexity of articulated motion.
Robust Model-Based Optimization for Challenging Fitness Landscapes
Protein design, a grand challenge of the day, involves optimization on a fitness landscape, and leading methods adopt a model-based approach where a model is trained on a training set (protein sequences and fitness) and proposes candidates to explore next.
MV-Map: Offboard HD-Map Generation with Multi-view Consistency
To further enhance multi-view consistency, we augment the uncertainty network with the global 3D structure optimized by a voxelized neural radiance field (Voxel-NeRF).
DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perception
In this work, we propose DualCross, a cross-modality cross-domain adaptation framework to facilitate the learning of a more robust monocular bird's-eye-view (BEV) perception model, which transfers the point cloud knowledge from a LiDAR sensor in one domain during the training phase to the camera-only testing scenario in a different domain.
Contrastive Mean Teacher for Domain Adaptive Object Detectors
Object detectors often suffer from the domain gap between training (source domain) and real-world applications (target domain).
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds
Our key insight is to exploit the explicit point cloud representation as the underlying structure to construct NeRFs, inspired by the intuitive interpretation of NeRF rendering as a process that projects or "plots" the associated 3D point cloud to a 2D image plane.
Object Discovery from Motion-Guided Tokens
Object discovery -- separating objects from the background without manual labels -- is a fundamental open challenge in computer vision.
Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors
Predicting diverse human motions given a sequence of historical poses has received increasing attention.
Ranked #1 on
Human Pose Forecasting
on Human3.6M
(MMADE metric)
Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking
It emphasizes spatio-temporal continuity and integrates both past and future reasoning for tracked objects.
Video State-Changing Object Segmentation
Learning about object state changes in Video Object Segmentation (VOS) is crucial for understanding and interacting with the visual world.
Contrastive Learning Relies More on Spatial Inductive Bias Than Supervised Learning: An Empirical Study
Though self-supervised contrastive learning (CL) has shown its potential to achieve state-of-the-art accuracy without any supervision, its behavior still remains under investigated by academia.
BEV-Guided Multi-Modality Fusion for Driving Perception
We design a BEV-guided multi-sensor attention block to take queries from BEV embeddings and learn the BEV representation from sensor-specific features.
Do Pre-trained Models Benefit Equally in Continual Learning?
Existing work on continual learning (CL) is primarily devoted to developing algorithms for models trained from scratch.
CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations
To better benefit from available demonstrations, we develop a method to Combine Explicit and Implicit Priors (CEIP).
Continual Learning with Evolving Class Ontologies
LECO requires learning classifiers in distinct time periods (TPs); each TP introduces a new ontology of "fine" labels that refines old ontologies of "coarse" labels (e. g., dog breeds that refine the previous ${\tt dog}$).
PointTree: Transformation-Robust Point Cloud Encoder with Relaxed K-D Trees
Being able to learn an effective semantic representation directly on raw point clouds has become a central topic in 3D understanding.
The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence
This novel insight contextualizes claims that transfer learning solutions are better than meta-learned solutions in the regime of low diversity under a fair comparison.
Is Self-Supervised Learning More Robust Than Supervised Learning?
Our insight has implications in improving the downstream robustness of supervised learning.
Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields
Comprehensive 3D scene understanding, both geometrically and semantically, is important for real-world applications such as robot perception.
Long-Tailed Recognition via Weight Balancing
In contrast, weight decay penalizes larger weights more heavily and so learns small balanced weights; the MaxNorm constraint encourages growing small weights within a norm ball but caps all the weights by the radius.
Ranked #8 on
Long-tail Learning
on CIFAR-100-LT (ρ=10)
Discovering Objects that Can Move
Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects.
The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence
We hypothesize that the diversity coefficient of the few-shot learning benchmark is predictive of whether meta-learning solutions will succeed or not.
Does MAML Only Work via Feature Re-use? A Data Centric Perspective
Recent work has suggested that a good embedding is all we need to solve many few-shot learning benchmarks.
Embracing Single Stride 3D Object Detector with Sparse Transformer
In LiDAR-based 3D object detection for autonomous driving, the ratio of the object size to input scene size is significantly smaller compared to 2D detection cases.
Ranked #3 on
3D Object Detection
on waymo cyclist
On the Importance of Firth Bias Reduction in Few-Shot Classification
In this work, we demonstrate the effectiveness of Firth bias reduction in few-shot classification.
Generative Modeling for Multitask Visual Learning
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
On the Importance of Distractors for Few-Shot Classification
An effective approach to few-shot classification involves a prior model trained on a large-sample base domain, which is then finetuned over the novel few-shot task to yield generalizable representations.
Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation
We leverage the pixel-level L2 loss and the pixel contrastive loss for the two purposes respectively.
Generative Modeling for Multi-task Visual Learning
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
Hallucination Improves Few-Shot Object Detection
One critical factor in improving few-shot detection is to address the lack of variation in training data.
Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level.
DAP: Detection-Aware Pre-training with Weak Supervision
This paper presents a detection-aware pre-training (DAP) approach, which leverages only weakly-labeled classification-style datasets (e. g., ImageNet) for pre-training, but is specifically tailored to benefit object detection tasks.
Learning To Hallucinate Examples From Extrinsic and Intrinsic Supervision
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks.
Cooperating RPN's Improve Few-Shot Object Detection
Learning to detect an object in an image from very few training examples - few-shot object detection - is challenging, because the classifier that sees proposal boxes has very little training data.
Few-Shot Learning with Intra-Class Knowledge Transfer
Recent works have proposed to solve this task by augmenting the training data of the few-shot classes using generative models with the few-shot training samples as the seeds.
AlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes.
Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis
We propose a novel task of joint few-shot recognition and novel-view synthesis: given only one or few images of a novel object from arbitrary views with only category annotation, we aim to simultaneously learn an object classifier and generate images of that type of object from new viewpoints.
Towards Streaming Perception
While past work has studied the algorithmic trade-off between latency and accuracy, there has not been a clear metric to compare different methods along the Pareto optimal latency-accuracy curve.
Ranked #2 on
Real-Time Object Detection
on Argoverse-HD (Detection-Only, Val)
(using extra training data)
Unlocking the Full Potential of Small Data with Diverse Supervision
Indeed, even the majority of few-shot learning methods rely on a large set of "base classes" for pretraining.
Meta-Learning to Detect Rare Objects
Few-shot learning, i. e., learning novel concepts from few examples, is fundamental to practical visual recognition systems.
Ranked #20 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Meta-Learning by Hallucinating Useful Examples
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks, which aim to learn novel concepts from very few examples.
Growing a Brain: Fine-Tuning by Increasing Model Capacity
One of their remarkable properties is the ability to transfer knowledge from a large source dataset to a (typically smaller) target dataset.
Image Deformation Meta-Networks for One-Shot Learning
Humans can robustly learn novel visual concepts even when images undergo various deformations and lose certain information.
Learning Compositional Representations for Few-Shot Recognition
One of the key limitations of modern deep learning approaches lies in the amount of data required to train them.
Adversarial Geometry-Aware Human Motion Prediction
We explore an approach to forecasting human motion in a few milliseconds given an input 3D skeleton sequence based on a recurrent encoder-decoder framework.
Few-Shot Human Motion Prediction via Meta-Learning
This paper addresses the problem of few-shot human motion prediction, in the spirit of the recent progress on few-shot learning and meta-learning.
Low-Shot Learning from Imaginary Data
Humans can quickly learn new visual concepts, perhaps because they can easily visualize or imagine what novel objects look like from different views.
Learning to Model the Tail
We cast this problem as transfer learning, where knowledge from the data-rich classes in the head of the distribution is transferred to the data-poor classes in the tail.
Learning from Small Sample Sets by Combining Unsupervised Meta-Training with CNNs
Inspired by the transferability properties of CNNs, we introduce an additional unsupervised meta-training stage that exposes multiple top layer units to a large amount of unlabeled real-world images.
Model Recommendation: Generating Object Detectors From Few Samples
In this paper, we explore an approach to generating detectors that is radically different from the conventional way of learning a detector from a large corpus of annotated positive and negative data samples.
