Search Results for author:
Found 104 papers, 65 papers with code
From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Models
To probe the extent of this internal knowledge, we introduce a few-shot fine-tuning framework that repurposes VDMs for new tasks using only a handful of examples.
VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection
Building on this, we further propose VoxDet, an instance-centric framework that reformulates the voxel-level occupancy prediction as dense object detection by decoupling it into two sub-tasks: offset regression and semantic prediction.
X-GRM: Large Gaussian Reconstruction Model for Sparse-view X-rays to Computed Tomography
Existing CT reconstruction works are limited to small-capacity model architecture and inflexible volume representation.
CAMELTrack: Context-Aware Multi-cue ExpLoitation for Online Multi-Object Tracking
However, the extensive usage of human-crafted rules for temporal associations makes these methods inherently limited in their ability to capture the complex interplay between various tracking cues.
Ranked #1 on
Online Multi-Object Tracking
on SportsMOT
Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors
Efficiently leveraging of the capabilities of contemporary large language models (LLMs) is increasingly challenging, particularly when direct fine-tuning is expensive and often impractical.
Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model
Omnidirectional depth perception is essential for mobile robotics applications that require scene understanding across a full 360{\deg} field of view.
Ranked #1 on
Omnnidirectional Stereo Depth Estimation
on Helvipad
Monocular Depth Estimation
Omnnidirectional Stereo Depth Estimation
+2
FG$^2$: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching
Our method then learns to select features along the height dimension to pool the 3D points to a Bird's-Eye-View (BEV) plane.
Unified Human Localization and Trajectory Prediction with Monocular Vision
On the curated dataset, MT achieves around 12% improvement over baseline models on BEV localization and trajectory prediction.
COARSE: Collaborative Pseudo-Labeling with Coarse Real Labels for Off-Road Semantic Segmentation
Autonomous off-road navigation faces challenges due to diverse, unstructured environments, requiring robust perception with both geometric and semantic understanding.
Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression
We address (2) through a simple neighborhood based heuristic algorithm which results in surprisingly effective pseudo labels for the covariance.
DRIVINGVQA: Analyzing Visual Chain-of-Thought Reasoning of Vision Language Models in Real-World Scenarios with Driving Theory Tests
Large vision-language models (LVLMs) augment language models with visual understanding, enabling multimodal reasoning.
Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting
Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.
Multi-Source Urban Traffic Flow Forecasting with Drone and Loop Detector Data
Traffic forecasting is a fundamental task in transportation research, however the scope of current research has mainly focused on a single data modality of loop detectors.
FG^2: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching
Our method then learns to select features along the height dimension to pool the 3D points to a Bird's-Eye-View (BEV) plane.
MotionMap: Representing Multimodality in Human Pose Forecasting
This raises key questions: (1) Can we capture multimodality by efficiently sampling a smaller number of predictions?
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control
We present GEM, a Generalizable Ego-vision Multimodal world model that predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories.
TAROT: Targeted Data Selection via Optimal Transport
Building on this, TAROT uses whitened feature distance to quantify and minimize the optimal transport distance between the selected data and target domains.
A Multi-Loss Strategy for Vehicle Trajectory Prediction: Combining Off-Road, Diversity, and Directional Consistency Losses
Trajectory prediction is essential for the safety and efficiency of planning in autonomous vehicles.
Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimation
To address this, we introduce necessary adaptations to stereo models, leading to improved performance.
Ranked #2 on
Omnnidirectional Stereo Depth Estimation
on Helvipad
Multi-Transmotion: Pre-trained Model for Human Motion Prediction
However, the complexity of human motion have prevented the development of a standardized dataset for human motion prediction, thereby hindering the establishment of pre-trained models.
Strada-LLM: Graph LLM for traffic prediction
Furthermore, we adopt a lightweight approach for efficient domain adaptation when facing new data distributions in few-shot fashion.
HEADS-UP: Head-Mounted Egocentric Dataset for Trajectory Prediction in Blind Assistance Systems
In this paper, we introduce HEADS-UP, the first egocentric dataset collected from head-mounted cameras, designed specifically for trajectory prediction in blind assistance systems.
SoccerNet 2024 Challenges Results
The SoccerNet 2024 challenges represent the fourth annual video understanding challenges organized by the SoccerNet team.
CODE: Confident Ordinary Differential Editing
However, conditioning on noisy or Out-of-Distribution (OoD) images poses significant challenges, particularly in balancing fidelity to the input and realism of the output.
MPL: Lifting 3D Human Pose from Multi-view 2D Poses
Estimating 3D human poses from 2D images is challenging due to occlusions and projective acquisition.
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses.
Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models
This tailored strategy, supplemented by our method's capability to efficiently adapt to different datasets, enhances model efficiency and ensures robust performance across datasets without the need for extensive retraining.
Keypoint Promptable Re-Identification
Inspired by recent work on prompting in vision, we introduce Keypoint Promptable ReID (KPR), a novel formulation of the ReID problem that explicitly complements the input bounding box with a set of semantic keypoints indicating the intended target.
Ranked #1 on
Person Re-Identification
on Occluded-DukeMTMC
SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap
This tracking and identification process is crucial for reconstructing the game state, defined by the athletes' positions and identities on a 2D top-view of the pitch, (i. e. a minimap).
Ranked #1 on
Game State Reconstruction
on SoccerNet-GSR
UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction
Vehicle trajectory prediction has increasingly relied on data-driven solutions, but their ability to scale to different data domains and the impact of larger dataset sizes on their generalization remain under-explored.
Ranked #1 on
Trajectory Prediction
on nuScenes
(using extra training data)
Certified Human Trajectory Prediction
Trajectory prediction plays an essential role in autonomous vehicles.
Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts
Steering the behavior of a strong model pre-trained on internet-scale data can be difficult due to the scarcity of competent supervisors.
Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness
This study explores the complexities of integrating Artificial Intelligence (AI) into Autonomous Vehicles (AVs), examining the challenges introduced by AI components and the impact on testing procedures, focusing on some of the essential requirements for trustworthy AI.
MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning
Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes.
Ranked #4 on
Skeleton Based Action Recognition
on NTU RGB+D
Social-Transmotion: Promptable Human Trajectory Prediction
We translate the idea of a prompt from Natural Language Processing (NLP) to the task of human trajectory prediction, where a prompt can be a sequence of x-y coordinates on the ground, bounding boxes in the image plane, or body pose keypoints in either 2D or 3D.
Images in Discrete Choice Modeling: Addressing Data Isomorphism in Multi-Modality Inputs
We propose and benchmark two methodologies to address this challenge: architectural design adjustments to segregate redundant information, and isomorphic information mitigation through source information masking and inpainting.
Manipulating Trajectory Prediction with Backdoors
In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction.
Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations
Modeling spatial-temporal interactions among neighboring agents is at the heart of multi-agent problems such as motion forecasting and crowd navigation.
Towards more Practical Threat Models in Artificial Intelligence Security
Recent works have identified a gap between research and practice in artificial intelligence security: threats studied in academia do not always reflect the practical use and security risks of AI.
JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds
To address this, we introduce a novel dataset for end-to-end trajectory forecasting, facilitating the evaluation of models in scenarios involving less-than-ideal preceding modules such as tracking.
TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression
However, recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation.
SoccerNet 2023 Challenges Results
More information on the tasks, challenges, and leaderboards are available on https://www. soccer-net. org.
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation.
Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?
In fact, conventional forecasting methods are usually not trained nor tested in real-world pipelines (e. g., with upstream detection, tracking, and mapping modules).
On Pitfalls of Test-Time Adaptation
Test-Time Adaptation (TTA) has recently emerged as a promising approach for tackling the robustness challenge under distribution shifts.
Toward Reliable Human Pose Forecasting with Uncertainty
Recently, there has been an arms race of pose forecasting methods aimed at solving the spatio-temporal task of predicting a sequence of future 3D poses of a person given a sequence of past observed ones.
Predicting the long-term collective behaviour of fish pairs with deep learning
Although analytical models dominate in studying collective behaviour, this study introduces a deep learning model to assess social interactions in the fish species Hemigrammus rhodostomus.
Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning
Recent years have seen a surge of interest in learning high-level causal representations from low-level image pairs under interventions.
1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results
The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection.
Body Part-Based Representation Learning for Occluded Person Re-Identification
Firstly, individual body part appearance is not as discriminative as global appearance (two distinct IDs might have the same local appearance), this means standard ReID training objectives using identity labels are not adapted to local feature learning.
Ranked #1 on
Person Re-Identification
on P-DukeMTMC-reID
Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting
To this end, we introduce two components that exploit our prior knowledge of motion style shifts: (i) a low-rank motion style adapter that projects and adjusts the style features at a low-dimensional bottleneck; and (ii) a modular adapter strategy that disentangles the features of scene context and motion history to facilitate a fine-grained choice of adaptation layers.
VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation
We begin with a simple premise: pose estimators often predict incoherent poses for out-of-distribution samples.
A generic diffusion-based approach for 3D human pose prediction in the wild
Predicting 3D human poses in real-world scenarios, also known as human pose forecasting, is inevitably subject to noisy inputs arising from inaccurate 3D pose estimations and occlusions.
Ranked #1 on
Human Pose Forecasting
on HumanEva-I
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting
Human trajectory forecasting in crowds presents the challenges of modelling social interactions and outputting collision-free multimodal distribution.
Pedestrian 3D Bounding Box Prediction
Safety is still the main issue of autonomous driving, and in order to be globally deployed, they need to predict pedestrians' motions sufficiently in advance.
Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion
Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic.
A Shared Representation for Photorealistic Driving Simulators
We build on successful cGAN models to propose a new semantically-aware discriminator that better guides the generator.
Do Pedestrians Pay Attention? Eye Contact Detection in the Wild
In urban or crowded environments, humans rely on eye contact for fast and efficient communication with nearby people.
Vehicle trajectory prediction works, but not everywhere
We further show that the generated scenes (i) are realistic since they do exist in the real world, and (ii) can be used to make existing models more robust, yielding 30-40 reductions in the off-road rate.
Causal Imitative Model for Autonomous Driving
CIM explicitly discovers the causal model and utilizes it to train the policy.
TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive?
In this work, we first provide an in-depth look at its limitations and show that TTT can possibly deteriorate, instead of improving, the test-time performance in the presence of severe distribution shifts.
Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
Learning behavioral patterns from observational data has been a de-facto approach to motion forecasting.
DriverGym: Democratising Reinforcement Learning for Autonomous Driving
Despite promising progress in reinforcement learning (RL), developing algorithms for autonomous driving (AD) remains challenging: one of the critical issues being the absence of an open-source platform capable of training and effectively validating the RL policies on real-world data.
SVG-Net: An SVG-based Trajectory Prediction Model
Finally, we illustrate how, by using SVG, one can benefit from datasets and advancements in other research fronts that also utilize the same input format.
Keypoint Communities
We present a fast bottom-up method that jointly detects over 100 keypoints on humans or objects, also referred to as human/object pose estimation.
Ranked #1 on
Car Pose Estimation
on ApolloCar3D
Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving
The ability to predict unseen vehicles is critical for safety in autonomous driving.
Are socially-aware trajectory prediction models really socially-aware?
An attack is a small yet carefully-crafted perturbations to fail predictors.
Interpretable Social Anchors for Human Trajectory Forecasting in Crowds
Human trajectory forecasting in crowds, at its core, is a sequence prediction problem with specific challenges of capturing inter-sequence dependencies (social interactions) and consequently predicting socially-compliant multimodal distributions.
Injecting Knowledge in Data-driven Vehicle Trajectory Predictors
On the other hand, recent works use data-driven approaches which can learn complex interactions from the data leading to superior performance.
OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association
We present a generic neural network architecture that uses Composite Fields to detect and construct a spatio-temporal pose which is a single, connected graph whose nodes are the semantic keypoints (e. g., a person's body joints) in multiple frames.
Ranked #2 on
Car Pose Estimation
on ApolloCar3D
Social NCE: Contrastive Learning of Socially-aware Motion Representations
Learning socially-aware motion representations is at the core of recent advances in multi-agent problems, such as human motion forecasting and robot navigation in crowds.
Ranked #1 on
Trajectory Prediction
on TrajNet++
Detecting 32 Pedestrian Attributes for Autonomous Vehicles
By increasing the number of attributes jointly learned, we highlight an issue related to the scales of gradients, which arises in MTL with numerous tasks.
Pedestrian Intention Prediction: A Multi-task Perspective
This work tries to solve this problem by jointly predicting the intention and visual states of pedestrians.
Perceiving Traffic from Aerial Images
Drones or UAVs, equipped with different sensors, have been deployed in many places especially for urban traffic monitoring or last-mile delivery.
Perceiving Humans: from Monocular 3D Localization to Social Distancing
Our neural network estimates human 3D body locations and their orientation with a measure of uncertainty.
MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization
Monocular and stereo visions are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots.
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
Scalable Vector Graphics (SVG) are ubiquitous in modern 2D interfaces due to their ability to scale to different resolutions.
Ranked #1 on
Vector Graphics Animation
on SVG-Icons8
Human Trajectory Forecasting in Crowds: A Deep Learning Perspective
In this work, we present an in-depth analysis of existing deep learning-based methods for modelling social interactions.
Ranked #3 on
Trajectory Prediction
on TrajNet++
Using Image Priors to Improve Scene Understanding
We propose a simple yet effective method for leveraging these image priors to improve semantic segmentation of images from sequential driving datasets.
MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation
We tackle the fundamentally ill-posed problem of 3D human localization from monocular RGB images.
Deep Visual Re-Identification with Confidence
We argue that such loss function is not suited for the visual re-identification task hence propose to model confidence in the representation learning framework.
Convolutional Relational Machine for Group Activity Recognition
We present an end-to-end deep Convolutional Neural Network called Convolutional Relational Machine (CRM) for recognizing group activities that utilizes the information in spatial relations between individual persons in image or video.
PifPaf: Composite Fields for Human Pose Estimation
We propose a new bottom-up method for multi-person 2D human pose estimation that is particularly well suited for urban mobility such as self-driving cars and delivery robots.
Ranked #10 on
Keypoint Detection
on COCO test-dev
Collaborative Sampling in Generative Adversarial Networks
The standard practice in Generative Adversarial Networks (GANs) discards the discriminator during sampling.
Enhancing Discrete Choice Models with Representation Learning
In discrete choice modeling (DCM), model misspecifications may lead to limited predictability and biased parameter estimates.
Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
We propose to (i) rethink pairwise interactions with a self-attention mechanism, and (ii) jointly model Human-Robot as well as Human-Human interactions in the deep reinforcement learning framework.
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments.
Ranked #4 on
Trajectory Prediction
on ETH
CAR-Net: Clairvoyant Attentive Recurrent Network
We exploit two sources of information: the past motion trajectory of the agent of interest and a wide top-view image of the navigation scene.
Towards Vision-Based Smart Hospitals: A System for Tracking and Monitoring Hand Hygiene Compliance
One in twenty-five patients admitted to a hospital will suffer from a hospital acquired infection.
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals
Physiological signals such as heart rate can provide valuable information about an individual's state and activity.
Characterizing and Improving Stability in Neural Style Transfer
Recent progress in style transfer on images has focused on improving the quality of stylized images and speed of methods.
Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies
To address this challenge, we present a structure of Recurrent Neural Networks (RNN) that jointly reasons on multiple cues over a temporal window.
Unsupervised Learning of Long-Term Motion Dynamics for Videos
We present an unsupervised representation learning approach that compactly encodes the motion dependencies in videos.
Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
We present a unified framework for understanding human social behaviors in raw image sequences.
Ranked #2 on
Action Recognition
on Volleyball
Recurrent Attention Models for Depth-Based Person Identification
We present an attention-based model that reasons on human body shape and motion dynamics to identify individuals in the absence of RGB information, hence in the dark.
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Different from the conventional LSTM, we share the information between multiple LSTMs through a new pooling layer.
Ranked #1 on
Trajectory Prediction
on Stanford Drone
(FDE(8/12) @K=5 metric)
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
We consider image transformation problems, where an input image is transformed into an output image.
Ranked #4 on
Nuclear Segmentation
on Cell17
Towards Viewpoint Invariant 3D Human Pose Estimation
We propose a viewpoint invariant model for 3D human pose estimation from a single depth image.
Ranked #4 on
Pose Estimation
on ITOP top-view
Knowledge Transfer for Scene-specific Motion Prediction
When given a single frame of the video, humans can not only interpret the content of the scene, but also they are able to forecast the near future.
Forecasting Social Navigation in Crowded Complex Scenes
We present an extensive evaluation where different methods for trajectory forecasting are evaluated and compared.
RGB-W: When Vision Meets Wireless
Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal (e. g., wifi or Bluetooth) emitted by individuals' cell phones, referred to as RGB-W.
Learning to Track: Online Multi-Object Tracking by Decision Making
Online Multi-Object Tracking (MOT) has wide applications in time-critical video analysis scenarios, such as robot navigation and autonomous driving.
Ranked #28 on
Multiple Object Tracking
on KITTI Test (Online Methods)
(MOTA metric)
Socially-aware Large-scale Crowd Forecasting
In crowded spaces such as city centers or train stations, human mobility looks complex, but is often influenced only by a few causes.
