Search Results for author:
Found 30 papers, 20 papers with code
SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
We approach this problem through the real-data-free setting in which the model is trained only on 3D simulation data and applied out-of-the-box to a wide variety of real cameras.
Ranked #1 on
Trajectory Forecasting
on ActEV
VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting
Combining CNNs or ViTs, with RNNs for spatiotemporal forecasting, has yielded unparalleled results in predicting temporal and spatial dynamics.
Adversarially Masked Video Consistency for Unsupervised Domain Adaptation
The second is a Masked Consistency Learning module to learn class-discriminative representations.
Prioritized Semantic Learning for Zero-shot Instance Navigation
Furthermore, for the popular HM3D environment, we present an Instance Navigation (InstanceNav) task that requires going to a specific object instance with detailed descriptions, as opposed to the Object Navigation (ObjectNav) task where the goal is defined merely by the object category.
ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognition
With the proposed ActionHub dataset, we further propose a novel Cross-modality and Cross-action Modeling (CoCo) framework for ZSAR, which consists of a Dual Cross-modality Alignment module and a Cross-action Invariance Mining module.
GeoDeformer: Geometric Deformable Transformer for Action Recognition
Specifically, at the core of GeoDeformer is the Geometric Deformation Predictor, a module designed to identify and quantify potential spatial and temporal geometric deformations within the given video.
AdaFocus: Towards End-to-end Weakly Supervised Learning for Long-Video Action Understanding
Under the weak supervision setting, action labels are provided for the whole video without precise start and end times of the action clip.
Ranked #1 on
Long-video Activity Recognition
on Breakfast
PostRainBench: A comprehensive benchmark and a new model for precipitation forecasting
To address these limitations, we introduce the PostRainBench, a comprehensive multi-variable NWP post-processing benchmark consisting of three datasets for NWP post-processing-based precipitation forecasting.
PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting
Although the Transformer has been the dominant architecture for time series forecasting tasks in recent years, a fundamental challenge remains: the permutation-invariant self-attention mechanism within Transformers leads to a loss of temporal information.
Ranked #1 on
Time Series Forecasting
on ETTh2 (336) Multivariate
TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation
In this paper, we present TFNet, a range-image-based LiDAR semantic segmentation method that utilizes temporal information to address this issue.
Ranked #1 on
Semantic Segmentation
on SemanticPOSS
An Examination of the Compositionality of Large Generative Vision-Language Models
A challenging new task is subsequently added to evaluate the robustness of GVLMs against inherent inclination toward syntactical correctness.
Ranked #83 on
Visual Reasoning
on Winoground
Spatial-Temporal Alignment Network for Action Recognition
This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture.
STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition
The model uses a hierarchical transformer with intra-frame off-set attention and inter-frame self-attention.
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval
To address this problem, we propose a Text-Adaptive Multiple Visual Prototype Matching model, which automatically captures multiple prototypes to describe a video by adaptive aggregation of video token features.
Multi-dataset Training of Transformers for Robust Action Recognition
We study the task of robust feature representations, aiming to generalize well on multiple datasets for action recognition.
Stargazer: A transformer-based driver action detection system for intelligent transportation
Distracted driver actions can be dangerous and cause severe accidents.
Weakly Supervised 3D Semantic Segmentation Using Cross-Image Consensus and Inter-Voxel Affinity Relations
We propose a novel weakly supervised approach for 3D semantic segmentation on volumetric images.
Spatial-Temporal Alignment Network for Action Recognition and Detection
The experimental results show that the STAN model can consistently improve the state of the arts in both action detection and action recognition tasks.
From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
With the advancement in computer vision deep learning, systems now are able to analyze an unprecedented amount of rich visual information from videos to enable applications such as autonomous driving, socially-aware robot assistant and public safety monitoring.
MSNet: A Multilevel Instance Segmentation Network for Natural Disaster Damage Assessment in Aerial Videos
This provides the first benchmark for quantitative evaluation of models to assess building damage using aerial videos.
SimAug: Learning Robust Representations from 3D Simulation for Pedestrian Trajectory Prediction in Unseen Cameras
We refer to our method as SimAug.
Ranked #2 on
Trajectory Prediction
on ActEV
Argus: Efficient Activity Detection System for Extended Video Analysis
We propose an Efficient Activity Detection System, Argus, for Extended Video Analysis in the surveillance scenario.
The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
The first contribution is a new dataset, created in a realistic 3D simulator, which is based on real world trajectory data, and then extrapolated by human annotators to achieve different latent goals.
Ranked #1 on
Multi-future Trajectory Prediction
on ForkingPaths
Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond
Among other uses, VERA enables the localization of a shooter from just a few videos that include the sound of gunshots.
Peeking into the Future: Predicting Future Person Activities and Locations in Videos
To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen.
Ranked #1 on
Activity Prediction
on ActEV
Focal Visual-Text Attention for Memex Question Answering
In addition to a text answer, a few grounding photos are also given to justify the answer.
Ranked #1 on
Memex Question Answering
on MemexQA
Focal Visual-Text Attention for Visual Question Answering
Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering.
Ranked #1 on
Memex Question Answering
on MemexQA
MemexQA: Visual Memex Question Answering
This paper proposes a new task, MemexQA: given a collection of photos or videos from a user, the goal is to automatically answer questions that help users recover their memory about events captured in the collection.
Exploiting Multi-modal Curriculum in Noisy Web Data for Large-scale Concept Learning
Learning video concept detectors automatically from the big but noisy web data with no additional manual annotations is a novel but challenging area in the multimedia and the machine learning community.
