Search Results for author:
Found 49 papers, 27 papers with code
G4: Grounding-guided Goal-oriented Dialogues Generation with Multiple Documents
Goal-oriented dialogues generation grounded in multiple documents(MultiDoc2Dial) is a challenging and realistic task.
Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generation
To fully Unleash the potential of evidence, we propose a framework to effectively incorporate Evidence in knowledge-Intensive Dialogue Generation (u-EIDG).
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
When pre-training on the large-scale Kinetics-710, we achieve 89. 7% on Kinetics-400 with a frozen ViT-L model, which verifies the scalability of DiST.
Towards Real-World Visual Tracking with Temporal Contexts
To handle those problems, we propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently.
RLIPv2: Fast Scaling of Relational Language-Image Pre-training
In this paper, we propose RLIPv2, a fast converging model that enables the scaling of relational pre-training to large-scale pseudo-labelled scene graph data.
Ranked #1 on
Zero-Shot Human-Object Interaction Detection
on HICO-DET
(using extra training data)
ModelScope Text-to-Video Technical Report
This paper introduces ModelScopeT2V, a text-to-video synthesis model that evolves from a text-to-image synthesis model (i. e., Stable Diffusion).
Ranked #2 on
Text-to-Video Generation
on MSR-VTT
Temporally-Adaptive Models for Efficient Video Understanding
Spatial convolutions are extensively used in numerous deep video models.
Ranked #3 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation
In this way, RGB images are not required during inference anymore since the 2D knowledge branch provides 2D information according to the 3D LIDAR input.
VideoComposer: Compositional Video Synthesis with Motion Controllability
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis.
Crowd Counting with Sparse Annotation
This paper presents a new annotation method called Sparse Annotation (SA) for crowd counting, which reduces human labeling efforts by sparsely labeling individuals in an image.
MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
To address these issues, we develop a Motion-augmented Long-short Contrastive Learning (MoLo) method that contains two crucial components, including a long-short contrastive objective and a motion autodecoder.
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
In this paper, we begin with analyzing the feature representation behavior in the open-set action recognition (OSAR) problem based on the information bottleneck (IB) theory, and propose to enlarge the instance-specific (IS) and class-specific (CS) information contained in the feature for better performance.
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task.
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment.
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment
This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- heterogeneous ML clusters.
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples, which tends to be more practical in real-world applications.
HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition
To be specific, HyRSM++ consists of two key components, a hybrid relation module and a temporal set matching metric.
LipFormer: High-Fidelity and Generalizable Talking Face Generation With a Pre-Learned Facial Codebook
Generating a talking face video from the input audio sequence is a practical yet challenging task.
Space-time Prompting for Video Class-incremental Learning
In this paper, we will fill this gap by learning multiple prompts based on a powerful image-language pre-trained model, i. e., CLIP, making it fit for video class-incremental learning (VCIL).
Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
The former is to reduce the memory cost by preserving only one condensed frame instead of the whole video, while the latter aims to compensate the lost spatio-temporal details in the Frame Condensing stage.
Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words
Prompt-based fine-tuning for pre-trained models has proven effective for many natural language processing tasks under few-shot settings in general domain.
MAR: Masked Autoencoders for Efficient Action Recognition
Inspired by this, we propose propose Masked Action Recognition (MAR), which reduces the redundant computation by discarding a proportion of patches and operating only on a part of the videos.
Ranked #9 on
Action Recognition
on Something-Something V2
Open-world Semantic Segmentation for LIDAR Point Clouds
Current methods for LIDAR semantic segmentation are not robust enough for real-world applications, e. g., autonomous driving, since it is closed-set and static.
Context-aware Proposal Network for Temporal Action Detection
This technical report presents our first place winning solution for temporal action detection task in CVPR-2022 AcitivityNet Challenge.
Hybrid Relation Guided Set Matching for Few-shot Action Recognition
To overcome the two limitations, we propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components: hybrid relation module and set matching metric.
Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency
In this work, we aim to learn representations by leveraging more abundant information in untrimmed videos.
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
Does QA-based intermediate training help fine-tuning language models for text classification?
Fine-tuning pre-trained language models for downstream tasks has become a norm for NLP.
Discovery-and-Selection: Towards Optimal Multiple Instance Learning for Weakly Supervised Object Detection
Weakly supervised object detection (WSOD) is a challenging task that requires simultaneously learn object classifiers and estimate object locations under the supervision of image category labels.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #57 on
Action Recognition
on Something-Something V2
(using extra training data)
Support-Set Based Cross-Supervision for Video Grounding
The contrastive objective aims to learn effective representations by contrastive learning, while the caption objective can train a powerful video encoder supervised by texts.
ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
The visualizations show that ParamCrop adaptively controls the center distance and the IoU between two augmented views, and the learned change in the disparity along the training process is beneficial to learning a strong representation.
Exploring Stronger Feature for Temporal Action Localization
Temporal action localization aims to localize starting and ending time with action category.
OadTR: Online Action Detection with Transformers
Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure.
Ranked #7 on
Online Action Detection
on THUMOS'14
Proposal Relation Network for Temporal Action Detection
We calculate the detection results by assigning the proposals with corresponding classification results.
Ranked #1 on
Temporal Action Localization
on ActivityNet-1.3
(using extra training data)
Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
Then our proposed Local-Global Background Modeling Network (LGBM-Net) is trained to localize instances by using only video-level labels based on Multi-Instance Learning (MIL).
Weakly-supervised Learning
Weakly-supervised Temporal Action Localization
+1
End-to-end Temporal Action Detection with Transformer
Temporal action detection (TAD) aims to determine the semantic label and the temporal interval of every action instance in an untrimmed video.
Ranked #5 on
Temporal Action Localization
on HACS
Relation Modeling in Spatio-Temporal Action Localization
This paper presents our solution to the AVA-Kinetics Crossover Challenge of ActivityNet workshop at CVPR 2021.
Ranked #2 on
Spatio-Temporal Action Localization
on AVA-Kinetics
(using extra training data)
A Stronger Baseline for Ego-Centric Action Detection
This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop.
Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
In this paper, we present empirical results for training a stronger video vision transformer on the EPIC-KITCHENS-100 Action Recognition dataset.
Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
In this paper, we focus on applying the power of self-supervised methods to improve semi-supervised action proposal generation.
Ranked #2 on
Semi-Supervised Action Detection
on THUMOS' 14
Self-Supervised Learning
Semi-Supervised Action Detection
+1
Self-supervised Motion Learning from Static Images
We furthermore introduce a static mask in pseudo motions to create local motion patterns, which forces the model to additionally locate notable motion areas for the correct classification. We demonstrate that MoSI can discover regions with large motion even without fine-tuning on the downstream datasets.
Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw
With the constrained jigsaw puzzles, instead of solving them directly, which could still be extremely hard, we carefully design four surrogate tasks that are more solvable but meanwhile still ensure that the learned representation is sensitive to spatiotemporal continuity at both the local and global levels.
Tuning the quantumness of simple Bose systems: A universal phase diagram
We present a comprehensive theoretical study of the phase diagram of a system of many Bose particles interacting with a two-body central potential of the so-called Lennard-Jones form.
Statistical Mechanics
Multi-Level Temporal Pyramid Network for Action Detection
By this means, the proposed MLTPN can learn rich and discriminative features for different action instances with different durations.
Less is More: Rejecting Unreliable Reviews for Product Question Answering
In the literature, PQA is formulated as a retrieval problem with the goal to search for the most relevant reviews to answer a given product question.
Temporal Fusion Network for Temporal Action Localization:Submission to ActivityNet Challenge 2020 (Task E)
This technical report analyzes a temporal action localization method we used in the HACS competition which is hosted in Activitynet Challenge 2020. The goal of our task is to locate the start time and end time of the action in the untrimmed video, and predict action category. Firstly, we utilize the video-level feature information to train multiple video-level action classification models.
CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)
In this report, we present our solution for the task of temporal action localization (detection) (task 1) in ActivityNet Challenge 2020.
TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection
In this paper, we define these ambiguous samples as "transitional states", and propose a Transition-Aware Context Network (TACNet) to distinguish transitional states.
Ranked #7 on
Action Detection
on J-HMDB
