Search Results for author:
Found 47 papers, 19 papers with code
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
We introduce Ego-R1, a novel framework for reasoning over ultra-long (i. e., in days and weeks) egocentric videos, which leverages a structured Chain-of-Tool-Thought (CoTT) process, orchestrated by an Ego-R1 Agent trained via reinforcement learning (RL).
Detecting Open World Objects via Partial Attribute Assignment
In this paper, we propose Partial Attribute Assignment (PASS), aiming to automatically select and optimize a small, relevant subset of attributes from a large attribute pool.
Robust Variational Contrastive Learning for Partially View-unaligned Clustering
To address these problems, we propose a robust framework, dubbed VariatIonal ConTrAstive Learning (VITAL), designed to learn both common and specific information simultaneously.
Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image
In the visual spatial understanding (VSU) area, spatial image-to-text (SI2T) and spatial text-to-image (ST2I) are two fundamental tasks that appear in dual form.
PointCloud-Text Matching: Benchmark Datasets and a Baseline
We observe that the data is challenging and with noisy correspondence due to the sparsity, noise, or disorder of point clouds and the ambiguity, vagueness, or incompleteness of texts, which make existing cross-modal matching methods ineffective for PTM.
Contributing Dimension Structure of Deep Feature for Coreset Selection
Existing methods typically measure both the representation and diversity of data based on similarity metrics, such as L2-norm.
Direct Distillation between Different Domains
Then, we build a fusion-activation mechanism to transfer the valuable domain-invariant knowledge to the student network, while simultaneously encouraging the adapter within the teacher network to learn the domain-specific knowledge of the target data.
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning
Recent progress in Large Multimodal Models (LMM) has opened up great possibilities for various applications in the field of human-machine interactions.
M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts
Furthermore, we establish a new benchmark for assessing the performance of large models in understanding multi-modal 3D prompts.
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
However, developing LMMs that can comprehend, reason, and plan in complex and diverse 3D environments remains a challenging topic, especially considering the demand for understanding permutation-invariant point cloud 3D representations of the 3D scene.
Towards Debiasing Frame Length Bias in Text-Video Retrieval via Causal Intervention
Many studies focus on improving pretraining or developing new backbones in text-video retrieval.
Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
Moreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture.
Ranked #3 on
3D dense captioning
on Nr3D
Self-Supervised Learning for WiFi CSI-Based Human Activity Recognition: A Systematic Study
Therefore, considerable efforts have been made to address the challenge of insufficient data in deep learning by leveraging SSL algorithms.
An Overview of Challenges in Egocentric Text-Video Retrieval
Text-video retrieval contains various challenges, including biases coming from diverse sources.
Multi-view Vision-Prompt Fusion Network: Can 2D Pre-trained Model Boost 3D Point Cloud Data-scarce Learning?
Then, a novel multi-view prompt fusion module is developed to effectively fuse information from different views to bridge the gap between 3D point cloud data and 2D pre-trained models.
A Closer Look at Few-Shot 3D Point Cloud Classification
In recent years, research on few-shot learning (FSL) has been fast-growing in the 2D image domain due to the less requirement for labeled training data and greater generalization for novel classes.
What Makes for Effective Few-shot Point Cloud Classification?
Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life.
End-to-End 3D Dense Captioning with Vote2Cap-DETR
Compared with prior arts, our framework has several appealing advantages: 1) Without resorting to numerous hand-crafted components, our method is based on a full transformer encoder-decoder architecture with a learnable vote query driven object decoder, and a caption decoder that produces the dense captions in a set-prediction manner.
Ranked #4 on
3D dense captioning
on Nr3D
RONO: Robust Discriminative Learning With Noisy Labels for 2D-3D Cross-Modal Retrieval
Recently, with the advent of Metaverse and AI Generated Content, cross-modal retrieval becomes popular with a burst of 2D and 3D data.
Zero-Shot Point Cloud Segmentation by Semantic-Visual Aware Synthesis
Given only the class-level semantic information for unseen objects, we strive to enhance the correspondence, alignment and consistency between the visual and semantic spaces, to synthesise diverse, generic and transferable visual features.
Rethinking Image Super Resolution From Long-Tailed Distribution Learning Perspective
Existing studies have empirically observed that the resolution of the low-frequency region is easier to enhance than that of the high-frequency one.
Exploiting Semantic Role Contextualized Video Features for Multi-Instance Text-Video Retrieval EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022
In this report, we present our approach for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022.
Ranked #11 on
Multi-Instance Retrieval
on EPIC-KITCHENS-100
RoME: Role-aware Mixture-of-Expert Transformer for Text-to-Video Retrieval
Most methods consider only one joint embedding space between global visual and textual features without considering the local structures of each modality.
Ranked #12 on
Video Retrieval
on YouCook2
Semantic Role Aware Correlation Transformer for Text to Video Retrieval
With the emergence of social media, voluminous video clips are uploaded every day, and retrieving the most relevant visual content with a language query becomes critical.
Ranked #13 on
Video Retrieval
on YouCook2
OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization
Numerous network compression methods such as pruning and quantization are proposed to reduce the model size significantly, of which the key is to find suitable compression allocation (e. g., pruning sparsity and quantization codebook) of each layer.
CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow
This is mainly because the correlation volume, the basis of pixel matching, is computed as the dot product of the convolutional features of the two images.
Ranked #11 on
Optical Flow Estimation
on KITTI 2015 (train)
Hierarchical Point Cloud Encoding and Decoding with Lightweight Self-Attention based Model
In this paper we present SA-CNN, a hierarchical and lightweight self-attention based encoding and decoding architecture for representation learning of point cloud data.
Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining
For stage two, the bounding box proposals with SPCR are grouped into some subsets, and the instance masks are mined inside each subset with a novel semantic propagation module and a property consistency graph module.
Learning Cross-Modal Retrieval With Noisy Labels
Recently, cross-modal retrieval is emerging with the help of deep multimodal learning.
A Survey of Embodied AI: From Simulators to Research Tasks
This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research.
Efficient Robotic Task Generalization Using Deep Model Fusion Reinforcement Learning
The proposed method is able to efficiently generalize the previously learned task by model fusion to solve the environment adaptation problem.
Cross-channel Communication Networks
Convolutional neural networks process input data by sending channel-wise feature response maps to subsequent layers.
6D Pose Estimation with Correlation Fusion
To our best knowledge, this is the first work to explore effective intra- and inter-modality fusion in 6D pose estimation.
Dual Adversarial Neural Transfer for Low-Resource Named Entity Recognition
We propose a new neural transfer method termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).
Clustering with Similarity Preserving
A key step of graph-based approach is the similarity graph construction.
DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition
We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).
Low Resource Named Entity Recognition
named-entity-recognition
+2
Scene Text Synthesis for Efficient and Effective Deep Network Training
Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique - the use of our synthesized images in deep network training is capable of achieving similar or even better scene text detection and scene text recognition performance as compared with using real images.
Spatial Fusion GAN for Image Synthesis
Recent advances in generative adversarial networks (GANs) have shown great potentials in realistic image synthesis whereas most existing works address synthesis realism in either appearance space or geometry space but few in both.
Holistic Multi-modal Memory Network for Movie Question Answering
In this paper, we present the Holistic Multi-modal Memory Network (HMMN) framework which fully considers the interactions between different input sources (multi-modal context, question) in each hop.
XAI Beyond Classification: Interpretable Neural Clustering
The second is implementing discrete $k$-means with a differentiable neural network that embraces the advantages of parallel computing, online clustering, and clustering-favorable representation learning.
TORNADO: A Spatio-Temporal Convolutional Regression Network for Video Action Proposal
Recently, the regression-based object detectors and long-term recurrent convolutional network (LRCN) have demonstrated superior performance in human action detection and recognition.
YoTube: Searching Action Proposal via Recurrent and Static Regression Networks
In this paper, we present YoTube-a novel network fusion framework for searching action proposals in untrimmed videos, where each action proposal corresponds to a spatialtemporal video tube that potentially locates one human action.
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text
Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text.
Discriminative Multi-Modal Feature Fusion for RGBD Indoor Scene Recognition
RGBD scene recognition has attracted increasingly attention due to the rapid development of depth sensors and their wide application scenarios.
Diagnosing State-Of-The-Art Object Proposal Methods
Recently, Hosang et al. conduct the first unified study of existing methods' in terms of various image-level degradations.
Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation
Image segmentation refers to the process to divide an image into nonoverlapping meaningful regions according to human perception, which has become a classic topic since the early ages of computer vision.
