Search Results for author:
Found 62 papers, 37 papers with code
Context-Aware Deep Spatio-Temporal Network for Hand Pose Estimation from Depth Images
As a fundamental and challenging problem in computer vision, hand pose estimation aims to estimate the hand joint locations from depth images.
An Early Study on Intelligent Analysis of Speech under COVID-19: Severity, Sleep Quality, Fatigue, and Anxiety
In particular, by analysing speech recordings from these patients, we construct audio-only-based models to automatically categorise the health state of patients from four aspects, including the severity of illness, sleep quality, fatigue, and anxiety.
Accurate RGB-D Salient Object Detection via Collaborative Learning
The explicitly extracted edge information goes together with saliency to give more emphasis to the salient regions and object boundaries.
Ranked #19 on
RGB-D Salient Object Detection
on NJU2K
Table2Charts: Recommending Charts by Learning Shared Table Representations
It is common for people to create different types of charts to explore a multi-dimensional dataset (table).
ChemistryQA: A Complex Question Answering Dataset from Chemistry
We notice that some real-world QA tasks are more complex, which cannot be solved by end-to-end neural networks or translated to any kind of formal representations.
Dynamic Context-Sensitive Filtering Network for Video Salient Object Detection
Our bidirectional dynamic fusion strategy encourages the interaction of spatial and temporal information in a dynamic manner.
Ranked #12 on
Video Polyp Segmentation
on SUN-SEG-Easy (Unseen)
Boundary Proposal Network for Two-Stage Natural Language Video Localization
State-of-the-art NLVL methods are almost in one-stage fashion, which can be typically grouped into two categories: 1) anchor-based approach: it first pre-defines a series of video segment candidates (e. g., by sliding window), and then does classification for each candidate; 2) anchor-free approach: it directly predicts the probabilities for each video frame as a boundary or intermediate frame inside the positive segment.
Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations
We extract degradation prior at task-level with the proposed ConditionNet, which will be used to adapt the parameters of the basic SR network (BaseNet).
Deep Learning for Weakly-Supervised Object Detection and Object Localization: A Survey
With the success of deep neural networks in object detection, both WSOD and WSOL have received unprecedented attention.
Deconfounded Video Moment Retrieval with Causal Intervention
To fill the research gap, we propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.
Calibrated RGB-D Salient Object Detection
Complex backgrounds and similar appearances between objects and their surroundings are generally recognized as challenging scenarios in Salient Object Detection (SOD).
Ranked #13 on
Thermal Image Segmentation
on RGB-T-Glass-Segmentation
Learning Calibrated Medical Image Segmentation via Multi-Rater Agreement Modeling
To our knowledge, our work is the first in producing calibrated predictions under different expertise levels for medical image segmentation.
Advancing biological super-resolution microscopy through deep learning: a brief review
We conclude with an outlook on how deep learning could shape the future of this new generation of light microscopy technology.
Decoupling Strategy and Surface Realization for Task-oriented Dialogues
The core is to construct a latent content space for strategy optimization and disentangle the surface style from it.
PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices
We investigate the applicability of the anchor-free strategy on lightweight object detection models.
Ranked #1 on
Object Detection
on MSCOCO
Meeting Summarization with Pre-training and Clustering Methods
Lastly, we compare the performance of our baseline models with BART, a state-of-the-art language model that is effective for summarization.
Joint Semantic Mining for Weakly Supervised RGB-D Salient Object Detection
As a by-product, a CapS dataset is constructed by augmenting existing benchmark training set with additional image tags and captions.
Rethinking the Two-Stage Framework for Grounded Situation Recognition
Since each verb is associated with a specific set of semantic roles, all existing GSR methods resort to a two-stage framework: predicting the verb in the first stage and detecting the semantic roles in the second stage.
Ranked #3 on
Situation Recognition
on imSitu
Video as Conditional Graph Hierarchy for Multi-Granular Question Answering
To align with the multi-granular essence of linguistic concepts in language queries, we propose to model video as a conditional graph hierarchy which weaves together visual facts of different granularity in a level-wise manner, with the guidance of corresponding textual cues.
Ranked #23 on
Video Question Answering
on NExT-QA
Generating Diverse and Natural 3D Human Motions From Text
Automated generation of 3D human motions from text is a challenging problem.
Ranked #6 on
Motion Synthesis
on InterHuman
Exploring Denoised Cross-Video Contrast for Weakly-Supervised Temporal Action Localization
Inspired by recent success in unsupervised contrastive representation learning, we propose a novel denoised cross-video contrastive algorithm, aiming to enhance the feature discrimination ability of video snippets for accurate temporal action localization in the weakly-supervised setting.
Content-Variant Reference Image Quality Assessment via Knowledge Distillation
The comparisons of distribution differences between HQ and LQ images can help our model better assess the image quality.
Video Question Answering: Datasets, Algorithms and Challenges
Video Question Answering (VideoQA) aims to answer natural language questions according to the given videos.
Fine-Grained Scene Graph Generation with Data Transfer
Scene graph generation (SGG) is designed to extract (subject, predicate, object) triplets in images.
Ranked #1 on
Predicate Classification
on Visual Genome
3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective
This research aims to study a self-supervised 3D clothing reconstruction method, which recovers the geometry shape and texture of human clothing from a single image.
Ranked #1 on
Single-View 3D Reconstruction
on CUB-200-2011
Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection
The laborious and time-consuming manual annotation has become a real bottleneck in various practical scenarios.
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models
We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on position-insensitive tasks with grounded inputs.
Ranked #1 on
Visual Commonsense Reasoning
on VCR (Q-AR) test
Invariant Grounding for Video Question Answering
At its core is understanding the alignments between visual scenes in video and linguistic semantics in question to yield the answer.
Structured and Natural Responses Co-generation for Conversational Search
Existing approaches either 1) predict structured dialog acts first and then generate natural response; or 2) map conversation context to natural responses directly in an end-to-end manner.
MetaComp: Learning to Adapt for Online Depth Completion
However, facing a new environment where the test data occurs online and differs from the training data in the RGB image content and depth sparsity, the trained model might suffer severe performance drop.
Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization
The key idea underpinning the proposed method is to integrate fine- and coarse-grained retrieval as matching data points with small and large fluctuations, respectively.
Composed Image Retrieval (CoIR)
Image Retrieval with Multi-Modal Query
+1
Driving Style Recognition at First Impression for Online Trajectory Prediction
This paper proposes a new driving style recognition approach that allows autonomous vehicles (AVs) to perform trajectory predictions for surrounding vehicles with minimal data.
Multi-queue Momentum Contrast for Microvideo-Product Retrieval
The booming development and huge market of micro-videos bring new e-commerce channels for merchants.
MRTNet: Multi-Resolution Temporal Network for Video Sentence Grounding
In this work, we propose a novel multi-resolution temporal video sentence grounding network: MRTNet, which consists of a multi-modal feature encoder, a Multi-Resolution Temporal (MRT) module, and a predictor module.
Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-Based Active Learning
Moreover, we treat the uncertainty score of frames in a video as a whole, and estimate the difficulty of each video, which can further relieve the burden of video selection.
Multispectral Video Semantic Segmentation: A Benchmark Dataset and Baseline
This gives rise to significantly more robust segmentation of image objects in complex scenes and under adverse conditions.
WINNER: Weakly-Supervised hIerarchical decompositioN and aligNment for Spatio-tEmporal Video gRounding
WINNER first builds the language decomposition tree in a bottom-up manner, upon which the structural attention mechanism and top-down feature backtracking jointly build a multi-modal decomposition tree, permitting a hierarchical understanding of unstructured videos.
MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer
To effectively integrate these two cutting-edge techniques for the Medical image segmentation, we propose a novel Transformer-based Diffusion framework, called MedSegDiff-V2.
Scalable Attribution of Adversarial Attacks via Multi-Task Learning
In this paper, AAP is defined as the recognition of three signatures, i. e., {\em attack algorithm}, {\em victim model} and {\em hyperparameter}.
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by learning the ``soft prompts'' to condition frozen pre-training models.
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Based on that, we further introduce a novel Entangled cross-modal prompt approach for open-world predicate scene graph generation (Epic), where models can generalize to unseen predicates in a zero-shot manner.
Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications
Recently, Meta AI Research approaches a general, promptable Segment Anything Model (SAM) pre-trained on an unprecedentedly large segmentation dataset (SA-1B).
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
In Med-SA, we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation.
VPGTrans: Transfer Visual Prompt Generator across LLMs
While developing a new multimodal LLM (MLLM) by pre-training on tremendous image-text pairs from scratch can be exceedingly resource-consuming, connecting an existing LLM with a comparatively lightweight visual prompt generator (VPG) becomes a feasible paradigm.
Generating Visual Spatial Description via Holistic 3D Scene Understanding
With an external 3D scene extractor, we obtain the 3D objects and scene features for input images, based on which we construct a target object-centered 3D spatial scene graph (Go3D-S2G), such that we model the spatial semantics of target objects within the holistic 3D scenes.
Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment
Unpaired cross-lingual image captioning has long suffered from irrelevancy and disfluency issues, due to the inconsistencies of the semantic scene and syntax attributes during transfer.
In Defense of Clip-based Video Relation Detection
While recent video-based methods utilizing video tubelets have shown promising results, we argue that the effective modeling of spatial and temporal context plays a more significant role than the choice between clip tubelets and video tubelets.
Panoptic Scene Graph Generation with Semantics-Prototype Learning
To promise consistency and accuracy during the transfer process, we propose to measure the invariance of representations in each predicate class, and learn unbiased prototypes of predicates with different intensities.
Ranked #3 on
Panoptic Scene Graph Generation
on PSG Dataset
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
This shortcoming results in MLLMs' underperformance in comprehending demonstrative instructions consisting of multiple, interleaved, and multimodal instructions that demonstrate the required context to complete a task.
Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation
To be specific, we first introduce an ID-aware Multi-modal Transformer module in the item representation learning stage to facilitate information interaction among different features.
ControlRetriever: Harnessing the Power of Instructions for Controllable Retrieval
Recent studies have shown that dense retrieval models, lacking dedicated training data, struggle to perform well across diverse retrieval tasks, as different retrieval tasks often entail distinct search intents.
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
Ranked #1 on
Animal Pose Estimation
on Animal3D
Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs
In this work, we investigate strengthening the awareness of video dynamics for DMs, for high-quality T2V generation.
NExT-GPT: Any-to-Any Multimodal LLM
While recently Multimodal Large Language Models (MM-LLMs) have made exciting strides, they mostly fall prey to the limitation of only input-side multimodal understanding, without the ability to produce content in multiple modalities.
Towards Complex-query Referring Image Segmentation: A Novel Benchmark
Referring Image Understanding (RIS) has been extensively studied over the past decade, leading to the development of advanced algorithms.
Domain-wise Invariant Learning for Panoptic Scene Graph Generation
Panoptic Scene Graph Generation (PSG) involves the detection of objects and the prediction of their corresponding relationships (predicates).
Towards Robust Multi-Modal Reasoning via Model Selection
The reasoning capabilities of LLM (Large Language Model) are widely acknowledged in recent research, inspiring studies on tool learning and autonomous agents.
NExT-Chat: An LMM for Chat, Detection and Segmentation
The development of large language models (LLMs) has greatly advanced the field of multimodal understanding, leading to the emergence of large multimodal models (LMMs).
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
Visual programming, a modular and generalizable paradigm, integrates different modules and Python operators to solve various vision-language tasks.
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
Navigating drones through natural language commands remains challenging due to the dearth of accessible multi-modal datasets and the stringent precision requirements for aligning visual and textual data.
Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization
To comprehensively learn the relation between informative patches and fine-grained semantics, the multi-instance knowledge distillation is implemented on both the region/image crop pairs from the teacher and student net, and the region-image crops inside the teacher / student net, which we term as intra-level multi-instance distillation and inter-level multi-instance distillation.
Fine-Grained Visual Categorization
Knowledge Distillation
+2
GOOD: Towards Domain Generalized Orientated Object Detection
In this paper, we propose the task of domain generalized oriented object detection, which intends to explore the generalization of oriented object detectors on arbitrary unseen target domains.
