Search Results for author:
Found 16 papers, 10 papers with code
Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models
BEV-InMLLM integrates multi-view, spatial awareness, and temporal semantics to enhance MLLMs' capabilities on NuInstruct tasks.
EtC: Temporal Boundary Expand then Clarify for Weakly Supervised Video Grounding with Multimodal Large Language Model
To further clarify the noise of expanded boundaries, we combine mutual learning with a tailored proposal-level contrastive objective to use a learnable approach to harmonize a balance between incomplete yet clean (initial) and comprehensive yet noisy (expanded) boundaries for more precise ones.
GraphEcho: Graph-Driven Unsupervised Domain Adaptation for Echocardiogram Video Segmentation
This paper studies the unsupervised domain adaption (UDA) for echocardiogram video segmentation, where the goal is to generalize the model trained on the source domain to other unlabelled target domains.
GL-Fusion: Global-Local Fusion Network for Multi-view Echocardiogram Video Segmentation
Additionally, a Multi-view Local-based Fusion Module (MLFM) is designed to extract correlations of cardiac structures from different views.
HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving
For the first time, we leverage singular multimodal large language models (MLLMs) to consolidate multiple autonomous driving tasks from videos, i. e., the Risk Object Localization and Intention and Suggestion Prediction (ROLISP) task.
Uniformly Distributed Category Prototype-Guided Vision-Language Framework for Long-Tail Recognition
Category prototype-guided mechanism for image-text matching makes the features of different classes converge to these distinct and uniformly distributed category prototypes, which maintain a uniform distribution in the feature space, and improve class boundaries.
Context-Aware Pseudo-Label Refinement for Source-Free Domain Adaptive Fundus Image Segmentation
To this end, we propose a context-aware pseudo-label refinement method for SF-UDA.
Boosting Weakly-Supervised Temporal Action Localization with Text Information
For the discriminative objective, we propose a Text-Segment Mining (TSM) mechanism, which constructs a text description based on the action class label, and regards the text as the query to mine all class-related segments.
Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint
The proposed Bi-SCC firstly adopts a temporal context augmentation to generate an augmented video that breaks the correlation between positive actions and their co-scene actions in the inter-video; Then, a semantic consistency constraint (SCC) is used to enforce the predictions of the original video and augmented video to be consistent, hence suppressing the co-scene actions.
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction from Echocardiogram Videos
We also introduce teacher-student distillation to distill the information from LV segmentation masks into an end-to-end LVEF regression model that only requires video inputs.
Learning Shadow Correspondence for Video Shadow Detection
We further design a new evaluation metric to evaluate the temporal stability of the video shadow detection results.
Free Lunch for Surgical Video Understanding by Distilling Self-Supervisions
Our key insight is to distill knowledge from publicly available models trained on large generic datasets4 to facilitate the self-supervised learning of surgical videos.
Less is More: Surgical Phase Recognition from Timestamp Supervision
Our study uncovers unique insights of surgical phase recognition with timestamp supervisions: 1) timestamp annotation can reduce 74% annotation time compared with the full annotation, and surgeons tend to annotate those timestamps near the middle of phases; 2) extensive experiments demonstrate that our method can achieve competitive results compared with full supervision methods, while reducing manual annotation cost; 3) less is more in surgical phase recognition, i. e., less but discriminative pseudo labels outperform full but containing ambiguous frames; 4) the proposed UATD can be used as a plug and play method to clean ambiguous labels near boundaries between phases, and improve the performance of the current surgical phase recognition methods.
Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videos
Automatic surgical phase recognition plays a vital role in robot-assisted surgeries.
Support-Set Based Cross-Supervision for Video Grounding
The contrastive objective aims to learn effective representations by contrastive learning, while the caption objective can train a powerful video encoder supervised by texts.
Weakly Supervised Temporal Action Localization with Segment-Level Labels
Specifically, we devise a partial segment loss regarded as a loss sampling to learn integral action parts from labeled segments.
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
