Search Results for author:
Found 28 papers, 14 papers with code
An Egocentric Vision-Language Model based Portable Real-time Smart Assistant
We present Vinci, a vision-language system designed to provide real-time, comprehensive AI assistance on portable devices.
Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning
However, existing egocentric video representation learning methods mainly focus on aligning video representation with high-level narrations, overlooking the intricate dynamics between hands and objects.
Text-promptable Propagation for Referring Medical Image Sequence Segmentation
We develop a strong baseline model, Text-Promptable Propagation (TPP), designed to exploit the intrinsic relationships among sequential images and their associated textual descriptions.
Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model
We introduce Vinci, a real-time embodied smart assistant built upon an egocentric vision-language model.
CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding
However, because of the inherent limitation of MCQ-based evaluation and the increasing reasoning ability of MLLMs, models can give the current answer purely by combining short video understanding with elimination, without genuinely understanding the video content.
EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation
In this report, we present our solutions to the EgoVis Challenges in CVPR 2024, including five tracks in the Ego4D challenge and three tracks in the EPIC-Kitchens challenge.
Ranked #1 on
Long Term Action Anticipation
on Ego4D
(using extra training data)
Concept-Attention Whitening for Interpretable Skin Lesion Diagnosis
In the former branch, we train a convolutional neural network (CNN) with an inserted CAW layer to perform skin lesion diagnosis.
QMix: Quality-aware Learning with Mixed Noise for Robust Retinal Disease Diagnosis
Previous noise learning methods mainly considered noise arising from images being mislabeled, i. e. label noise, assuming that all mislabeled images are of high image quality.
EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World
Along with the videos we record high-quality gaze data and provide detailed multimodal annotations, formulating a playground for modeling the human ability to bridge asynchronous procedural actions from different viewpoints.
Ranked #1 on
Action Anticipation
on EgoExoLearn
(using extra training data)
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
We introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
Ranked #1 on
Action Classification
on MIT
Domain Adaptation Using Pseudo Labels for COVID-19 Detection
In response to the need for rapid and accurate COVID-19 diagnosis during the global pandemic, we present a two-stage framework that leverages pseudo labels for domain adaptation to enhance the detection of COVID-19 from CT scans.
Advancing COVID-19 Detection in 3D CT Scans
To make a more accurate diagnosis of COVID-19, we propose a straightforward yet effective model.
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks.
Ranked #2 on
Temporal Action Localization
on FineAction
Anatomical Structure-Guided Medical Vision-Language Pre-training
For finding and existence, we regard them as image tags, applying an image-tag recognition decoder to associate image features with their respective tags within each sample and constructing soft labels for contrastive learning to improve the semantic association of different image-report pairs.
Retrieval-Augmented Egocentric Video Captioning
In this paper, (1) we develop EgoInstructor, a retrieval-augmented multimodal captioning model that automatically retrieves semantically relevant third-person instructional videos to enhance the video captioning of egocentric videos.
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
Enhanced Knowledge Injection for Radiology Report Generation
Automatic generation of radiology reports holds crucial clinical value, as it can alleviate substantial workload on radiologists and remind less experienced ones of potential anomalies.
VideoLLM: Modeling Video Sequence with Large Language Models
Building upon this insight, we propose a novel framework called VideoLLM that leverages the sequence reasoning capabilities of pre-trained LLMs from natural language processing (NLP) for video sequence understanding.
Mask Hierarchical Features For Self-Supervised Learning
We mask part of patches in the representation space and then utilize sparse visible patches to reconstruct high semantic image representation.
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision
The former aims to infer all masked entities in the caption given the group tokens, that enables the model to learn fine-grained alignment between visual groups and text entities.
Open Vocabulary Semantic Segmentation
Open-Vocabulary Semantic Segmentation
+1
InternVideo: General Video Foundation Models via Generative and Discriminative Learning
Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications.
Ranked #1 on
Action Recognition
on Something-Something V1
(using extra training data)
Cross-Field Transformer for Diabetic Retinopathy Grading on Two-field Fundus Images
However, automatic DR grading based on two-field fundus photography remains a challenging task due to the lack of publicly available datasets and effective fusion strategies.
Boosting COVID-19 Severity Detection with Infection-aware Contrastive Mixup Classification
In our approach, we devise a novel infection-aware 3D Contrastive Mixup Classification network for severity grading.
CMC v2: Towards More Accurate COVID-19 Detection with Discriminative Video Priors
This paper presents our solution for the 2nd COVID-19 Competition, occurring in the framework of the AIMIA Workshop at the European Conference on Computer Vision (ECCV 2022).
Deep-OCTA: Ensemble Deep Learning Approaches for Diabetic Retinopathy Analysis on OCTA Images
In the image quality assessment task, we create an ensemble of InceptionV3, SE-ResNeXt, and Vision Transformer models.
FDVTS's Solution for 2nd COV19D Competition on COVID-19 Detection and Severity Analysis
This paper presents our solution for the 2nd COVID-19 Competition, occurring in the framework of the AIMIA Workshop in the European Conference on Computer Vision (ECCV 2022).
CREAM: Weakly Supervised Object Localization via Class RE-Activation Mapping
In this paper, we empirically prove that this problem is associated with the mixup of the activation values between less discriminative foreground regions and the background.
MDU-Net: Multi-scale Densely Connected U-Net for biomedical image segmentation
The three multi-scale dense connections improve U-net performance by up to 1. 8% on test A and 3. 5% on test B in the MICCAI Gland dataset.
