Search Results for author:
Found 33 papers, 2 papers with code
ViTAR: Vision Transformer with Any Resolution
Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration.
InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
Multimodal Large Language Models (MLLMs) have experienced significant advancements recently.
Ranked #37 on
Visual Question Answering
on MM-Vet
COCO is "ALL'' You Need for Visual Instruction Fine-tuning
We argue that datasets with diverse and high-quality detailed instruction following annotations are essential and adequate for MLLMs IFT.
Ranked #43 on
Visual Question Answering
on MM-Vet
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
In this survey, we comprehensively review the existing evaluation protocols of multimodal reasoning, categorize and illustrate the frontiers of MLLMs, introduce recent trends in applications of MLLMs on reasoning-intensive tasks, and finally discuss current practices and future directions.
Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Prompts
In light of the remarkable success of in-context learning in large language models, its potential extension to the vision domain, particularly with visual foundation models like Stable Diffusion, has sparked considerable interest.
Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis
Recent advancements in text-to-image (T2I) generative models have shown remarkable capabilities in producing diverse and imaginative visuals based on text prompts.
InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
To mitigate this issue, we manually curate a benchmark dataset specifically designed for MLLMs, with a focus on complex reasoning tasks.
Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling
Diffusion models excel at generating photo-realistic images but come with significant computational costs in both training and sampling.
RefineVIS: Video Instance Segmentation with Temporal Attention Refinement
We introduce a novel framework called RefineVIS for Video Instance Segmentation (VIS) that achieves good object association between frames and accurate segmentation masks by iteratively refining the representations using sequence context.
Ranked #3 on
Video Instance Segmentation
on YouTube-VIS 2021
(using extra training data)
Consistent Video Instance Segmentation with Inter-Frame Recurrent Attention
We propose a consistent end-to-end video instance segmentation framework with Inter-Frame Recurrent Attention to model both the temporal instance consistency for adjacent frames and the global temporal context.
Deep Frequency Filtering for Domain Generalization
Improving the generalization ability of Deep Neural Networks (DNNs) is critical for their practical uses, which has been a longstanding challenge.
SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answering
As a multi-modality task, it is challenging since it requires not only visual and textual understanding, but also the ability to align cross-modality representations.
Lifelong Unsupervised Domain Adaptive Person Re-identification with Coordinated Anti-forgetting and Adaptation
In this paper, to address more practical scenarios, we propose a new task, Lifelong Unsupervised Domain Adaptive (LUDA) person ReID.
Domain Adaptive Person Re-Identification
Knowledge Distillation
+4
MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark
This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments.
Ranked #2 on
Object Tracking
on MMPTRACK
Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation
Medical report generation is one of the most challenging tasks in medical image analysis.
TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking
TransMOT effectively models the interactions of a large number of objects by arranging the trajectories of the tracked objects as a set of sparse weighted graphs, and constructing a spatial graph transformer encoder layer, a temporal transformer encoder layer, and a spatial graph transformer decoder layer based on the graphs.
Ranked #2 on
Multi-Object Tracking
on 2DMOT15
(using extra training data)
Disentanglement-based Cross-Domain Feature Augmentation for Effective Unsupervised Domain Adaptive Person Re-identification
Each recomposed feature, obtained based on the domain-invariant feature (which enables a reliable inheritance of identity) and an enhancement from a domain specific feature (which enables the approximation of real distributions), is thus an "ideal" augmentation.
Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation
To fairly evaluate the performance, we also propose three specific evaluation metrics.
Real-time 3D Deep Multi-Camera Tracking
Our DMCT consists of 1) a fast and novel perspective-aware Deep GroudPoint Network, 2) a fusion procedure for ground-plane occupancy heatmap estimation, 3) a novel Deep Glimpse Network for person detection and 4) a fast and accurate online tracker.
Ranked #5 on
Multi-Object Tracking
on Wildtrack
Action4D: Online Action Recognition in the Crowd and Clutter
Recognizing every person's action in a crowded and cluttered environment is a challenging task in computer vision.
Real-time Multiple People Hand Localization in 4D Point Clouds
Different from the traditional multiple view approaches, which find key points in 2D and then triangulate to recover the 3D locations, our method directly processes the dynamic 3D data that involve both clutter and crowd.
``Factual'' or ``Emotional'': Stylized Image Captioning with Adaptive Learning and Attention
It uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context.
Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM
Sentiment analysis on large-scale social media data is important to bridge the gaps between social media contents and real world activities including political election prediction, individual and public emotional status monitoring and analysis, and so on.
"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention
It uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context.
Action4D: Real-time Action Recognition in the Crowd and Clutter
In this paper, we propose a real-time action recognition method, Action4D, which gives reliable and accurate results in the real-world settings.
End-to-End Convolutional Semantic Embeddings
Usually, Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are employed for learning image and sentence representations, respectively.
Image Captioning at Will: A Versatile Scheme for Effectively Injecting Sentiments into Image Descriptions
In this work, we propose two different models, which employ different schemes for injecting sentiments into image captions.
Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks
Existing work solves this problem by employing recurrent neural networks (RNNs) to model EHR data and utilizing simple attention mechanism to interpret the results.
Cultural Diffusion and Trends in Facebook Photographs
Online social media is a social vehicle in which people share various moments of their lives with their friends, such as playing sports, cooking dinner or just taking a selfie for fun, via visual means, that is, photographs.
Image Based Appraisal of Real Estate Properties
Real estate appraisal, which is the process of estimating the price for real estate properties, is crucial for both buys and sellers as the basis for negotiation and transaction.
Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
We hope that this data set encourages further research on visual emotion analysis.
Image Captioning with Semantic Attention
Automatically generating a natural language description of an image has attracted interests recently both because of its importance in practical applications and because it connects two major artificial intelligence fields: computer vision and natural language processing.
Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis.
