Search Results for author:
Found 19 papers, 16 papers with code
LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?
Diffusion models have exhibited remarkable capabilities in text-to-image generation.
Towards Multimodal Video Paragraph Captioning Models Robust to Missing Modality
Video paragraph captioning (VPC) involves generating detailed narratives for long videos, utilizing supportive modalities such as speech and event boundaries.
TempCompass: Do Video LLMs Really Understand Videos?
Motivated by these two problems, we propose the \textbf{TempCompass} benchmark, which introduces a diversity of temporal aspects and task formats.
PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chain
To address this, we introduce Embodied-Instruction-Evolution (EIE), an automatic framework for synthesizing instruction tuning examples in multimodal embodied environments.
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models
The ability to perceive how objects change over time is a crucial ingredient in human intelligence.
FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generation
The multi-aspect categorization of FETV enables fine-grained analysis of the metrics' reliability in different scenarios.
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding
TESTA can reduce the number of visual tokens by 75% and thus accelerate video encoding.
Ranked #1 on
Video Retrieval
on Condensed Movies
(using extra training data)
Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond
In this study, we explore the potential of Multimodal Large Language Models (MLLMs) in improving embodied decision-making processes for agents.
M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
To tackle this challenge and promote research in the vision-language field, we introduce the Multi-Modal, Multilingual Instruction Tuning (M$^3$IT) dataset, designed to optimize VLM alignment with human instructions.
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition
This work proposes POMP, a prompt pre-training method for vision-language models.
Delving into the Openness of CLIP
However, evaluating the openness of CLIP-like models is challenging, as the models are open to arbitrary vocabulary in theory, but their accuracy varies in practice.
CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark
We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic.
Dynamic Knowledge Distillation for Pre-trained Language Models
Knowledge distillation~(KD) has been proved effective for compressing large-scale pre-trained language models.
Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification
Data augmentation aims to enrich training samples for alleviating the overfitting issue in low-resource or class-imbalanced situations.
Learning Relation Alignment for Calibrated Cross-modal Retrieval
To bridge the semantic gap between the two modalities, previous studies mainly focus on word-region alignment at the object level, lacking the matching between the linguistic relation among the words and the visual relation among the regions.
Ranked #4 on
Image-to-Text Retrieval
on MS COCO
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade
On the other hand, the exiting decisions made by internal classifiers are unreliable, leading to wrongly emitted early predictions.
DCA: Diversified Co-Attention towards Informative Live Video Commenting
In this paper, we aim to collect diversified information from video and text for informative comment generation.
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency
Experiments on three popular datasets using convolutional as well as LSTM models show that PWWS reduces the classification accuracy to the most extent, and keeps a very low word substitution rate.
