Search Results for author:
Found 15 papers, 14 papers with code
Multimodal Long Video Modeling Based on Temporal Dynamic Context
Secondly, we propose a novel temporal context compressor to reduce the number of tokens within each segment.
Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation
Solving complex long-horizon robotic manipulation problems requires sophisticated high-level planning capabilities, the ability to reason about the physical world, and reactively choose appropriate motor skills.
AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
Recently, multimodal large language models (MLLMs), such as GPT-4o, Gemini 1. 5 Pro, and Reka Core, have expanded their capabilities to include vision and audio modalities.
RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models
To further improve generation quality and alignment with user-specific information, we design a pipeline for data collection and create a specialized dataset for personalized training of MLLMs.
OneLLM: One Framework to Align All Modalities with Language
In detail, we first train an image projection module to connect a vision encoder with LLM.
Ranked #211 on
Visual Question Answering
on MM-Vet
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models
We present SPHINX, a versatile multi-modal large language model (MLLM) with a joint mixing of model weights, tuning tasks, and visual embeddings.
Ranked #2 on
Visual Question Answering
on BenchLMM
(using extra training data)
ImageBind-LLM: Multi-modality Instruction Tuning
During training, we adopt a learnable bind network to align the embedding space between LLaMA and ImageBind's image encoder.
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video.
Ranked #5 on
3D Question Answering (3D-QA)
on 3D MM-Vet
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
This strategy effectively alleviates the interference between the two tasks of image-text alignment and instruction following and achieves strong multi-modal reasoning with only a small-scale image-text and instruction dataset.
Ranked #6 on
Visual Question Answering (VQA)
on InfiMM-Eval
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
Ranked #2 on
Music Question Answering
on MusicQA
Few-Shot Object Detection via Variational Feature Aggregation
As few-shot object detectors are often trained with abundant base samples and fine-tuned on few-shot novel examples, the learned models are usually biased to base classes and sensitive to the variance of novel examples.
Expanding Low-Density Latent Regions for Open-Set Object Detection
Thus, unknown objects in low-density regions can be easily identified with the learned unknown probability.
LUAI Challenge 2021 on Learning to Understand Aerial Images
This report summarizes the results of Learning to Understand Aerial Images (LUAI) 2021 challenge held on ICCV 2021, which focuses on object detection and semantic segmentation in aerial images.
ReDet: A Rotation-equivariant Detector for Aerial Object Detection
More precisely, we incorporate rotation-equivariant networks into the detector to extract rotation-equivariant features, which can accurately predict the orientation and lead to a huge reduction of model size.
Ranked #21 on
Object Detection In Aerial Images
on DOTA
(using extra training data)
Align Deep Features for Oriented Object Detection
However most of existing methods rely on heuristically defined anchors with different scales, angles and aspect ratios and usually suffer from severe misalignment between anchor boxes and axis-aligned convolutional features, which leads to the common inconsistency between the classification score and localization accuracy.
Ranked #26 on
Object Detection In Aerial Images
on DOTA
(using extra training data)
