Search Results for author:
Found 20 papers, 16 papers with code
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward
Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM).
VBench: Comprehensive Benchmark Suite for Video Generative Models
We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.
OtterHD: A High-Resolution Multi-modality Model
In this paper, we present OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision.
Ranked #81 on
Visual Question Answering
on MM-Vet
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
In particular, we instruction tune vision-language models to generate detailed visual descriptions of camera trap images using similar terminology to experts.
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Large vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
FunQA: Towards Surprising Video Comprehension
Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention.
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
We release the MIMIC-IT dataset, instruction-response collection pipeline, benchmarks, and the Otter model.
Ranked #83 on
Visual Question Answering
on MM-Vet
Learning without Forgetting for Vision-Language Models
While traditional CIL methods focus on visual information to grasp core features, recent advances in Vision-Language Models (VLM) have shown promising capabilities in learning generalizable representations with the aid of textual information.
Latent Distribution Adjusting for Face Anti-Spoofing
In this work, we propose a unified framework called Latent Distribution Adjusting (LDA) with properties of latent, discriminative, adaptive, generic to improve the robustness of the FAS model by adjusting complex data distribution with multiple prototypes.
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Large language models (LLMs) have demonstrated significant universal capabilities as few/zero-shot learners in various tasks due to their pre-training on vast amounts of text data, as exemplified by GPT-3, which boosted to InstrctGPT and ChatGPT, effectively following natural language instructions to accomplish real-world tasks.
Ranked #8 on
Visual Question Answering
on BenchLMM
What Makes Good Examples for Visual In-Context Learning?
To overcome the problem, we propose a prompt retrieval framework to automate the selection of in-context examples.
3D Point Cloud Pre-training with Knowledge Distillation from 2D Images
The recent success of pre-trained 2D vision models is mostly attributable to learning from large-scale datasets.
On-Device Domain Generalization
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
Benchmarking Omni-Vision Representation through the Lens of Visual Realms
We benchmark ReCo and other advances in omni-vision representation studies that are different in architectures (from CNNs to transformers) and in learning paradigms (from supervised learning to self-supervised learning) on OmniBenchmark.
Neural Prompt Search
The size of vision models has grown exponentially over the last few years, especially after the emergence of Vision Transformer.
Ranked #1 on
Image Classification
on OmniBenchmark
(using extra training data)
Robust Face Anti-Spoofing with Dual Probabilistic Modeling
In this work, we attempt to fill this gap by automatically addressing the noise problem from both label and data perspectives in a probabilistic manner.
Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy
This work thus proposes a novel active learning framework for realistic dataset annotation.
Ranked #1 on
Image Classification
on Food-101
(using extra training data)
CelebA-Spoof Challenge 2020 on Face Anti-Spoofing: Methods and Results
It is the largest face anti-spoofing dataset in terms of the numbers of the data and the subjects.
CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations
The main reason is that current face anti-spoofing datasets are limited in both quantity and diversity.
