Search Results for author:
Found 10 papers, 5 papers with code
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Recognizing the need for a comprehensive evaluation of LMM chart understanding, we also propose a MultiModal Chart Benchmark (\textbf{MMC-Benchmark}), a comprehensive human-annotated benchmark with nine distinct tasks evaluating reasoning capabilities over charts.
HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs.
Ranked #1 on
Visual Question Answering (VQA)
on HallusionBench
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts.
Ranked #4 on
Visual Question Answering (VQA)
on HallusionBench
COVID-VTS: Fact Extraction and Verification on Short Video Platforms
We introduce a new benchmark, COVID-VTS, for fact-checking multi-modal information involving short-duration videos with COVID19- focused information from both the real world and machine generation.
One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2
While recent research has progressively overcome the low-resolution constraint of one-shot face video re-enactment with the help of StyleGAN's high-fidelity portrait generation, these approaches rely on at least one of the following: explicit 2D/3D priors, optical flow based warping as motion descriptors, off-the-shelf encoders, etc., which constrain their performance (e. g., inconsistent predictions, inability to capture fine facial details and accessories, poor generalization, artifacts).
Robust One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2
Addressing these limitations, we propose a novel framework exploiting the implicit 3D prior and inherent latent properties of StyleGAN2 to facilitate one-shot face re-enactment at 1024x1024 (1) with zero dependencies on explicit structural priors, (2) accommodating attribute edits, and (3) robust to diverse facial expressions and head poses of the source frame.
Expressive Talking Head Video Encoding in StyleGAN2 Latent-Space
To this end, we propose an end-to-end expressive face video encoding approach that facilitates data-efficient high-quality video re-synthesis by optimizing low-dimensional edits of a single Identity-latent.
Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Faces
We propose to train a deep Convolutional Neural Network (CNN) to regress lighting parameters from a single face image.
Label Denoising Adversarial Network (LDAN) for Inverse Lighting of Face Images
We propose to train a deep Convolutional Neural Network (CNN) to regress lighting parameters from a single face image.
