Search Results for author:
Found 10 papers, 9 papers with code
Ola: Pushing the Frontiers of Omni-Modal Language Model with Progressive Modality Alignment
Our training pipeline begins with the most distinct modalities: image and text, then gradually expands the skill sets of the model using speech data that connects language and audio knowledge, and video data that connects all modalities.
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
In this paper, we present Insight-V, an early effort to 1) scalably produce long and robust reasoning data for complex multi-modal tasks, and 2) an effective training pipeline to enhance the reasoning capabilities of multi-modal large language models (MLLMs).
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution
Visual data comes in various forms, ranging from small icons of just a few pixels to long videos spanning hours.
Ranked #1 on
Video Question Answering
on Perception Test
Efficient Inference of Vision Instruction-Following Models with Elastic Cache
Surrounding less important caches are then merged with these anchors, enhancing the preservation of contextual information in the KV caches while yielding an arbitrary acceleration ratio.
Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models
In the realm of vision-language understanding, the proficiency of models in interpreting and reasoning over visual content has become a cornerstone for numerous applications.
Ranked #130 on
Visual Question Answering
on MM-Vet
HandMIM: Pose-Aware Self-Supervised Learning for 3D Hand Mesh Estimation
Our approach involves a unified and multi-granularity strategy that includes a pseudo keypoint alignment module in the teacher-student framework for learning pose-aware semantic class tokens.
Unleashing Text-to-Image Diffusion Models for Visual Perception
In this paper, we propose VPD (Visual Perception with a pre-trained Diffusion model), a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks.
DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion
Unlike previous work that relies on carefully designed network architectures and loss functions to fuse the information from the source and target faces, we reformulate the face swapping as a conditional inpainting task, performed by a powerful diffusion model guided by the desired face attributes (e. g., identity and landmarks).
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks
We extend our method to hierarchical models including CNNs and hierarchical vision Transformers as well as more complex dense prediction tasks that require structured feature maps by formulating a more generic dynamic spatial sparsification framework with progressive sparsification and asymmetric computation for different spatial locations.
PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers
In this paper, we present a new method that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr that adopts a transformer encoder-decoder architecture for point cloud completion.
Ranked #1 on
Point Cloud Completion
on ShapeNet
(Chamfer Distance L2 metric)
