Search Results for author:
Found 32 papers, 14 papers with code
Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
Large Vision-Language Models offer a new paradigm for AI-driven image understanding, enabling models to perform tasks without task-specific training.
A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI
Despite the excitement behind biomedical artificial intelligence (AI), access to high-quality, diverse, and large-scale data - the foundation for modern AI systems - is still a bottleneck to unlocking its full potential.
MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
Scientific research demands sophisticated reasoning over multimodal data, a challenge especially prevalent in biology.
SurgiSAM2: Fine-tuning a foundational model for surgical video anatomy segmentation and detection
Conclusion: SAM 2 achieves remarkable zero-shot and fine-tuned performance for surgical scene segmentation, surpassing prior SOTA models across several organ classes of diverse datasets.
Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models
Large pre-trained models, or foundation models, have shown impressive performance when adapted to a variety of downstream tasks, often out-performing specialized models.
CellFlow: Simulating Cellular Morphology Changes via Flow Matching
Building a virtual cell capable of accurately simulating cellular behaviors in silico has long been a dream in computational biology.
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature
The development of vision-language models (VLMs) is driven by large-scale and diverse multimodal datasets.
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation
The rapid development of vision language models (VLMs) demands rigorous and reliable evaluation.
Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration
In this work, we examine the popular acceleration approach of early pruning of visual tokens inside the language model and find that its strong performance across many tasks is not due to an exceptional ability to compress visual information, but rather the benchmarks' limited ability to assess fine-grained visual capabilities.
DeforHMR: Vision Transformer with Deformable Cross-Attention for 3D Human Mesh Recovery
DeforHMR leverages a novel query-agnostic deformable cross-attention mechanism within the transformer decoder to effectively regress the visual features extracted from a frozen pretrained vision transformer (ViT) encoder.
Motion Diffusion-Guided 3D Global HMR from a Dynamic Camera
Motion capture technologies have transformed numerous fields, from the film and gaming industries to sports science and healthcare, by providing a tool to capture and analyze human movement in great detail.
Zero-shot Action Localization via the Confidence of Large Vision-Language Models
Precise action localization in untrimmed video is vital for fields such as professional sports and minimally invasive surgery, where the delineation of particular motions in recordings can dramatically enhance analysis.
Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models
Social dynamics in close human interactions pose significant challenges for Human Mesh Estimation (HME), particularly due to the complexity of physical contacts and the scarcity of training data.
How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities
Accurate modeling of cells is important for this understanding as well as for determining the root causes of disease.
Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision
The performance of Large Vision Language Models (LVLMs) is dependent on the size and quality of their training datasets.
μ-Bench: A Vision-Language Benchmark for Microscopy Understanding
Recent advances in microscopy have enabled the rapid generation of terabytes of image data in cell biology and biomedical research.
Why are Visually-Grounded Language Models Bad at Image Classification?
Image classification is one of the most fundamental capabilities of machine vision intelligence.
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models
Advancements in vision-language models (VLMs) have propelled the field of computer vision, particularly in the zero-shot learning setting.
Depth-guided NeRF Training via Earth Mover's Distance
This work proposes a novel approach to uncertainty in depth priors for NeRF supervision.
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Long-form video understanding represents a significant challenge within computer vision, demanding a model capable of reasoning over long multi-modal sequences.
Ranked #6 on
Zero-Shot Video Question Answer
on NExT-QA
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Frontier general-domain models such as GPT-4V still have significant performance gaps in multimodal biomedical applications.
Multi-Human Mesh Recovery with Transformers
Conventional approaches to human mesh recovery predominantly employ a region-based strategy.
Revisiting Active Learning in the Era of Vision Foundation Models
Foundation vision or vision-language models are trained on large unlabeled or noisy data and learn robust representations that can achieve impressive zero- or few-shot performance on diverse tasks.
Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM
We present Human-LRM, a diffusion-guided feed-forward model that predicts the implicit field of a human from a single image.
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data
However, this assumption is under-explored due to the poorly understood geometry of the multi-modal contrastive space, where a modality gap exists.
Describing Differences in Image Sets with Natural Language
To aid in this discovery process, we explore the task of automatically describing the differences between two $\textbf{sets}$ of images, which we term Set Difference Captioning.
Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Models
We conclude that since the view token controls the 3D `rendering' viewpoint, there is likely a scene representation embedded in frozen 2D diffusion models.
Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains
Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images.
