Search Results for author:
Found 24 papers, 16 papers with code
DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset
Building on our innovative model design, training recipe, and datasets, we develop BLIP3-o, a suite of state-of-the-art unified multimodal models.
SemiSAM+: Rethinking Semi-Supervised Medical Image Segmentation in the Era of Foundation Models
SemiSAM+ consists of one or multiple promptable foundation models as generalist models, and a trainable task-specific segmentation model as specialist model.
SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Images
Positron Emission Tomography (PET) imaging plays a crucial role in modern medical diagnostics by revealing the metabolic processes within a patient's body, which is essential for quantification of therapy response and monitoring treatment progress.
From My View to Yours: Ego-Augmented Learning in Large Vision Language Models for Understanding Exocentric Daily Living Activities
To address this, we aim to leverage the complementary nature of egocentric views to enhance LVLM's understanding of exocentric ADL videos.
Human-Object Interaction Detection
Knowledge Distillation
+1
ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Models
Our multi-image instruction data leads to an 8% improvement on Mantis-Eval.
Ranked #107 on
Visual Question Answering
on MM-Vet
BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions
We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that bridges the gap between descriptive synthetic captions and factual web-scale alt-text.
xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs
We present xGen-MM-Vid (BLIP-3-Video): a multimodal language model for videos, particularly designed to efficiently capture temporal information over multiple frames.
xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
We present xGen-VideoSyn-1, a text-to-video (T2V) generation model capable of producing realistic scenes from textual descriptions.
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models
The framework comprises meticulously curated datasets, a training recipe, model architectures, and a resulting suite of LMMs.
MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens
Multimodal interleaved datasets featuring free-form interleaved sequences of images and text are crucial for training frontier large multimodal models (LMMs).
LLAVIDAL: A Large LAnguage VIsion Model for Daily Activities of Living
To address this, we propose a semi-automated framework for curating ADL datasets, creating ADL-X, a multiview, multimodal RGBS instruction-tuning dataset.
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning
To enable this framework, we devise a scalable pipeline that automatically generates high-quality, instruction-tuning datasets from readily available captioning data across different modalities, and contribute 24K QA data for audio and 250K QA data for 3D.
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs).
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
This demonstrates that using policy gradient optimization to improve language agents, for which we believe our work is one of the first, seems promising and can be applied to optimize other models in the agent architecture to enhance agent performances over time.
REX: Rapid Exploration and eXploitation for AI Agents
In this paper, we propose an enhanced approach for Rapid Exploration and eXploitation for AI Agents called REX.
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding
It achieves a new SOTA of 50. 6% (top-1) on Objaverse-LVIS and 84. 7% (top-1) on ModelNet40 in zero-shot classification.
Ranked #10 on
3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Hierarchical Point Attention for Indoor 3D Object Detection
By plugging our proposed modules into the state-of-the-art transformer-based 3D detectors, we improve the previous best results on both benchmarks, with more significant improvements on smaller objects.
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding
Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets.
Ranked #4 on
Training-free 3D Point Cloud Classification
on ModelNet40
(using extra training data)
OTFPF: Optimal Transport-Based Feature Pyramid Fusion Network for Brain Age Estimation with 3D Overlapped ConvNeXt
In this paper, we propose an end-to-end neural network architecture, referred to as optimal transport based feature pyramid fusion (OTFPF) network, for the brain age estimation with T1 MRIs.
Activate index: an integrated index to reveal disrupted brain network organizations of major depressive disorder patients
This study for the first time discussed the differences between MDD and HC using both rich club and diverse club metrics and found the complementarity of them in analyzing brain networks.
A resource-efficient deep learning framework for low-dose brain PET image reconstruction and analysis
18F-fluorodeoxyglucose (18F-FDG) Positron Emission Tomography (PET) imaging usually needs a full-dose radioactive tracer to obtain satisfactory diagnostic results, which raises concerns about the potential health risks of radiation exposure, especially for pediatric patients.
Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
Robustness Evaluation of Transformer-based Form Field Extractors via Form Attacks
We propose a novel framework to evaluate the robustness of transformer-based form field extraction methods via form attacks.
