Search Results for author:
Found 19 papers, 15 papers with code
CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions
Previous works show that noisy, web-crawled image-text pairs may limit vision-language pretraining like CLIP and propose learning with synthetic captions as a promising alternative.
MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine
We then build a comprehensive knowledge base and prompt multimodal large language models to perform retrieval-augmented generation with the identified ROIs as guidance, resulting in multigranular texual descriptions.
Ranked #1 on
Medical Visual Question Answering
on SLAKE-English
(using extra training data)
Medical Visual Question Answering
Visual Question Answering (VQA)
What If We Recaption Billions of Web Images with LLaMA-3?
For discriminative models like CLIP, we observe enhanced zero-shot performance in cross-modal retrieval tasks.
Ranked #96 on
Visual Question Answering
on MM-Vet
Autoregressive Pretraining with Mamba in Vision
The vision community has started to build with the recently developed state space model, Mamba, as the new backbone for a range of tasks.
Medical Vision Generalist: Unifying Medical Imaging Tasks in Context
This study presents Medical Vision Generalist (MVG), the first foundation model capable of handling various medical imaging tasks -- such as cross-modal synthesis, image segmentation, denoising, and inpainting -- within a unified image-to-image generation framework.
Scaling White-Box Transformers for Vision
CRATE, a white-box transformer architecture designed to learn compressed and sparse representations, offers an intriguing alternative to standard vision transformers (ViTs) due to its inherent mathematical interpretability.
3D-TransUNet for Brain Metastases Segmentation in the BraTS2023 Challenge
We identify that the Decoder-only 3D-TransUNet model should offer enhanced efficacy in the segmentation of brain metastases, as indicated by our 5-fold cross-validation on the training set.
Revisiting Adversarial Training at Scale
For example, by training on DataComp-1B dataset, our AdvXL empowers a vanilla ViT-g model to substantially surpass the previous records of $l_{\infty}$-, $l_{2}$-, and $l_{1}$-robust accuracy by margins of 11. 4%, 14. 2% and 12. 9%, respectively.
3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers
In this paper, we extend the 2D TransUNet architecture to a 3D network by building upon the state-of-the-art nnU-Net architecture, and fully exploring Transformers' potential in both the encoder and decoder design.
Consistency-guided Meta-Learning for Bootstrapping Semi-Supervised Medical Image Segmentation
Specifically, our approach first involves training a segmentation model on a small set of clean labeled images to generate initial labels for unlabeled data.
CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a \$10,000 Budget; An Extra \$4,000 Unlocks 81.8% Accuracy
The recent work CLIPA presents an inverse scaling law for CLIP training -- whereby the larger the image/text encoders used, the shorter the sequence length of image/text tokens that can be applied in training.
An Inverse Scaling Law for CLIP Training
However, its associated training cost is prohibitively high, imposing a significant barrier to its widespread exploration.
Unleashing the Power of Visual Prompting At the Pixel Level
This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks.
In Defense of Image Pre-Training for Spatiotemporal Recognition
Inspired by this observation, we hypothesize that the key to effectively leveraging image pre-training lies in the decomposition of learning spatial and temporal features, and revisiting image pre-training as the appearance prior to initializing 3D kernels.
Fast AdvProp
Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e. g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified.
L2B: Learning to Bootstrap Robust Models for Combating Label Noise
Extensive experiments demonstrate that our method effectively mitigates the challenges of noisy labels, often necessitating few to no validation samples, and is well generalized to other tasks such as image segmentation.
Ranked #8 on
Image Classification
on Clothing1M (using clean data)
(using extra training data)
Pose-guided Generative Adversarial Net for Novel View Action Synthesis
We focus on the problem of novel-view human action synthesis.
CT-Net: Channel Tensorization Network for Video Classification
It can learn to exploit spatial, temporal and channel attention in a high-dimensional manner, to improve the cooperative power of all the feature dimensions in our CT-Module.
Ranked #19 on
Action Recognition
on Something-Something V1
SmallBigNet: Integrating Core and Contextual Views for Video Classification
Our SmallBig network outperforms a number of recent state-of-the-art approaches, in terms of accuracy and/or efficiency.
