Search Results for author:
Found 17 papers, 5 papers with code
Prompt-Guided Mask Proposal for Two-Stage Open-Vocabulary Segmentation
To overcome this challenge, existing methods often use multi-modal models like CLIP, which combine image and text features in a shared embedding space to bridge the gap between limited and extensive vocabulary recognition, resulting in a two-stage approach: In the first stage, a mask generator takes an input image to generate mask proposals, and the in the second stage the target mask is picked based on the query.
Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach
Self-improvement in multimodal large language models (MLLMs) is crucial for enhancing their reliability and robustness.
Treat Visual Tokens as Text? But Your MLLM Only Needs Fewer Efforts to See
In this study, we investigate the redundancy in visual computation at both the parameter and computational pattern levels within LLaVA, a representative MLLM, and introduce a suite of streamlined strategies to enhance efficiency.
Enhanced Controllability of Diffusion Models via Feature Disentanglement and Realism-Enhanced Sampling Methods
Second, we propose timestep-dependent weight scheduling for content and style features to further improve the performance.
PRedItOR: Text Guided Image Editing with Diffusion Prior
We combine this with structure preserving edits on the image decoder using existing approaches such as reverse DDIM to perform text guided image editing.
Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models
Based on this finding, we further propose a simple, light-weight image editing algorithm where the mixing weights of the two text embeddings are optimized for style matching and content preservation.
Fine-grained Image Captioning with CLIP Reward
Toward more descriptive and distinctive caption generation, we propose using CLIP, a multimodal encoder trained on huge image-text pairs from web, to calculate multimodal similarity and use it as a reward function.
Ranked #26 on
Image Captioning
on COCO Captions
StyleBabel: Artistic Style Tagging and Captioning
We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools.
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval
recommendation, and marketing services.
Towards Zero-shot Cross-lingual Image Retrieval and Tagging
There has been a recent spike in interest in multi-modal Language and Vision problems.
Multimodal Contrastive Training for Visual Representation Learning
We first train our model on COCO and evaluate the learned visual representations on various downstream tasks including image classification, object detection, and instance segmentation.
Towards Zero-shot Cross-lingual Image Retrieval
There has been a recent spike in interest in multi-modal Language and Vision problems.
Multi-Modal Retrieval using Graph Neural Networks
Most real world applications of image retrieval such as Adobe Stock, which is a marketplace for stock photography and illustrations, need a way for users to find images which are both visually (i. e. aesthetically) and conceptually (i. e. containing the same salient objects) as a query image.
Search Query Language Identification Using Weak Labeling
Language identification is a well-known task for natural language documents.
Towards Semantic Query Segmentation
Query Segmentation is one of the critical components for understanding users' search intent in Information Retrieval tasks.
Visual Search at eBay
We harness the availability of large image collection of eBay listings and state-of-the-art deep learning techniques to perform visual search at scale.
