Search Results for author:
Found 15 papers, 7 papers with code
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes.
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations
We propose an architecture for solving PerVL that operates by extending the input vocabulary of a pretrained model with new word embeddings for the new personalized concepts.
Ranked #11 on
Zero-Shot Composed Image Retrieval (ZS-CIR)
on CIRCO
Training-Free Consistent Text-to-Image Generation
Text-to-image models offer a new level of creative flexibility by allowing users to guide the image generation process through natural language.
From Generalized zero-shot learning to long-tail with class descriptors
Real-world data is predominantly unbalanced and long-tailed, but deep models struggle to recognize rare classes in the presence of frequent classes.
Ranked #1 on
Long-tail learning with class descriptors
on CUB-LT
A causal view of compositional zero-shot recognition
This leads to consistent misclassification of samples from a new distribution, like new combinations of known components.
ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization
Specifically, given birds' images with free-text descriptions of their species, we learn to classify images of previously-unseen species based on specie descriptions.
Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning
The soft group structure can be learned from data jointly as part of the model, and can also readily incorporate prior knowledge about groups if available.
Learning to generalize to new compositions in image understanding
Recurrent neural networks have recently been used for learning to describe images using natural language.
Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
Specifically, our model consists of three classifiers: A "gating" model that makes soft decisions if a sample is from a "seen" class, and two experts: a ZSL expert, and an expert model for seen classes.
Few-Shot Learning with Per-Sample Rich Supervision
Here we describe a new approach to learn with fewer samples, by using additional information that is provided per sample.
Cooperative image captioning
Second, we show that the generated descriptions can be kept close to natural by constraining them to be similar to human descriptions.
Learning to reason about and to act on physical cascading events
Reasoning and interacting with dynamic environments is a fundamental problem in AI, but it becomes extremely challenging when actions can trigger cascades of cross-dependent events.
Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models
Specifically, we employ two components: First, an encoder that takes as an input a single image of a target concept from a given domain, e. g. a specific face, and learns to map it into a word-embedding representing the concept.
Key-Locked Rank One Editing for Text-to-Image Personalization
The task of T2I personalization poses multiple hard challenges, such as maintaining high visual fidelity while allowing creative control, combining multiple personalized concepts in a single image, and keeping a small model size.
Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models
Text-to-image (T2I) personalization allows users to guide the creative image generation process by combining their own visual concepts in natural language prompts.
