Search Results for author:
Found 75 papers, 38 papers with code
Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models
Text-to-image (T2I) personalization allows users to guide the creative image generation process by combining their own visual concepts in natural language prompts.
Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
Point-cloud data collected in real-world applications are often incomplete.
Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment
This reflects an impaired mapping between linguistic binding of entities and modifiers in the prompt and visual binding of the corresponding elements in the generated image.
Norm-guided latent space exploration for text-to-image generation
We describe a simple yet efficient algorithm for approximating this metric and use it to further define centroids in the latent seed space.
DisCLIP: Open-Vocabulary Referring Expression Generation
Our results highlight the potential of using pre-trained visual-semantic models for generating high-quality contextual descriptions.
CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters.
Key-Locked Rank One Editing for Text-to-Image Personalization
The task of T2I personalization poses multiple hard challenges, such as maintaining high visual fidelity while allowing creative control, combining multiple personalized concepts in a single image, and keeping a small model size.
It is all about where you start: Text-to-image generation with seed selection
We further evaluate SeedSelect on correcting images of hands, a well-known pitfall of current diffusion models, and show that it improves hand generation substantially.
Graph Positional Encoding via Random Feature Propagation
Two main families of node feature augmentation schemes have been explored for enhancing GNNs: random features and spectral positional encoding.
Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models
Specifically, we employ two components: First, an encoder that takes as an input a single image of a target concept from a given domain, e. g. a specific face, and learns to map it into a word-embedding representing the concept.
Guided Deep Kernel Learning
Combining Gaussian processes with the expressive power of deep neural networks is commonly done nowadays through deep kernel learning (DKL).
Auxiliary Learning as an Asymmetric Bargaining Game
Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets.
SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search
We prove that the resulting variance decays exponentially with the planning horizon as a function of the expansion policy.
Equivariant Architectures for Learning in Deep Weight Spaces
Designing machine learning architectures for processing neural networks in their raw weight matrix form is a newly introduced research direction.
Train Hard, Fight Easy: Robust Meta Reinforcement Learning
We prove that the former disappears in MRL, and address the latter via the novel Robust Meta RL algorithm (RoML).
Text2Model: Model Induction for Zero-shot Generalization Using Task Descriptions
We study the problem of generating a training-free task-dependent visual classifier from text descriptions without visual samples.
SoftTreeMax: Policy Gradient with Tree Search
This allows us to reduce the variance of gradients by three orders of magnitude and to benefit from better sample complexity compared with standard policy gradient.
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes.
Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs
As communication protocols evolve, datacenter network utilization increases.
Reinforcement Learning with a Terminator
We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds.
Optimizing Tensor Network Contraction Using Reinforcement Learning
Quantum Computing (QC) stands to revolutionize computing, but is currently still limited.
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations
We propose an architecture for solving PerVL that operates by extending the input vocabulary of a pretrained model with new word embeddings for the new personalized concepts.
Ranked #4 on
Zero-Shot Composed Image Retrieval (ZS-CIR)
on CIRCO
Example-based Hypernetworks for Out-of-Distribution Generalization
While Natural Language Processing (NLP) algorithms keep reaching unprecedented milestones, out-of-distribution generalization is still challenging.
Multi-Task Learning as a Bargaining Game
In this paper, we propose viewing the gradients combination step as a bargaining game, where tasks negotiate to reach an agreement on a joint direction of parameter update.
Ranked #1 on
Multi-Task Learning
on QM9
Learning to reason about and to act on physical cascading events
Reasoning and interacting with dynamic environments is a fundamental problem in AI, but it becomes extremely challenging when actions can trigger cascades of cross-dependent events.
Planning and Learning with Adaptive Lookahead
Some of the most powerful reinforcement learning frameworks use planning for action selection.
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks
Standard Federated Learning (FL) techniques are limited to clients with identical network architectures.
On-Demand Unlabeled Personalized Federated Learning
In both FL and PFL, all clients participate in the training process and their labeled data are used for training.
On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning
We analyze the limitations of learning from such data with and without external reward, and propose an adjustment of standard imitation learning algorithms to fit this setup.
Object-Region Video Transformers
In this work, we present Object-Region Video Transformers (ORViT), an \emph{object-centric} approach that extends video transformer layers with a block that directly incorporates object representations.
Ranked #3 on
Action Recognition
on Diving-48
Inference-Time Personalized Federated Learning
That client representation is fed to a hypernetwork that generates a personalized model for that client.
StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators
Can a generative model be trained to produce images from a specific domain, guided by a text prompt only, without seeing any image?
Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction
We first discover and analyze a counter-intuitive phenomenon: action selection through TS and a pre-trained value function often leads to lower performance compared to the original pre-trained agent, even when having access to the exact state and reward in future steps.
Personalized Federated Learning with Gaussian Processes
A key challenge in this setting is to learn effectively across clients even though each client has unique data that is often limited in size.
Ranked #1 on
Personalized Federated Learning
on CIFAR-100
DETReg: Unsupervised Pretraining with Region Priors for Object Detection
Recent self-supervised pretraining methods for object detection largely focus on pretraining the backbone of the object detector, neglecting key parts of detection architecture.
Ranked #1 on
Few-Shot Object Detection
on COCO 2017
Distributional Robustness Loss for Long-tail Learning
The new robustness loss can be combined with various classifier balancing techniques and can be applied to representations at several layers of the deep model.
Ranked #15 on
Long-tail Learning
on CIFAR-100-LT (ρ=10)
Personalized Federated Learning using Hypernetworks
In this approach, a central hypernetwork model is trained to generate a set of models, one model for each client.
Ranked #1 on
Personalized Federated Learning
on CIFAR-10
Reinforcement Learning for Datacenter Congestion Control
We approach the task of network congestion control in datacenters using Reinforcement Learning (RL).
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
As a result, our method scales well with both the number of classes and data size.
ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning
We demonstrate that our approach finds videos with high audio-visual correspondence and show that self-supervised models trained on our data achieve competitive performances compared to models trained on existing manually curated datasets.
Teacher-Student Consistency For Multi-Source Domain Adaptation
Then, we train a student network using the pseudo labels and regularized the teacher to fit the student predictions.
From Local Structures to Size Generalization in Graph Neural Networks
In this paper, we identify an important type of data where generalization from small to large graphs is challenging: graph distributions for which the local structure depends on the graph size.
Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks
We consider the problem of controlling a partially-observed dynamic process on a graph by a limited number of interventions.
Learning the Pareto Front with Hypernetworks
Here, we tackle the problem of learning the entire Pareto front, with the capability of selecting a desired operating point on the front after training.
ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization
Specifically, given birds' images with free-text descriptions of their species, we learn to classify images of previously-unseen species based on specie descriptions.
Learning Object Detection from Captions via Textual Scene Attributes
In this work, we argue that captions contain much richer information about the image, including attributes of objects and their relations.
On Size Generalization in Graph Neural Networks
We further demonstrate on several tasks, that training GNNs on small graphs results in solutions which do not generalize to larger graphs.
Compositional Video Synthesis with Action Graphs
Our generative model for this task (AG2Vid) disentangles motion and appearance features, and by incorporating a scheduling mechanism for actions facilitates a timely and coordinated video generation.
A causal view of compositional zero-shot recognition
This leads to consistent misclassification of samples from a new distribution, like new combinations of known components.
Auxiliary Learning by Implicit Differentiation
Two main challenges arise in this multi-task learning setting: (i) designing useful auxiliary tasks; and (ii) combining auxiliary tasks into a single coherent loss.
Contrastive Learning for Weakly Supervised Phrase Grounding
Given pairs of images and captions, we maximize compatibility of the attention-weighted regions and the words in the corresponding caption, compared to non-corresponding pairs of images and captions.
From Generalized zero-shot learning to long-tail with class descriptors
Real-world data is predominantly unbalanced and long-tailed, but deep models struggle to recognize rare classes in the presence of frequent classes.
Ranked #1 on
Long-tail learning with class descriptors
on CUB-LT
Self-Supervised Learning for Domain Adaptation on Point-Clouds
Self-supervised learning (SSL) is a technique for learning useful representations from unlabeled data.
Learning Object Permanence from Video
The fourth subtask, where a target object is carried by a containing object, is particularly challenging because it requires a system to reason about a moving location of an invisible object.
Ranked #3 on
Video Object Tracking
on CATER
On Learning Sets of Symmetric Elements
We first characterize the space of linear layers that are equivariant both to element reordering and to the inherent symmetries of elements, like translation in the case of images.
Learning Canonical Representations for Scene Graph to Image Generation
Generating realistic images of complex visual scenes becomes challenging when one wishes to control the structure of the generated images.
Ranked #3 on
Layout-to-Image Generation
on Visual Genome 256x256
A Multi-Pairwise Extension of Procrustes Analysis for Multilingual Word Translation
In this paper we present a novel approach to simultaneously representing multiple languages in a common space.
Multilingual word translation using auxiliary languages
For each source word, we first search for the most relevant auxiliary languages.
Cooperative image captioning
Second, we show that the generated descriptions can be kept close to natural by constraining them to be similar to human descriptions.
Few-Shot Learning with Per-Sample Rich Supervision
Here we describe a new approach to learn with fewer samples, by using additional information that is provided per sample.
Differentiable Scene Graphs
In many domains, it is preferable to train systems jointly in an end-to-end manner, but SGs are not commonly used as intermediate components in visual reasoning systems because being discrete and sparse, scene-graph representations are non-differentiable and difficult to optimize.
Informative Object Annotations: Tell Me Something I Don't Know
Capturing the interesting components of an image is a key aspect of image understanding.
Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
Specifically, our model consists of three classifiers: A "gating" model that makes soft decisions if a sample is from a "seen" class, and two experts: a ZSL expert, and an expert model for seen classes.
Metric Learning for Phoneme Perception
This study presents a new framework for learning a metric function for perceptual distances among pairs of phonemes.
Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning
The soft group structure can be learned from data jointly as part of the model, and can also readily incorporate prior knowledge about groups if available.
Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
Machine understanding of complex images is a key goal of artificial intelligence.
DeepBrain: Functional Representation of Neural In-Situ Hybridization Images for Gene Ontology Classification Using Deep Convolutional Autoencoders
This paper presents a novel deep learning-based method for learning a functional representation of mammalian neural images.
Context-aware Captions from Context-agnostic Supervision
We introduce an inference technique to produce discriminative context-aware image captions (captions that describe differences between images or visual concepts) using only generic context-agnostic training data (captions that describe a concept or an image in isolation).
Learning From Noisy Large-Scale Datasets With Minimal Supervision
For the small clean set of annotations we use a quarter of the validation set with ~40k images.
Learning to generalize to new compositions in image understanding
Recurrent neural networks have recently been used for learning to describe images using natural language.
Gradual Training Method for Denoising Auto Encoders
Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network.
Gradual training of deep denoising auto encoders
Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network.
Efficient coordinate-descent for orthogonal matrices through Givens rotations
Optimizing over the set of orthogonal matrices is a central component in problems like sparse-PCA or tensor decomposition.
Online Learning in The Manifold of Low-Rank Matrices
When learning models that are represented in matrix forms, enforcing a low-rank constraint can dramatically improve the memory and run time complexity, while providing a natural regularization of the model.
An Online Algorithm for Large Scale Image Similarity Learning
We describe OASIS, a method for learning pairwise similarity that is fast and scales linearly with the number of objects and the number of non-zero features.
