Search Results for author:
Found 130 papers, 81 papers with code
Context-Aware Multimodal Pretraining
Large-scale multimodal representation learning successfully optimizes for zero-shot transfer at test time.
Building, Reusing, and Generalizing Abstract Representations from Concrete Sequences
In contrast, large language models (LLMs) struggle to transfer abstract variables as effectively as humans.
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)
Pre-trained large language models (LLMs) have been reliably integrated with visual input for multimodal tasks.
Scalable Ranked Preference Optimization for Text-to-Image Generation
In this work, we investigate a scalable approach for collecting large-scale and fully synthetic datasets for DPO training.
A Practitioner's Guide to Continual Multimodal Pretraining
In this work, we complement current perspectives on continual pretraining through a research test bed as well as provide comprehensive guidance for effective continual model updates in such scenarios.
Geometry Fidelity for Spherical Images
Spherical or omni-directional images offer an immersive visual format appealing to a wide range of computer vision applications.
EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval
In this work, we introduce EgoCVR, a new evaluation benchmark for fine-grained Composed Video Retrieval using large-scale egocentric video datasets.
DataDream: Few-shot Guided Dataset Generation
While text-to-image diffusion models have been shown to achieve state-of-the-art results in image synthesis, they have yet to prove their effectiveness in downstream applications.
Disentangled Representation Learning with the Gromov-Monge Gap
However, matching the prior while preserving geometric features is challenging, as a mapping that fully preserves these features while aligning the data distribution with the prior does not exist in general.
SemioLLM: Assessing Large Language Models for Semiological Analysis in Epilepsy Research
In summary, our work provides the first extensive benchmark comparing current SOTA LLMs in the medical domain of epilepsy and highlights their ability to leverage unstructured texts from patients' medical history to aid diagnostic processes in health care.
Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models
(2) Indeed, we show how most often, P-RFCL techniques can be matched by a simple and lightweight PEFT baseline.
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization
Moreover, given the same computational resources, a ReNO-optimized one-step model outperforms widely-used open-source models such as SDXL and PixArt-$\alpha$, highlighting the efficiency and effectiveness of ReNO in enhancing T2I model performance at inference time.
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections
Parameter-efficient finetuning (PEFT) has become ubiquitous to adapt foundation models to downstream task requirements while retaining their generalization ability.
Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Models
In this paper, we find that this is noticeably driven by an independent treatment of concepts during intervention, wherein a change of one concept does not influence the use of other ones in the model's final decision.
Audio-Visual Generalized Zero-Shot Learning using Pre-Trained Large Multi-Modal Models
However, existing benchmarks predate the popularization of large multi-modal models, such as CLIP and CLAP.
Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing
In recent years, black-box machine learning approaches have become a dominant modeling paradigm for knowledge extraction in remote sensing.
How should the advent of large language models affect the practice of science?
For this opinion piece, we have invited four diverse groups of scientists to reflect on this query, sharing their perspectives and engaging in debate.
Unbalancedness in Neural Monge Maps Improves Unpaired Domain Translation
We hence establish UOT-FM as a principled method for unpaired image translation.
Zero-shot audio captioning with audio-language model guidance and audio context keywords
In particular, our framework exploits a pre-trained large language model (LLM) for generating the text which is guided by a pre-trained audio-language model to produce captions that describe the audio content.
Ranked #1 on
Zero-shot Audio Captioning
on Clotho
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language
Converting a model's internals to text can yield human-understandable insights about the model.
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model
Training deep networks requires various design decisions regarding for instance their architecture, data augmentation, or optimization.
Vision-by-Language for Training-Free Compositional Image Retrieval
Finally, we show that CIReVL makes CIR human-understandable by composing image and text in a modular fashion in the language domain, thereby making it intervenable, allowing to post-hoc re-align failure cases.
Ranked #6 on
Zero-Shot Composed Image Retrieval (ZS-CIR)
on CIRCO
Video-adverb retrieval with compositional adverb-action embeddings
We propose a framework for video-to-adverb retrieval (and vice versa) that aligns video embeddings with their matching compositional adverb-action text embedding in a joint embedding space.
Text-to-feature diffusion for audio-visual few-shot learning
Training deep learning models for video classification from audio-visual data commonly requires immense amounts of labeled training data collected via a costly process.
PDiscoNet: Semantically consistent part discovery for fine-grained recognition
Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds.
Iterative Superquadric Recomposition of 3D Objects from Multiple Views
We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views without training a model that uses 3D supervision.
DeViL: Decoding Vision features into Language
We demonstrate that DeViL generates textual descriptions relevant to the image content on CC3M surpassing previous lightweight captioning models and attribution maps uncovering the learned concepts of the vision backbone.
Image-free Classifier Injection for Zero-Shot Classification
We achieve this with our proposed Image-free Classifier Injection with Semantics (ICIS) that injects classifiers for new, unseen classes into pre-trained classification models in a post-hoc fashion without relying on image data.
Addressing caveats of neural persistence with deep graph persistence
Whilst this captures useful information for linear classifiers, we find that no relevant spatial structure is present in later layers of deep neural networks, making neural persistence roughly equivalent to the variance of weights.
ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models
We propose ProbVLM, a probabilistic adapter that estimates probability distributions for the embeddings of pre-trained VLMs via inter/intra-modal alignment in a post-hoc manner without needing large-scale datasets or computing.
Waffling around for Performance: Visual Classification with Random Words and Broad Concepts
The visual classification performance of vision-language models such as CLIP has been shown to benefit from additional semantic knowledge from large language models (LLMs) such as GPT-3.
USIM-DAL: Uncertainty-aware Statistical Image Modeling-based Dense Active Learning for Super-resolution
Dense regression is a widely used approach in computer vision for tasks such as image super-resolution, enhancement, depth estimation, etc.
In-Context Impersonation Reveals Large Language Models' Strengths and Biases
These findings demonstrate that LLMs are capable of taking on diverse roles and that this in-context impersonation can be used to uncover their hidden strengths and biases.
If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection
Despite their impressive capabilities, diffusion-based text-to-image (T2I) models can lack faithfulness to the text prompt, where generated images may not contain all the mentioned objects, attributes or relations.
Inducing anxiety in large language models can induce bias
Anxiety-induction not only influences LLMs' scores on an anxiety questionnaire but also influences their behavior in a previously-established benchmark measuring biases such as racism and ageism.
Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval
In this work, we introduce ODmAP@k, an object decorrelation metric that measures a model's robustness to spurious correlations in the training data.
Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification
Based on these findings, we propose to boost the attribution scores of the model trained with partial labels to make its explanation resemble that of the model trained with full labels.
Machine Psychology
Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains.
Likelihood Annealing: Fast Calibrated Uncertainty for Regression
Recent advances in deep learning have shown that uncertainty estimation is becoming increasingly important in applications such as medical imaging, natural language processing, and autonomous systems.
Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation
For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets.
Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment
In this work, we formulate a new knowledge distillation framework to transfer the knowledge from self-supervised pre-trained models to any other student network by a novel approach named Embedding Graph Alignment.
Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning
Large pre-trained, zero-shot capable models have shown considerable success both for standard transfer and adaptation tasks, with particular robustness towards distribution shifts.
PlanT: Explainable Planning Transformers via Object-Level Representations
Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene.
Ranked #6 on
CARLA longest6
on CARLA
Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval
Representation learning for sketch-based image retrieval has mostly been tackled by learning embeddings that discard modality-specific information.
Disentanglement of Correlated Factors via Hausdorff Factorized Support
We show that the use of HFS consistently facilitates disentanglement and recovery of ground-truth factors across a variety of correlation settings and benchmarks, even under severe training correlations and correlation shifts, with in parts over $+60\%$ in relative improvement over existing disentanglement methods.
Relational Proxies: Emergent Relationships as Fine-Grained Discriminators
Fine-grained categories that largely share the same set of parts cannot be discriminated based on part information alone, as they mostly differ in the way the local parts relate to the overall global structure of the object.
Semantic Image Synthesis with Semantically Coupled VQ-Model
Semantic image synthesis enables control over unconditional image generation by allowing guidance on what is being generated.
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
Semi-supervised learning and unsupervised learning offer promising paradigms to learn from an abundance of unlabeled visual data.
Abstracting Sketches through Simple Primitives
Toward equipping machines with such capabilities, we propose the Primitive-based Sketch Abstraction task where the goal is to represent sketches using a fixed set of drawing primitives under the influence of a budget.
Temporal and cross-modal attention for audio-visual zero-shot learning
We show that our proposed framework that ingests temporal features yields state-of-the-art performance on the \ucf, \vgg, and \activity benchmarks for (generalised) zero-shot learning.
Ranked #2 on
GZSL Video Classification
on UCF-GZSL(main)
BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks
Moreover, many of the high-performing deep learning models that are already trained and deployed are non-Bayesian in nature and do not provide uncertainty estimates.
A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning
We model images as directional von Mises-Fisher (vMF) distributions on the hypersphere that can reflect image-intrinsic uncertainties.
The Manifold Hypothesis for Gradient-Based Explanations
We propose a criterion: the feature attributions need to be aligned with the tangent space of the data manifold.
Compositional Mixture Representations for Vision and Text
Learning a common representation space between vision and language allows deep networks to relate objects in the image to the corresponding semantic meaning.
Large Loss Matters in Weakly Supervised Multi-Label Classification
In this work, we first regard unobserved labels as negative labels, casting the WSML task into noisy multi-label classification.
KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning
The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images, given only a subset of them during training and no prior on the unseen compositions.
Attention Consistency on Visual Corruptions for Single-Source Domain Generalization
Generalizing visual recognition models trained on a single distribution to unseen input distributions (i. e. domains) requires making them robust to superfluous correlations in the training set.
Probabilistic Compositional Embeddings for Multimodal Image Retrieval
Without bells and whistles, we show that our probabilistic model formulation significantly outperforms existing related methods on multimodal image retrieval while generalizing well to query with different amounts of inputs given in arbitrary visual and (or) textual modalities.
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
We present baseline results for generating natural language explanations in the context of VQA using two state-of-the-art frameworks on the CLEVR-X dataset.
Ranked #1 on
Explanation Generation
on CLEVR-X
Attribute Prototype Network for Any-Shot Learning
While a visual-semantic embedding layer learns global features, local features are learned through an attribute prototype network that simultaneously regresses and decorrelates attributes from intermediate features.
Ranked #5 on
GZSL Video Classification
on ActivityNet-GZSL(main)
VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learning
Our model visually divides a set of images from seen classes into clusters of local image regions according to their visual similarity, and further imposes their class discrimination and semantic relatedness.
Non-isotropy Regularization for Proxy-based Deep Metric Learning
Deep Metric Learning (DML) aims to learn representation spaces on which semantic relations can simply be expressed through predefined distance metrics.
Ranked #11 on
Metric Learning
on CUB-200-2011
(using extra training data)
Integrating Language Guidance into Vision-based Deep Metric Learning
This causes learned embedding spaces to encode incomplete semantic context and misrepresent the semantic relation between classes, impacting the generalizability of the learned metric space.
Ranked #8 on
Metric Learning
on CARS196
(using extra training data)
Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language
Focusing on the relatively underexplored task of audio-visual zero-shot learning, we propose to learn multi-modal representations from audio-visual data using cross-modal attention and exploit textual label embeddings for transferring knowledge from seen classes to unseen classes.
Ranked #1 on
ZSL Video Classification
on UCF-GZSL (cls)
BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR
The efficacy of zero-shot sketch-based image retrieval (ZS-SBIR) models is governed by two challenges.
Audio Retrieval with Natural Language Queries: A Benchmark Study
Additionally, we introduce the SoundDescs benchmark, which consists of paired audio and natural language descriptions for a diverse collection of sounds that are complementary to those found in AudioCaps and Clotho.
Ranked #1 on
Audio to Text Retrieval
on SoundDescs
Human Attention in Fine-grained Classification
The way humans attend to, process and classify a given image has the potential to vastly benefit the performance of deep learning models.
Ranked #44 on
Fine-Grained Image Classification
on CUB-200-2011
Robustness via Uncertainty-aware Cycle Consistency
Unpaired image-to-image translation refers to learning inter-image-domain mapping without corresponding image pairs.
Conditional De-Identification of 3D Magnetic Resonance Images
Solutions have been developed to de-identify diagnostic scans by obfuscating or removing parts of the face.
Variational Perturbations for Visual Feature Attribution
Explaining a complex black-box system in a post-hoc manner is important to understand its predictions.
Fine-Grained Zero-Shot Learning with DNA as Side Information
Fine-grained zero-shot learning task requires some form of side-information to transfer discriminative information from seen to unseen classes.
Concurrent Discrimination and Alignment for Self-Supervised Feature Learning
Existing self-supervised learning methods learn representation by means of pretext tasks which are either (1) discriminating that explicitly specify which features should be separated or (2) aligning that precisely indicate which features should be closed together, but ignore the fact how to jointly and principally define which features to be repelled and which ones to be attracted.
Uncertainty-Guided Progressive GANs for Medical Image Translation
However, the state-of-the-art GAN-based frameworks do not estimate the uncertainty in the predictions made by the network that is essential for making informed medical decisions and subsequent revision by medical experts and has recently been shown to improve the performance and interpretability of the model.
Keep CALM and Improve Visual Feature Attribution
The class activation mapping, or CAM, has been the cornerstone of feature attribution methods for multiple vision tasks.
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
e-ViL is a benchmark for explainable vision-language tasks that establishes a unified evaluation framework and provides the first comprehensive comparison of existing approaches that generate NLEs for VL tasks.
Audio Retrieval with Natural Language Queries
We consider the task of retrieving audio using free-form natural language queries.
Ranked #1 on
Audio/Video to Text Retrieval
on AudioCaps
Where and When: Space-Time Attention for Audio-Visual Explanations
Recent advances in XAI provide explanations for models trained on still images.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions.
Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
Having access to multi-modal cues (e. g. vision and audio) empowers some cognitive tasks to be done faster compared to learning from a single modality.
A Closer Look at Self-training for Zero-Label Semantic Segmentation
Being able to segment unseen classes not observed during training is an important technical challenge in deep learning, because of its potential to reduce the expensive annotation required for semantic segmentation.
Ranked #8 on
Zero-Shot Semantic Segmentation
on PASCAL VOC
Uncertainty-aware Generalized Adaptive CycleGAN
Unpaired image-to-image translation refers to learning inter-image-domain mapping in an unsupervised manner.
Learning Graph Embeddings for Compositional Zero-shot Learning
In compositional zero-shot learning, the goal is to recognize unseen compositions (e. g. old dog) of observed visual primitives states (e. g. old, cute) and objects (e. g. car, dog) in the training set.
Open World Compositional Zero-Shot Learning
After estimating the feasibility score of each composition, we use these scores to either directly mask the output space or as a margin for the cosine similarity between visual features and compositional embeddings during training.
Adversarial Privacy Preservation in MRI Scans of the Brain
De-identification of magnetic resonance imagery (MRI) is intrinsically difficult since, even with all metadata removed, a person's face can easily be rendered and matched against a database.
Prototype-based Incremental Few-Shot Semantic Segmentation
Semantic segmentation models have two fundamental weaknesses: i) they require large training sets with costly pixel-level annotations, and ii) they have a static output space, constrained to the classes of the training set.
Attribute Prototype Network for Zero-Shot Learning
As an additional benefit, our model points to the visual evidence of the attributes in an image, e. g. for the CUB dataset, confirming the improved attribute localization ability of our image representation.
Towards Recognizing Unseen Categories in Unseen Domains
The key idea of CuMix is to simulate the test-time domain and semantic shift using images and features from unseen domains and categories generated by mixing up the multiple source domains and categories available during training.
Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
Few-shot learning aims to recognize novel classes from a few examples.
Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets
In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.
Semantically Tied Paired Cycle Consistency for Any-Shot Sketch-based Image Retrieval
Low-shot sketch-based image retrieval is an emerging task in computer vision, allowing to retrieve natural images relevant to hand-drawn sketch queries that are rarely seen during the training phase.
Driver Intention Anticipation Based on In-Cabin and Driving SceneMonitoring
Numerous car accidents are caused by improper driving maneuvers.
e-SNLI-VE: Corrected Visual-Textual Entailment with Natural Language Explanations
The recently proposed SNLI-VE corpus for recognising visual-textual entailment is a large, real-world dataset for fine-grained multimodal reasoning.
Learning Robust Representations via Multi-View Information Bottleneck
This enables us to identify superfluous information as that not shared by both views.
Evaluating Weakly Supervised Object Localization Methods Right
In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.
Understanding Misclassifications by Attributes
We study the changes in attributes for clean as well as adversarial images in both standard and adversarially robust networks.
Modeling Conceptual Understanding in Image Reference Games
An agent who interacts with a wide population of other agents needs to be aware that there may be variations in their understanding of the world.
Relational Generalized Few-Shot Learning
Instead, we consider the extended setup of generalized few-shot learning (GFSL), where the model is required to perform classification on the joint label space consisting of both previously seen and novel classes.
Bayesian Zero-Shot Learning
Object classes that surround us have a natural tendency to emerge at varying levels of abstraction.
Combining Generative and Discriminative Models for Hybrid Inference
A graphical model is a structured representation of the data generating process.
Interpreting Adversarial Examples with Attributes
Deep computer vision systems being vulnerable to imperceptible and carefully crafted noise have raised questions regarding the robustness of their decisions.
f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
When labeled training data is scarce, a promising data augmentation approach is to generate visual features of unknown classes using their attributes.
Ranked #4 on
Generalized Zero-Shot Learning
on SUN Attribute
Cross-Linked Variational Autoencoders for Generalized Zero-Shot Learning
While following the same direction, we also take artificial feature generation one step further and propose a model where a shared latent space of image features and class embeddings is learned by aligned variational autoencoders, for the purpose of generating latent features to train a softmax classifier.
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval
Existing works either require aligned sketch-image pairs or inefficient memory fusion layer for mapping the visual information to a semantic space.
Learning Decision Trees Recurrently Through Communication
Integrated interpretability without sacrificing the prediction accuracy of decision making algorithms has the potential of greatly improving their value to the user.
Visual Rationalizations in Deep Reinforcement Learning for Atari Games
Due to the capability of deep learning to perform well in high dimensional problems, deep reinforcement learning agents perform well in challenging tasks such as Atari 2600 games.
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders
Many approaches in generalized zero-shot learning rely on cross-modal mapping between the image feature space and the class embedding space.
Ranked #2 on
Generalized Few-Shot Learning
on AwA2
Manipulating Attributes of Natural Scenes via Hallucination
In this study, we explore building a two-stage framework for enabling users to directly manipulate high-level attributes of a natural scene.
Textual Explanations for Self-Driving Vehicles
Finally, we explore a version of our model that generates rationalizations, and compare with introspective explanations on the same video segments.
Grounding Visual Explanations
Our model improves the textual explanation quality of fine-grained classification decisions on the CUB dataset by mentioning phrases that are grounded in the image.
Generating Counterfactual Explanations with Natural Language
We call such textual explanations counterfactual explanations, and propose an intuitive method to generate counterfactual explanations by inspecting which evidence in an input is missing, but might contribute to a different classification decision if present in the image.
Primal-Dual Wasserstein GAN
We introduce Primal-Dual Wasserstein GAN, a new learning algorithm for building latent variable models of the data distribution based on the primal and the dual formulations of the optimal transport (OT) problem.
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
We propose a multimodal approach to explanation, and argue that the two modalities provide complementary explanatory strengths.
Feature Generating Networks for Zero-Shot Learning
Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task.
Ranked #6 on
Generalized Zero-Shot Learning
on SUN Attribute
Generalized Zero-Shot Learning
Generative Adversarial Network
Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
We also introduce a multimodal methodology for generating visual and textual explanations simultaneously.
Grounding Visual Explanations (Extended Abstract)
Existing models which generate textual explanations enforce task relevance through a discriminative term loss function, but such mechanisms only weakly constrain mentioned object parts to actually be present in the image.
Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
Due to the importance of zero-shot learning, i. e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily.
Zero-Shot Learning -- The Good, the Bad and the Ugly
Due to the importance of zero-shot learning, the number of proposed approaches has increased steadily recently.
Exploiting saliency for object segmentation from image level labels
We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.
Ranked #26 on
Semantic Segmentation
on PASCAL VOC 2012 val
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which led to their decisions.
Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
Automatic image synthesis research has been rapidly growing with deep networks getting more and more expressive.
Gaze Embeddings for Zero-Shot Image Classification
Zero-shot image classification using auxiliary information, such as attributes describing discriminative object properties, requires time-consuming annotation by domain experts.
Learning What and Where to Draw
Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers.
Ranked #14 on
Text-to-Image Generation
on CUB
(using extra training data)
Learning Deep Representations of Fine-grained Visual Descriptions
State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information.
Generative Adversarial Text to Image Synthesis
Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal.
Latent Embeddings for Zero-shot Classification
We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image.
Generating Visual Explanations
Clearly explaining a rationale for a classification decision to an end-user can be as important as the decision itself.
Label-Embedding for Image Classification
Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce.
Ranked #7 on
Multi-label zero-shot learning
on Open Images V4
Evaluation of Output Embeddings for Fine-Grained Image Classification
Image classification has advanced significantly in recent years with the availability of large-scale image sets.
Label-Embedding for Attribute-Based Classification
The label embedding framework offers other advantages such as the ability to leverage alternative sources of information in addition to attributes (e. g. class hierarchies) or to transition smoothly from zero-shot learning to learning with large quantities of data.
