Search Results for author:
Found 60 papers, 28 papers with code
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.
It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collection
As a step in this direction, the ArtEmis dataset was recently introduced as a large-scale dataset of emotional reactions to images along with language explanations of these chosen emotions.
Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimation
AMD is a metric that quantifies how close the whole generated samples are to the ground truth.
Ranked #1 on
Trajectory Prediction
on ETH
Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classification
The main question we address in this paper is how to scale up visual recognition of unseen classes, also known as zero-shot learning, to tens of thousands of categories as in the ImageNet-21K benchmark.
Efficiently Disentangle Causal Representations
This paper proposes an efficient approach to learning disentangled representations with causal mechanisms based on the difference of conditional probabilities in original and new distributions.
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
We build our model on top of StyleGAN2 and it is just ${\approx}5\%$ more expensive to train at the same resolution while achieving almost the same image quality.
Domain-Aware Continual Zero-Shot Learning
To encourage the private network to capture the domain and task-specific representation, we train our model with a novel adversarial knowledge disentanglement setting to make our global network task-invariant and domain-invariant over all the tasks.
CausalDyna: Improving Generalization of Dyna-style Reinforcement Learning via Counterfactual-Based Data Augmentation
Deep reinforcement learning agents trained in real-world environments with a limited diversity of object properties to learn manipulation tasks tend to suffer overfitting and fail to generalize to unseen testing environments.
HyperCGAN: Text-to-Image Synthesis with HyperNet-Modulated Conditional Generative Adversarial Networks
To the best of our knowledge, our work is the first one which explores text-controllable continuous image generation.
RelTransformer: A Transformer-Based Long-Tail Visual Relationship Recognition
This paper shows that modeling an effective message-passing flow through an attention mechanism can be critical to tackling the compositionality and long-tail challenges in VRR.
Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
By generating representations of unseen classes based on their semantic descriptions, e. g., attributes or text, generative ZSL attempts to differentiate unseen from seen categories.
Aligning Latent and Image Spaces to Connect the Unconnectable
In this work, we develop a method to generate infinite high-resolution images with diverse and complex content.
Ranked #1 on
Infinite Image Generation
on LHQ
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning
To the best of our knowledge, this is the first work that improves data efficiency of image captioning by utilizing LM pretrained on unimodal data.
ArtEmis: Affective Language for Visual Art
We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.
Transferability of Compositionality
To our best knowledge, this is the first work to focus on the transferability of compositionality, and it is orthogonal to existing efforts of learning compositional representations in training distribution.
Gradient Descent Resists Compositionality
In this paper, we argue that gradient descent is one of the reasons that make compositionality learning hard during neural network optimization.
Class Normalization for Zero-Shot Learning
Normalization techniques have proved to be a crucial ingredient of successful training in a traditional supervised learning regime.
Motion Forecasting with Unlikelihood Training
We propose a new objective, unlikelihood training, which forces generated trajectories that conflicts with contextual information to be assigned a lower probability by our model.
HalentNet: Multimodal Trajectory Forecasting with Hallucinative Intents
Our model's learned representation leads to better and more semantically meaningful coverage of the trajectory distribution.
CIZSL++: Creativity Inspired Generative Zero-Shot Learning
To improve the discriminative power of ZSL, we model the visual learning process of unseen categories with inspiration from the psychology of human creativity for producing novel art.
Adversarial Generation of Continuous Images
In most existing learning systems, images are typically viewed as 2D pixel arrays.
Ranked #7 on
Image Generation
on FFHQ 256 x 256
Temporal Positive-unlabeled Learning for Biomedical Hypothesis Generation via Risk Estimation
Then, the key is to capture the temporal evolution of node pair (term pair) relations from just the positive and unlabeled data.
Class Normalization for (Continual)? Generalized Zero-Shot Learning
Normalization techniques have proved to be a crucial ingredient of successful training in a traditional supervised learning regime.
Inner Ensemble Networks: Average Ensemble as an Effective Regularizer
Also, we show empirically and theoretically that IENs lead to a greater variance reduction in comparison with other similar approaches such as dropout and maxout.
Compositional Language Continual Learning
It also shows significant improvement in machine translation task.
Efficient long-distance relation extraction with DG-SpanBERT
In natural language processing, relation extraction seeks to rationally understand unstructured text.
Ranked #11 on
Relation Extraction
on TACRED
Exploring Long Tail Visual Relationship Recognition with Large Vocabulary
We use these benchmarks to study the performance of several state-of-the-art long-tail models on the LTVRR setup.
Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction
Better machine understanding of pedestrian behaviors enables faster progress in modeling interactions between agents such as autonomous vehicles and humans.
Ranked #3 on
Trajectory Prediction
on ETH
Uncertainty-guided Continual Learning with Bayesian Neural Networks
Continual learning aims to learn new tasks without forgetting previously learned ones.
Learning Diverse Generations using Determinantal Point Processes
Embedded in an adversarial training and variational autoencoder, our Generative DPP approach shows a consistent resistance to mode-collapse on a wide-variety of synthetic data and natural image datasets including MNIST, CIFAR10, and CelebA, while outperforming state-of-the-art methods for data-efficiency, convergence-time, and generation quality.
Creativity Inspired Zero-Shot Learning
We relate ZSL to human creativity by observing that zero-shot learning is about recognizing the unseen and creativity is about creating a likable unseen.
Semi-Supervised Few-Shot Learning with Prototypical Random Walks
We develop a random walk semi-supervised loss that enables the network to learn representations that are compact and well-separated.
On Tiny Episodic Memories in Continual Learning
But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks.
Exploring the Challenges towards Lifelong Fact Learning
So far life-long learning (LLL) has been studied in relatively small-scale and relatively artificial setups.
Efficient Lifelong Learning with A-GEM
In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task.
Ranked #6 on
Continual Learning
on ASC (19 tasks)
GDPP: Learning Diverse Generations Using Determinantal Point Process
Generative models have proven to be an outstanding tool for representing high-dimensional probability distributions and generating realistic-looking images.
Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting
A human teacher can show potential objects of interest to the robot, which is able to self adapt to the teaching signal without providing manual segmentation labels.
Ranked #13 on
Unsupervised Video Object Segmentation
on DAVIS 2016
Uncertainty-guided Lifelong Learning in Bayesian Networks
Sequentially learning of tasks arriving in a continuous stream is a complex problem and becomes more challenging when the model has a fixed capacity.
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
Our approach, which we call Neuron Importance-AwareWeight Transfer (NIWT), learns to map domain knowledge about novel "unseen" classes onto this dictionary of learned concepts and then optimizes for network parameters that can effectively combine these concepts - essentially learning classifiers by discovering and composing learned semantic concepts in deep networks.
Large-Scale Visual Relationship Understanding
Large scale visual understanding is challenging, as it requires a model to handle the widely-spread and imbalanced distribution of <subject, relation, object> triples.
DeSIGN: Design Inspiration from Generative Networks
Can an algorithm create original and compelling fashion designs to serve as an inspirational assistant?
A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts
Most existing zero-shot learning methods consider the problem as a visual semantic embedding one.
Memory Aware Synapses: Learning what (not) to forget
We show state-of-the-art performance and, for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.
Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precision
We propose a learning framework that is able to connect text terms to its relevant parts and suppress connections to non-visual text terms without any part-text annotations.
Relationship Proposal Networks
We demonstrate the ability of our Rel-PN to localize relationships with only a few thousand proposals.
CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms
We argue that such networks are limited in their ability to generate creative products in their original design.
Overlapping Cover Local Regression Machines
We present the Overlapping Domain Cover (ODC) notion for kernel machines, as a set of overlapping subsets of the data that covers the entire training set and optimized to be spatially cohesive as possible.
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition
In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDA-CNN) for fine-grained classification.
Automatic Annotation of Structured Facts in Images
Motivated by the application of fact-level image understanding, we present an automatic method for data collection of structured visual facts from images with captions.
Write a Classifier: Predicting Visual Classifiers from Unstructured Text
Then, we propose a new constrained optimization formulation that combines a regression function and a knowledge transfer function with additional constraints to predict the parameters of a linear classifier.
Zero-Shot Event Detection by Multimodal Distributional Semantic Embedding of Videos
To our knowledge, this is the first Zero-Shot event detection model that is built on top of distributional semantics and extends it in the following directions: (a) semantic embedding of multimodal information in videos (with focus on the visual modalities), (b) automatically determining relevance of concepts/attributes to a free text query, which could be useful for other applications, and (c) retrieving videos by free text event query (e. g., "changing a vehicle tire") based on their content.
Convolutional Models for Joint Object Categorization and Pose Estimation
In the task of Object Recognition, there exists a dichotomy between the categorization of objects and estimating object pose, where the former necessitates a view-invariant representation, while the latter requires a representation capable of capturing pose information over different categories of objects.
Sherlock: Scalable Fact Learning in Images
We show that learning visual facts in a structured way enables not only a uniform but also generalizable visual understanding.
Digging Deep into the layers of CNNs: In Search of How CNNs Achieve View Invariance
How does fine-tuning of a pre-trained CNN on a multi-view dataset affect the representation at each layer of the network?
Tell and Predict: Kernel Classifier Prediction for Unseen Visual Classes from Unstructured Text Descriptions
In this paper we propose a framework for predicting kernelized classifiers in the visual domain for categories with no training images where the knowledge comes from textual description about these categories.
Learning Hypergraph-regularized Attribute Predictors
Then the attribute prediction problem is casted as a regularized hypergraph cut problem in which HAP jointly learns a collection of attribute projections from the feature space to a hypergraph embedding space aligned with the attribute space.
Generalized Twin Gaussian Processes using Sharma-Mittal Divergence
In this paper, we present a generalized structured regression framework based on Shama-Mittal divergence, a relative entropy measure, which is introduced to the Machine Learning community in this work.
Text to Multi-level MindMaps: A Novel Method for Hierarchical Visual Abstraction of Natural Language Text
This work firstly introduces MindMap Multilevel Visualization concept which is to jointly visualize and summarize textual information.
