Search Results for author:
Found 14 papers, 5 papers with code
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations.
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
We propose a multimodal approach to explanation, and argue that the two modalities provide complementary explanatory strengths.
Robust Change Captioning
We present a novel Dual Dynamic Attention Model (DUDA) to perform robust Change Captioning.
Discovering Autoregressive Orderings with Variational Inference
One strategy to recover this information is to decode both the content and location of tokens.
Discovering Non-monotonic Autoregressive Orderings with Variational Inference
Permutations then serve as target generation orders for training an insertion-based Transformer language model.
Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
We also introduce a multimodal methodology for generating visual and textual explanations simultaneously.
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which led to their decisions.
Learning a Unified Embedding for Visual Search at Pinterest
The solution we present not only allows us to train for multiple application objectives in a single deep neural network architecture, but takes advantage of correlated information in the combination of all training data from each application to generate a unified embedding that outperforms all specialized embeddings previously deployed for each product.
Novelty Detection with Rotated Contrastive Predictive Coding
To this end, reconstruction-based learning is often used in which the normality of an observation is expressed in how well it can be reconstructed.
Toward Transformer-Based Object Detection
The Vision Transformer was the first major attempt to apply a pure transformer model directly to images as input, demonstrating that as compared to convolutional networks, transformer-based architectures can achieve competitive results on benchmark classification tasks.
Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations
Large-scale pretraining of visual representations has led to state-of-the-art performance on a range of benchmark computer vision tasks, yet the benefits of these techniques at extreme scale in complex production systems has been relatively unexplored.
Ranked #26 on
Image Classification
on ObjectNet
(using extra training data)
More Control for Free! Image Synthesis with Semantic Diffusion Guidance
We investigate fine-grained, continuous control of this model class, and introduce a novel unified framework for semantic diffusion guidance, which allows either language or image guidance, or both.
Shape-Guided Diffusion with Inside-Outside Attention
We introduce precise object silhouette as a new form of user control in text-to-image diffusion models, which we dub Shape-Guided Diffusion.
Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
We propose Diffusion Hyperfeatures, a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors that can be used for downstream tasks.
