Search Results for author:
Found 33 papers, 20 papers with code
ReNoise: Real Image Inversion Through Iterative Noising
However, applying these methods to real images necessitates the inversion of the images into the domain of the pretrained diffusion model.
ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation
Web-scale training on paired text-image data is becoming increasingly central to multimodal learning, but is challenged by the highly noisy nature of datasets in the wild.
Mitigating Open-Vocabulary Caption Hallucinations
While recent years have seen rapid progress in image-conditioned text generation, image captioning still suffers from the fundamental issue of hallucinations, namely, the generation of spurious details that cannot be inferred from the given image.
SPiC-E : Structural Priors in 3D Diffusion Models using Cross-Entity Attention
We are witnessing rapid progress in automatically generating and manipulating 3D assets due to the availability of pretrained text-image diffusion models.
Cross-Image Attention for Zero-Shot Appearance Transfer
Recent advancements in text-to-image generative models have demonstrated a remarkable ability to capture a deep semantic understanding of images.
Kiki or Bouba? Sound Symbolism in Vision-and-Language Models
Although the mapping between sound and meaning in human language is assumed to be largely arbitrary, research in cognitive science has shown that there are non-trivial correlations between particular sounds and meanings across languages and demographic groups, a phenomenon known as sound symbolism.
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models
Key to tasks that require reasoning about natural language in visual contexts is grounding words and phrases to image regions.
Doppelgangers: Learning to Disambiguate Images of Similar Structures
Our evaluation shows that our method can distinguish illusory matches in difficult cases, and can be integrated into SfM pipelines to produce correct, disambiguated 3D reconstructions.
Neural Scene Chronology
Specifically, we represent the scene as a space-time radiance field with a per-image illumination embedding, where temporally-varying scene changes are encoded using a set of learned step functions.
Learning Human-Human Interactions in Images from Weak Textual Supervision
We show that the pseudo-labels produced by this procedure can be used to train a captioning model to effectively understand human-human interactions in images, as measured by a variety of metrics that measure textual and semantic faithfulness and factual groundedness of our predictions.
Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding
We show that SOTA multimodally trained text encoders outperform unimodally trained text encoders on the VLU tasks while being underperformed by them on the NLU tasks, lending new context to previously mixed results regarding the NLU capabilities of multimodal models.
Vox-E: Text-guided Voxel Editing of 3D Objects
Our method takes oriented 2D images of a 3D object as input and learns a grid-based volumetric representation of it.
What's in a Decade? Transforming Faces Through Time
How can one visually characterize people in a decade?
Neural 3D Reconstruction in the Wild
We are witnessing an explosion of neural implicit representations in computer vision and graphics.
To SMOTE, or not to SMOTE?
Balancing the data before training a classifier is a popular technique to address the challenges of imbalanced binary classification in tabular data.
Who's Waldo? Linking People Across Text and Images
We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.
Ranked #1 on
Person-centric Visual Grounding
on Who’s Waldo
(using extra training data)
Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision
The abundance and richness of Internet photos of landmarks and cities has led to significant progress in 3D vision over the past two decades, including automated 3D reconstructions of the world's landmarks from tourist photos.
Extreme Rotation Estimation using Dense Correlation Volumes
We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting, where the images have little or no overlap.
Learning Multimodal Affinities for Textual Editing in Images
Nowadays, as cameras are rapidly adopted in our daily routine, images of documents are becoming both abundant and prevalent.
An Ethical Highlighter for People-Centric Dataset Creation
Important ethical concerns arising from computer vision datasets of people have been receiving significant attention, and a number of datasets have been withdrawn as a result.
Hidden Footprints: Learning Contextual Walkability from 3D Human Trails
Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis.
Learning Gradient Fields for Shape Generation
Point cloud generation thus amounts to moving randomly sampled points to high-density areas.
Co-occurrence Based Texture Synthesis
As image generation techniques mature, there is a growing interest in explainable representations that are easy to understand and intuitive to manipulate.
What is Learned in Visually Grounded Neural Syntax Acquisition
Visual features are a promising signal for learning bootstrap textual models.
DualSDF: Semantic Shape Manipulation using a Two-Level Representation
We are seeing a Cambrian explosion of 3D shape representations for use in machine learning.
ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation
This is especially true for handwritten text recognition (HTR), where each author has a unique style, unlike printed text, where the variation is smaller by design.
READ: Recursive Autoencoders for Document Layout Generation
Creating large varieties of plausible document layouts can be a tedious task, requiring numerous constraints to be satisfied, including local ones relating different semantic elements and global constraints on the general appearance and spacing.
Implicit Pairs for Boosting Unpaired Image-to-Image Translation
As a result, in recent years more attention has been given to techniques that learn the mapping from unpaired sets.
Clustering-driven Deep Embedding with Pairwise Constraints
In this paper, we depart from centroid-based models and suggest a new framework, called Clustering-driven deep embedding with PAirwise Constraints (CPAC), for non-parametric clustering using a neural network.
Co-segmentation for Space-Time Co-located Collections
Thus, to disambiguate what the common foreground object is, we introduce a weakly-supervised technique, where we assume only a small seed, given in the form of a single segmented image.
Spherical Embedding of Inlier Silhouette Dissimilarities
In this paper, we introduce a spherical embedding technique to position a given set of silhouettes of an object as observed from a set of cameras arbitrarily positioned around the object.
