Search Results for author:
Found 38 papers, 26 papers with code
Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control
Dynamical generative models that produce samples through an iterative process, such as Flow Matching and denoising diffusion models, have seen widespread use, but there has not been many theoretically-sound methods for improving these models with reward fine-tuning.
Consistency-diversity-realism Pareto fronts of conditional image generative models
Building world models that accurately and comprehensively represent the real world is the utmost aspiration for conditional image generative models as it would enable their use as world simulators.
Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance
With the growing popularity of text-to-image generative models, there has been increasing focus on understanding their risks and biases.
Towards Geographic Inclusion in the Evaluation of Text-to-Image Models
We contrast human annotations with common automated metrics, finding that human preferences vary notably across geographic location and that current metrics do not fully account for this diversity.
Improving Text-to-Image Consistency via Automatic Prompt Optimization
In this paper, we address these challenges and introduce a T2I optimization-by-prompting framework, OPT2I, which leverages a large language model (LLM) to improve prompt-image consistency in T2I models.
InCoRo: In-Context Learning for Robotics Control with Feedback Loops
We highlight the generalization capabilities of our system and show that (1) in-context learning in combination with the current state-of-the-art LLMs is an effective way to implement a robotic controller; (2) in static environments, InCoRo surpasses the prior art in terms of the success rate; (3) in dynamic environments, we establish new state-of-the-art for the SCARA and DELTA units, respectively.
Feedback-guided Data Synthesis for Imbalanced Classification
In this work, we introduce a framework for augmenting static datasets with useful synthetic samples, which leverages one-shot feedback from the classifier to drive the sampling of the generative model.
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity
The unprecedented photorealistic results achieved by recent text-to-image generative systems and their increasing use as plug-and-play content creation solutions make it crucial to understand their potential biases.
Improved baselines for vision-language pre-training
Indeed, we find that a simple CLIP baseline can also be improved substantially, up to a 25% relative improvement on downstream zero-shot tasks, by using well-known training techniques that are popular in other subfields.
Controllable Image Generation via Collage Representations
Our experiments on the OI dataset show that M&Ms outperforms baselines in terms of fine-grained scene controllability while being very competitive in terms of image quality and sample diversity.
Instance-Conditioned GAN Data Augmentation for Representation Learning
We showcase the benefits of DA_IC-GAN by plugging it out-of-the-box into the supervised training of ResNets and DeiT models on the ImageNet dataset, and achieving accuracy boosts up to between 1%p and 2%p with the highest capacity models.
Learning to Substitute Ingredients in Recipes
Recipe personalization through ingredient substitution has the potential to help people meet their dietary needs and preferences, avoid potential allergens, and ease culinary exploration in everyone's kitchen.
ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations
Equipped with ImageNet-X, we investigate 2, 200 current recognition models and study the types of mistakes as a function of model's (1) architecture, e. g. transformer vs. convolutional, (2) learning paradigm, e. g. supervised vs. self-supervised, and (3) training procedures, e. g., data augmentation.
Uncertainty-Driven Active Vision for Implicit Scene Reconstruction
We evaluate our proposed approach on the ABC dataset and the in the wild CO3D dataset, and show that: (1) we are able to obtain high quality state-of-the-art occupancy reconstructions; (2) our perspective conditioned uncertainty definition is effective to drive improvements in next best view selection and outperforms strong baseline approaches; and (3) we can further improve shape understanding by performing a gradient-based search on the view selection candidates.
On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction
Most current approaches to undersampled multi-coil MRI reconstruction focus on learning the reconstruction model for a fixed, equidistant acquisition trajectory.
Parameter Prediction for Unseen Deep Architectures
We introduce a large-scale dataset of diverse computational graphs of neural architectures - DeepNets-1M - and use it to explore parameter prediction on CIFAR-10 and ImageNet.
Ranked #1 on
Parameter Prediction
on CIFAR10
Instance-Conditioned GAN
Generative Adversarial Networks (GANs) can generate near photo realistic images in narrow domains such as human faces.
Ranked #1 on
Conditional Image Generation
on ImageNet 64x64
Active 3D Shape Reconstruction from Vision and Touch
In this paper, we focus on this problem and introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2)a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile signals; and 3) a set of data-driven solutions with either tactile or visuotactile priors to guide the shape exploration.
Generating unseen complex scenes: are we there yet?
In this paper, we propose a methodology to compare complex scene conditional generation models, and provide an in-depth analysis that assesses the ability of each model to (1) fit the training distribution and hence perform well on seen conditionings, (2) to generalize to unseen conditionings composed of seen object combinations, and (3) generalize to unseen conditionings composed of unseen object combinations.
Instance Selection for GANs
By refining the empirical data distribution before training, we redirect model capacity towards high-density regions, which ultimately improves sample fidelity, lowers model capacity requirements, and significantly reduces training time.
Ranked #2 on
Conditional Image Generation
on ImageNet 64x64
Active MR k-space Sampling with Reinforcement Learning
Deep learning approaches have recently shown great promise in accelerating magnetic resonance image (MRI) acquisition.
3D Shape Reconstruction from Vision and Touch
When a toddler is presented a new toy, their instinctual behaviour is to pick it upand inspect it with their hand and eyes in tandem, clearly searching over its surface to properly understand what they are playing with.
Extending Unsupervised Neural Image Compression With Supervised Multitask Learning
Additionally, we trained multiple encoders with different training objectives, e. g. unsupervised and variants of MTL, and observed a positive correlation between the number of tasks in MTL and the system performance on the TUPAC16 dataset.
Needles in Haystacks: On Classifying Tiny Objects in Large Images
In some important computer vision domains, such as medical or hyperspectral imaging, we care about the classification of tiny objects in large images.
On the Evaluation of Conditional GANs
We show that FJD can be used as a promising single metric for cGAN benchmarking and model selection.
Elucidating image-to-set prediction: An analysis of models, losses and datasets
In this paper, we identify an important reproducibility challenge in the image-to-set prediction literature that impedes proper comparisons among published methods, namely, researchers use different evaluation protocols to assess their contributions.
Reducing Uncertainty in Undersampled MRI Reconstruction with Active Acquisition
The goal of MRI reconstruction is to restore a high fidelity image from partially observed measurements.
The Liver Tumor Segmentation Benchmark (LiTS)
In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018.
Inverse Cooking: Recipe Generation from Food Images
Our system predicts ingredients as sets by means of a novel architecture, modeling their dependencies without imposing any order, and then generates cooking instructions by attending to both image and its inferred ingredients simultaneously.
Ranked #1 on
Recipe Generation
on Recipe1M
fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively slow or expensive.
On the iterative refinement of densely connected representation levels for semantic segmentation
State-of-the-art semantic segmentation approaches increase the receptive field of their models by using either a downsampling path composed of poolings/strided convolutions or successive dilated convolutions.
Learnable Explicit Density for Continuous Latent Space and Variational Inference
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its corresponding posterior.
Image Segmentation by Iterative Inference from Conditional Score Estimation
We experimentally find that the proposed iterative inference from conditional score estimation by conditional denoising autoencoders performs better than comparable models based on CRFs or those not using any explicit modeling of the conditional joint distribution of outputs.
Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
Moreover, when applying our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we reach results that are competitive even when compared to 3D methods.
A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images
Colorectal cancer (CRC) is the third cause of cancer death worldwide.
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
State-of-the-art approaches for semantic image segmentation are built on Convolutional Neural Networks (CNNs).
Ranked #12 on
Semantic Segmentation
on CamVid
The Importance of Skip Connections in Biomedical Image Segmentation
In this paper, we study the influence of both long and short skip connections on Fully Convolutional Networks (FCN) for biomedical image segmentation.
Generic Feature Learning for Wireless Capsule Endoscopy Analysis
Most of the CAD systems in the capsule endoscopy share a common system design, but use very different image and video representations.
