Search Results for author:
Found 57 papers, 24 papers with code
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs
The abilities of large language models (LLMs) have recently progressed to unprecedented levels, paving the way to novel applications in a wide variety of areas.
Better (pseudo-)labels for semi-supervised instance segmentation
Despite the availability of large datasets for tasks like image classification and image-text alignment, labeled data for more complex recognition tasks, such as detection and segmentation, is less abundant.
Residual Quantization with Implicit Neural Codebooks
In this paper, we propose QINCo, a neural RQ variant which predicts specialized codebooks per vector using a neural network that is conditioned on the approximation of the vector from previous steps.
Unlocking Pre-trained Image Backbones for Semantic Image Synthesis
Semantic image synthesis, i. e., generating images from user-provided semantic label maps, is an important conditional image generation task as it allows to control both the content as well as the spatial layout of generated images.
Towards image compression with perfect realism at ultra-low bitrates
We find that our model leads to reconstructions with state-of-the-art visual quality as measured by FID and KID.
Guided Distillation for Semi-Supervised Instance Segmentation
Although instance segmentation methods have improved considerably, the dominant paradigm is to rely on fully-annotated training images, which are tedious to obtain.
Multi-Domain Learning with Modulation Adapters
Deep convolutional networks are ubiquitous in computer vision, due to their excellent performance across different tasks for various domains.
Zero-shot spatial layout conditioning for text-to-image diffusion models
Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling and allow for an intuitive and powerful user interface to drive the image generation process.
Improved baselines for vision-language pre-training
Indeed, we find that a simple CLIP baseline can also be improved substantially, up to a 25% relative improvement on downstream zero-shot tasks, by using well-known training techniques that are popular in other subfields.
Controllable Image Generation via Collage Representations
Our experiments on the OI dataset show that M&Ms outperforms baselines in terms of fine-grained scene controllability while being very competitive in terms of image quality and sample diversity.
Are Visual Recognition Models Robust to Image Compression?
For example, for segmentation mIoU is reduced from 44. 5 to 30. 5 mIoU when compressing to 0. 1 bpp using the best compression model we evaluated.
Few-shot Semantic Image Synthesis with Class Affinity Transfer
The class affinity matrix is introduced as a first layer to the source model to make it compatible with the target label maps, and the source model is then further finetuned for the target domain.
Instance-Conditioned GAN Data Augmentation for Representation Learning
We showcase the benefits of DA_IC-GAN by plugging it out-of-the-box into the supervised training of ResNets and DeiT models on the ImageNet dataset, and achieving accuracy boosts up to between 1%p and 2%p with the highest capacity models.
Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models
Lossy image compression aims to represent images in as few bits as possible while maintaining fidelity to the original.
Co-Training 2L Submodels for Visual Recognition
Given a neural network to be trained, for each sample we implicitly instantiate two altered networks, "submodels", with stochastic depth: i. e. activating only a subset of the layers and skipping others.
Image Compression with Product Quantized Masked Image Modeling
In this work, we attempt to bring these lines of research closer by revisiting vector quantization for image compression.
Co-training $2^L$ Submodels for Visual Recognition
We introduce submodel co-training, a regularization method related to co-training, self-distillation and stochastic depth.
Ranked #69 on
Image Classification
on ImageNet
Unifying conditional and unconditional semantic image synthesis with OCO-GAN
To allow for more control, image synthesis can be conditioned on semantic segmentation maps that instruct the generator the position of objects in the image.
DiffEdit: Diffusion-based semantic image editing with mask guidance
Semantic image editing is an extension of image generation, with the additional constraint that the generated image should be as similar as possible to a given input image.
Three things everyone should know about Vision Transformers
(2) Fine-tuning the weights of the attention layers is sufficient to adapt vision transformers to a higher resolution and to other classification tasks.
Ranked #8 on
Image Classification
on CIFAR-10
(using extra training data)
FlexIT: Towards Flexible Semantic Image Translation
Via the latent space of an auto-encoder, we iteratively transform the input image toward the target point, ensuring coherence and quality with a variety of novel regularization terms.
Instance-Conditioned GAN
Generative Adversarial Networks (GANs) can generate near photo realistic images in narrow domains such as human faces.
Ranked #1 on
Conditional Image Generation
on ImageNet 64x64
XCiT: Cross-Covariance Image Transformers
We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries.
Ranked #55 on
Instance Segmentation
on COCO minival
ResMLP: Feedforward networks for image classification with data-efficient training
We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification.
Ranked #1 on
Image Classification
on Certificate Verification
Dual Mesh Convolutional Networks for Human Shape Correspondence
Convolutional networks have been extremely successful for regular data structures such as 2D images and 3D voxel grids.
Meta-Learning with Shared Amortized Variational Inference
We propose a novel amortized variational inference scheme for an empirical Bayes meta-learning model, where model parameters are treated as latent variables.
Discrete Point Flow Networks for Efficient Point Cloud Generation
Generative models have proven effective at modeling 3D shapes and their statistical variations.
Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English
We conduct in this work an evaluation study comparing offline and online neural machine translation architectures.
Efficient Wait-k Models for Simultaneous Machine Translation
We also show that the 2D-convolution architecture is competitive with Transformers for simultaneous translation of spoken language.
Anytime Inference with Distilled Hierarchical Neural Ensembles
We propose Hierarchical Neural Ensembles (HNE), a novel framework to embed an ensemble of multiple networks in a hierarchical tree structure, sharing intermediate layers.
Improved Training Techniques for Online Neural Machine Translation
We investigate the sensitivity of such models to the value of k that is used during training and when deploying the model, and the effect of updating the hidden states in transformer models as new source tokens are read.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Hierarchical Scene Coordinate Classification and Regression for Visual Localization
In this work, we present a new hierarchical scene coordinate network to predict pixel scene coordinates in a coarse-to-fine manner from a single RGB image.
Probabilistic Reconstruction Networks for 3D Shape Inference from a Single Image
We study end-to-end learning strategies for 3D shape inference from images, in particular from a single image.
Adaptative Inference Cost With Convolutional Neural Mixture Models
Despite the outstanding performance of convolutional neural networks (CNNs) for many vision tasks, the required computational cost during inference is problematic when resources are limited.
Reference-based Variational Autoencoders
Given a pool of unlabelled images, the goal is to learn a representation where a set of target factors are disentangled from others.
Learning Disentangled Representations with Reference-Based Variational Autoencoders
Given a pool of unlabeled images, the goal is to learn a representation where a set of target factors are disentangled from others.
Adaptive Density Estimation for Generative Models
We show that our model significantly improves over existing hybrid models: offering GAN-like samples, IS and FID scores that are competitive with fully adversarial models, and improved likelihood scores.
Understanding Priors in Bayesian Neural Networks at the Unit Level
We investigate deep Bayesian neural networks with Gaussian weight priors and a class of ReLU-like nonlinearities.
Coverage and Quality Driven Training of Generative Image Models
First, we propose a model that extends variational autoencoders by using deterministic invertible transformation layers to map samples from the decoder to the image space.
Scene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization
Image-based camera relocalization is an important problem in computer vision and robotics.
Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
Current state-of-the-art machine translation systems are based on encoder-decoder architectures, that first encode the input sequence, and then generate an output sequence based on the input encoding.
Ranked #2 on
Machine Translation
on IWSLT2015 German-English
Mixed batches and symmetric discriminators for GAN training
We propose to feed the discriminator with mixed batches of true and fake samples, and train it to predict the ratio of true samples in the batch.
Token-level and sequence-level loss smoothing for RNN language models
We extend this approach to token-level loss smoothing, and propose improvements to the sequence-level smoothing approach.
Predicting Future Instance Segmentation by Forecasting Convolutional Features
We apply the "detection head'" of Mask R-CNN on the predicted features to produce the instance segmentation of future frames.
Auxiliary Guided Autoregressive Variational Autoencoders
Our contribution is a training procedure relying on an auxiliary loss function that controls which information is captured by the latent variables and what is left to the autoregressive decoder.
FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis
Convolutional neural networks (CNNs) have massively impacted visual recognition in 2D images, and are now ubiquitous in state-of-the-art approaches.
Predicting Deeper into the Future of Semantic Segmentation
The ability to predict and therefore to anticipate the future is an important attribute of intelligence.
Areas of Attention for Image Captioning
We propose "Areas of Attention", a novel attention-based model for automatic image captioning.
Semantic Segmentation using Adversarial Networks
Adversarial training has been shown to produce state of the art results for generative image modeling.
Convolutional Neural Fabrics
Despite the success of CNNs, selecting the optimal architecture for a given task remains an open problem.
Frankenstein: Learning Deep Face Representations using Small Data
To train such networks, very large training sets are needed with millions of labeled images.
Approximate Fisher Kernels of non-iid Image Models for Image Categorization
It has been experimentally observed that the performance of BoW and FV representations can be improved by employing discounting transformations such as power normalization.
Circulant temporal encoding for video retrieval and temporal alignment
We address the problem of specific video event retrieval.
A robust and efficient video representation for action recognition
We also use the homography to cancel out camera motion from the optical flow.
Weakly Supervised Object Localization with Multi-fold Multiple Instance Learning
In this case, the supervised information is restricted to binary labels that indicate the absence/presence of object instances in the image, without their locations.
Multi-fold MIL Training for Weakly Supervised Object Localization
In this case, the supervised information is restricted to binary labels that indicate the absence/presence of object instances in the image, without their locations.
Efficient Action Localization with Approximately Normalized Fisher Vectors
Transformation of the FV by power and L2 normalizations has shown to significantly improve its performance, and led to state-of-the-art results for a range of image and video classification and retrieval tasks.
