Search Results for author:
Found 42 papers, 15 papers with code
Many-to-many Image Generation with Auto-regressive Diffusion Models
This paper introduces a domain-general framework for many-to-many image generation, capable of producing interrelated image series from a given set of images, offering a scalable solution that obviates the need for task-specific solutions across different multi-image scenarios.
How Far Are We from Intelligent Visual Deductive Reasoning?
Vision-Language Models (VLMs) such as GPT-4V have recently demonstrated incredible strides on diverse vision language tasks.
Scalable Pre-training of Large Autoregressive Image Models
Specifically, we highlight two key findings: (1) the performance of the visual features scale with both the model capacity and the quantity of data, (2) the value of the objective function correlates with the performance of the model on downstream tasks.
Ranked #333 on
Image Classification
on ImageNet
(using extra training data)
Matryoshka Diffusion Models
Diffusion models are the de facto approach for generating high-quality images and videos, but learning high-dimensional models remains a formidable task due to computational and optimization challenges.
Generative Modeling with Phase Stochastic Bridges
In this work, we introduce a novel generative modeling framework grounded in \textbf{phase space dynamics}, where a phase space is defined as {an augmented space encompassing both position and velocity.}
BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping
Diffusion models have demonstrated excellent potential for generating diverse images.
PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation.
AutoFocusFormer: Image Segmentation off the Grid
To achieve this, we propose AutoFocusFormer (AFF), a local-attention transformer image recognition backbone, which performs adaptive downsampling by learning to retain the most important pixels for the task.
Ranked #4 on
Instance Segmentation
on Cityscapes val
Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Images
To address these challenges, We present Control3Diff, a 3D diffusion model that combines the strengths of diffusion models and 3D GANs for versatile, controllable 3D-aware image synthesis for single-view datasets.
Stabilizing Transformer Training by Preventing Attention Entropy Collapse
We show that $\sigma$Reparam provides stability and robustness with respect to the choice of hyperparameters, going so far as enabling training (a) a Vision Transformer {to competitive performance} without warmup, weight decay, layer normalization or adaptive optimizers; (b) deep architectures in machine translation and (c) speech recognition to competitive performance without warmup and adaptive optimizers.
TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation
Denoising Diffusion models have demonstrated their proficiency for generative sampling.
f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation
In this work, we propose f-DM, a generalized family of DMs which allows progressive signal transformation.
GAUDI: A Neural Architect for Immersive 3D Scene Generation
We introduce GAUDI, a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera.
Ranked #1 on
Image Generation
on ARKitScenes
Position Prediction as an Effective Pretraining Strategy
This pretraining strategy which has been used in BERT models in NLP, Wav2Vec models in Speech and, recently, in MAE models in Vision, forces the model to learn about relationships between the content in different parts of the input using autoencoding related objectives.
The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon
While common and easily reproduced in more general settings, the Slingshot Mechanism does not follow from any known optimization theories that we are aware of, and can be easily overlooked without an in depth examination.
Learning Representation from Neural Fisher Kernel with Low-rank Approximation
We show that the low-rank approximation of NFKs derived from unsupervised generative models and supervised learning models gives rise to high-quality compact representations of data, achieving competitive results on a variety of machine learning tasks.
Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models
Modeling the world can benefit robot learning by providing a rich training signal for shaping an agent's latent state space.
A Dot Product Attention Free Transformer
We introduce Dot Product Attention Free Transformer (DAFT), an efficient variant of Transformers \citep{transformer} that eliminates the query-key dot product in self attention.
Ranked #620 on
Image Classification
on ImageNet
Regularized Training of Nearest Neighbor Language Models
In particular, we find that the added L2 regularization seems to improve the performance for high-frequency words without deteriorating the performance for low frequency ones.
Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottlenecks
We analyze the learning dynamics of infinitely wide neural networks with a finite sized bottle-neck.
An Attention Free Transformer
We introduce Attention Free Transformer (AFT), an efficient variant of Transformers that eliminates the need for dot product self attention.
Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning
Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration.
MetricOpt: Learning to Optimize Black-Box Evaluation Metrics
This leads to consistent improvements since the value function provides effective metric supervision during finetuning, and helps to correct the potential bias of loss-only supervision.
Uncertainty Weighted Offline Reinforcement Learning
Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration.
On the generalization of learning-based 3D reconstruction
State-of-the-art learning-based monocular 3D reconstruction methods learn priors over object categories on the training set, and as a result struggle to achieve reasonable generalization to object categories unseen during training.
Set Distribution Networks: a Generative Model for Sets of Images
We introduce Set Distribution Networks (SDNs), a novel framework that learns to autoencode and freely generate sets.
Adversarial Fisher Vectors for Unsupervised Representation Learning
In contrast to a traditional view where the discriminator learns a constant function when reaching convergence, here we show that it can provide useful information for downstream tasks, e. g., feature extraction for classification.
Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking
Deep neural networks require collecting and annotating large amounts of data to train successfully.
Ranked #44 on
Self-Supervised Action Recognition
on UCF101
Hierarchical Bayes Autoencoders
During inference time, the HBAE consists of two sampling steps: first a latent code for the input is sampled, and then this code is passed to the conditional generator to output a stochastic reconstruction.
Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment
In most machine learning training paradigms a fixed, often handcrafted, loss function is assumed to be a good proxy for an underlying evaluation metric.
A Deep Learning Approach for Expert Identification in Question Answering Communities
This approach uses the convolutional neural network and combines user feature representations with question feature representations to compute scores that the user who gets the highest score is the expert on this question.
Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records
We use this generative model together with a convolutional neural network (CNN) based prediction model to improve the onset prediction performance.
Structural Correspondence Learning for Cross-lingual Sentiment Classification with One-to-many Mappings
For simplicity, however, it assumes that the word translation oracle maps each pivot feature in source language to exactly only one word in target language.
Cross-Lingual Sentiment Classification
General Classification
+4
Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
Multi-task learning aims to improve generalization performance of multiple prediction tasks by appropriately sharing relevant information across them.
S3Pool: Pooling with Stochastic Spatial Sampling
We view the pooling operation in CNNs as a two-step procedure: first, a pooling window (e. g., $2\times 2$) slides over the feature map with stride one which leaves the spatial resolution intact, and second, downsampling is performed by selecting one pixel from each non-overlapping pooling window in an often uniform and deterministic (e. g., top-left) manner.
Generative Adversarial Networks as Variational Training of Energy Based Models
We propose VGAN, which works by minimizing a variational lower bound of the negative log likelihood (NLL) of an energy based model (EBM), where the model density $p(\mathbf{x})$ is approximated by a variational distribution $q(\mathbf{x})$ that is easy to sample from.
Doubly Convolutional Neural Networks
Building large models with parameter sharing accounts for most of the success of deep convolutional neural networks (CNNs).
Deep Structured Energy Based Models for Anomaly Detection
In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures.
Semisupervised Autoencoder for Sentiment Analysis
In this paper, we investigate the usage of autoencoders in modeling textual data.
Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs
Matrix factorization (MF) and Autoencoder (AE) are among the most successful approaches of unsupervised learning.
Manifold Regularized Discriminative Neural Networks
The first one, named Label-Aware Manifold Regularization, assumes the availability of labels and penalizes large norms of the loss function w. r. t.
