Search Results for author:
Found 39 papers, 19 papers with code
A Personalized Video-Based Hand Taxonomy: Application for Individuals with Spinal Cord Injury
A comprehensive evaluation of function in home and community settings requires a hand grasp taxonomy for individuals with impaired hand function.
Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
In contrast, we advocate a generic, task-agnostic diffusion model, with several advancements such as log-scale depth parameterization to enable joint modeling of indoor and outdoor scenes, conditioning on the field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV during training to generalize beyond the limited camera intrinsics in training datasets.
The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation
Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity.
Synthetic Data from Diffusion Models Improves ImageNet Classification
Deep generative models are becoming increasingly powerful, now generating diverse high fidelity photo-realistic samples given text prompts.
Monocular Depth Estimation using Diffusion Models
To cope with the limited availability of data for supervised training, we leverage pre-training on self-supervised image-to-image translation tasks.
Ranked #22 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
RobustNeRF: Ignoring Distractors with Robust Losses
To cope with distractors, we advocate a form of robust estimation for NeRF training, modeling distractors in training data as outliers of an optimization problem.
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.
Gaussian-Bernoulli RBMs Without Tears
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations.
A Generalist Framework for Panoptic Segmentation of Images and Videos
Panoptic segmentation assigns semantic and instance ID labels to every pixel of an image.
Imagen Video: High Definition Video Generation with Diffusion Models
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models.
Ranked #1 on
Video Generation
on LAION-400M
A Unified Sequence Interface for Vision Tasks
Despite that, by formulating the output of each task as a sequence of discrete tokens with a unified interface, we show that one can train a neural network with a single model architecture and loss function on all these tasks, with no task-specific customization.
Residual Multiplicative Filter Networks for Multiscale Reconstruction
Coordinate networks like Multiplicative Filter Networks (MFNs) and BACON offer some control over the frequency spectrum used to represent continuous signals such as images or 3D volumes.
Video Diffusion Models
Generating temporally coherent high fidelity video is an important milestone in generative modeling research.
Kubric: A scalable dataset generator
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details.
Palette: Image-to-Image Diffusion Models
We expect this standardized evaluation protocol to play a role in advancing image-to-image translation research.
Ranked #1 on
Colorization
on ImageNet ctest10k
Pix2seq: A Language Modeling Framework for Object Detection
We present Pix2Seq, a simple and generic framework for object detection.
Ranked #77 on
Object Detection
on COCO minival
(using extra training data)
Cascaded Diffusion Models for High Fidelity Image Generation
We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation benchmark, without any assistance from auxiliary image classifiers to boost sample quality.
Ranked #3 on
Image Generation
on ImageNet 64x64
Image Super-Resolution via Iterative Refinement
We present SR3, an approach to image Super-Resolution via Repeated Refinement.
Bridging the Gap Between Adversarial Robustness and Optimization Bias
We demonstrate that the choice of optimizer, neural network architecture, and regularizer significantly affect the adversarial robustness of linear neural networks, providing guarantees without the need for adversarial training.
Unsupervised part representation by Flow Capsules
Capsule networks aim to parse images into a hierarchy of objects, parts and relations.
A Study of Gradient Variance in Deep Learning
We introduce a method, Gradient Clustering, to minimize the variance of average mini-batch gradient with stratified sampling.
Exemplar VAE: Linking Generative Models, Nearest Neighbor Retrieval, and Data Augmentation
We introduce Exemplar VAEs, a family of generative models that bridge the gap between parametric and non-parametric, exemplar based generative models.
SentenceMIM: A Latent Variable Language Model
MIM learning encourages high mutual information between observations and latent variables, and is robust against posterior collapse.
Ranked #1 on
Question Answering
on YahooCQA
(using extra training data)
MIM: Mutual Information Machine
Experiments show that MIM learns representations with high mutual information, consistent encoding and decoding distributions, effective latent clustering, and data log likelihood comparable to VAE, while avoiding posterior collapse.
High Mutual Information in Representation Learning with Symmetric Variational Inference
We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework.
Walking on Thin Air: Environment-Free Physics-based Markerless Motion Capture
To our knowledge, this is the first approach to take physics into account without explicit {\em a priori} knowledge of the environment or body dimensions.
TzK Flow - Conditional Generative Model
Unlike autoencoders, the bottleneck does not limit model expressiveness, similar to flow-based ML.
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
We present a new technique for learning visual-semantic embeddings for cross-modal retrieval.
Ranked #22 on
Cross-Modal Retrieval
on Flickr30k
Transductive Log Opinion Pool of Gaussian Process Experts
We introduce a framework for analyzing transductive combination of Gaussian process (GP) experts, where independently trained GP experts are combined in a way that depends on test point location, in order to scale GPs to big data.
Adversarial Manipulation of Deep Representations
We show that the representation of an image in a deep neural network (DNN) can be manipulated to mimic those of other natural images, with only minor, imperceptible perturbations to the original image.
Efficient non-greedy optimization of decision trees
In this paper, we present an algorithm for optimizing the split functions at all levels of the tree jointly with the leaf parameters, based on a global objective.
CO2 Forest: Improved Random Forest by Continuous Optimization of Oblique Splits
We develop a convex-concave upper bound on the classification loss for a one-level decision tree, and optimize the bound by stochastic gradient descent at each internal node of the tree.
Building Proteins in a Day: Efficient 3D Molecular Reconstruction
A new framework for estimation is introduced which relies on modern stochastic optimization techniques to scale to large datasets.
Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
In this work, we propose a generalized product of experts (gPoE) framework for combining the predictions of multiple probabilistic models.
Posebits for Monocular Human Pose Estimation
We advocate the inference of qualitative information about 3D human pose, called posebits, from images.
Efficient Optimization for Sparse Gaussian Process Regression
We propose an efficient optimization algorithm for selecting a subset of training data to induce sparsity for Gaussian process regression.
Fast Exact Search in Hamming Space with Multi-Index Hashing
There is growing interest in representing image data and feature descriptors using compact binary codes for fast near neighbor search.
Cartesian K-Means
A fundamental limitation of quantization techniques like the k-means clustering algorithm is the storage and runtime cost associated with the large numbers of clusters required to keep quantization errors small and model fidelity high.
Hamming Distance Metric Learning
Motivated by large-scale multimedia applications we propose to learn mappings from high-dimensional data to binary codes that preserve semantic similarity.
