Search Results for author:
Found 35 papers, 19 papers with code
Beyond Pixel Norm-Balls: Parametric Adversaries using an Analytically Differentiable Renderer
As such, we propose the direct perturbation of physical parameters that underly image formation: lighting and geometry.
Hyperparameter Optimization in Black-box Image Processing using Differentiable Proxies
We present a fully automatic system to optimize the parameters of black-box hardware and software image processing pipelines according to any arbitrary (i. e., application-specific) metric.
Pix2Scene: Learning Implicit 3D Representations from Images
Modelling 3D scenes from 2D images is a long-standing problem in computer vision with implications in, e. g., simulation and robotics.
Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning
Finally, we analyze the effects of our proposed methods on the policies that our agents learn and show that our methods successfully enforce the qualities that we propose as proxies for coordinated behaviors.
Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models
End-to-end models are an attractive new approach to spoken language understanding (SLU) in which the meaning of an utterance is inferred directly from the raw audio without employing the standard pipeline composed of a separately trained speech recognizer and natural language understanding module.
Ranked #7 on
Spoken Language Understanding
on Snips-SmartLights
Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments
In our endeavor to create a navigation assistant for the BVI, we found that existing Reinforcement Learning (RL) environments were unsuitable for the task.
Robo-PlaNet: Learning to Poke in a Day
Recently, the Deep Planning Network (PlaNet) approach was introduced as a model-based reinforcement learning method that learns environment dynamics directly from pixel observations.
Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic
Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems where we seek to recover both policies for our agents and reward functions that promote expert-like behavior.
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation
We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2. 5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution.
Surprisal-Triggered Conditional Computation with Neural Networks
The surprisal of the input, measured as the negative log-likelihood of the current observation according to the autoregressive model, is used as a measure of input difficulty.
Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator.
A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images
Thus, we propose a consistency-based (CB) loss function that encourages the output predictions to be consistent with spatial transformations of the input images.
A Weakly Supervised Region-Based Active Learning Method for COVID-19 Segmentation in CT Images
We address this challenge introducing a scalable, fast, and accurate active learning system that accelerates the labeling of CT scan images.
On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes
Many prior works have focused on _latent-encoded_ neural implicits, where a latent vector encoding of a specific shape is also fed as input.
Regularized Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) aims to facilitate a learner's ability to imitate expert behavior by acquiring reward functions that explain the expert's decisions.
Affinity LCFCN: Learning to Segment Fish with Weak Supervision
Leading automatic approaches rely on fully-supervised segmentation models to acquire these measurements but these require collecting per-pixel labels -- also time consuming and laborious: i. e., it can take up to two minutes per fish to generate accurate segmentation labels, almost always requiring at least some manual intervention.
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality.
Robust Motion In-betweening
To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3. 6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation.
Ranked #3 on
Motion Synthesis
on LaFAN1
gradSim: Differentiable simulation for system identification and visuomotor control
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences.
SSR: Semi-supervised Soft Rasterizer for single-view 2D to 3D Reconstruction
In order to obtain the viewpoints for these unlabeled images, we propose to use a Siamese network that takes two images as input and outputs whether they correspond to the same viewpoint.
Learning to Guide and to Be Guided in the Architect-Builder Problem
ABIG results in a low-level, high-frequency, guiding communication protocol that not only enables an architect-builder pair to solve the task at hand, but that can also generalize to unseen tasks.
Kubric: A scalable dataset generator
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details.
Overcoming challenges in leveraging GANs for few-shot data augmentation
In this paper, we explore the use of GAN-based few-shot data augmentation as a method to improve few-shot classification performance.
Latent Variable Models for Bayesian Causal Discovery
Learning predictors that do not rely on spurious correlations involves building causal representations.
Uncertainty-Driven Active Vision for Implicit Scene Reconstruction
We evaluate our proposed approach on the ABC dataset and the in the wild CO3D dataset, and show that: (1) we are able to obtain high quality state-of-the-art occupancy reconstructions; (2) our perspective conditioned uncertainty definition is effective to drive improvements in next best view selection and outperforms strong baseline approaches; and (3) we can further improve shape understanding by performing a gradient-based search on the view selection candidates.
Learning Latent Structural Causal Models
For linear Gaussian additive noise SCMs, we present a tractable approximate inference method which performs joint inference over the causal variables, structure and parameters of the latent SCM from random, known interventions.
Learning Neural Implicit Representations with Surface Signal Parameterizations
Traditional explicit object representations commonly couple the 3D shape data with auxiliary surface-mapped image data, such as diffuse color textures and fine-scale geometric details in normal maps that typically require a mapping of the 3D surface onto a plane, i. e., a surface parameterization; implicit representations, on the other hand, cannot be easily textured due to lack of configurable surface parameterization.
Attention-based Neural Cellular Automata
We introduce an instance of this class named $\textit{Vision Transformer Cellular Automata}$ (ViTCA).
Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests
Different types of mental rotation tests have been used extensively in psychology to understand human visual reasoning and perception.
MeshDiffusion: Score-based Generative 3D Mesh Modeling
We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and physical simulation.
A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
Training multiple agents to coordinate is an essential problem with applications in robotics, game theory, economics, and social sciences.
Efficient Graphics Representation with Differentiable Indirection
We introduce differentiable indirection -- a novel learned primitive that employs differentiable multi-scale lookup tables as an effective substitute for traditional compute and data operations across the graphics pipeline.
STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent
In this paper, we propose a novel approach to TAMP that relaxes discrete-and-continuous TAMP problems into inference problems on a continuous domain.
Differentiable Visual Computing for Inverse Problems and Machine Learning
This approach is predicated by neural network differentiability, the requirement that analytic derivatives of a given problem's task metric can be computed with respect to neural network's parameters.
Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play
In this paper, we propose the Minimax Exploiter, a game theoretic approach to exploiting Main Agents that leverages knowledge of its opponents, leading to significant increases in data efficiency.
