Search Results for author: Phillip Isola

Found 46 papers, 29 papers with code

The Neural MMO Platform for Massively Multiagent Research

no code implementations14 Oct 2021 Joseph Suarez, Yilun Du, Clare Zhu, Igor Mordatch, Phillip Isola

Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems.

OPEn: An Open-ended Physics Environment for Learning Without a Task

no code implementations13 Oct 2021 Chuang Gan, Abhishek Bhandwaldar, Antonio Torralba, Joshua B. Tenenbaum, Phillip Isola

We test several existing RL-based exploration methods on this benchmark and find that an agent using unsupervised contrastive learning for representation learning, and impact-driven learning for exploration, achieved the best results.

Contrastive Learning Representation Learning

Adaptable Agent Populations via a Generative Model of Policies

1 code implementation15 Jul 2021 Kenneth Derek, Phillip Isola

To this end, we introduce a generative model of policies, which maps a low-dimensional latent space to an agent policy space.

Learning to See before Learning to Act: Visual Pre-training for Manipulation

no code implementations1 Jul 2021 Lin Yen-Chen, Andy Zeng, Shuran Song, Phillip Isola, Tsung-Yi Lin

With just a small amount of robotic experience, we can further fine-tune the affordance model to achieve better results.

Transfer Learning

Learning to See by Looking at Noise

1 code implementation10 Jun 2021 Manel Baradad, Jonas Wulff, Tongzhou Wang, Phillip Isola, Antonio Torralba

We investigate a suite of image generation models that produce images from simple random processes.

Image Generation

Generative Models as a Data Source for Multiview Representation Learning

1 code implementation9 Jun 2021 Ali Jahanian, Xavier Puig, Yonglong Tian, Phillip Isola

We investigate this question in the setting of learning general-purpose visual representations from a black-box generative model rather than directly from data.

Representation Learning

Ensembling with Deep Generative Views

no code implementations CVPR 2021 Lucy Chai, Jun-Yan Zhu, Eli Shechtman, Phillip Isola, Richard Zhang

Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification.

Image Classification

Using latent space regression to analyze and leverage compositionality in GANs

no code implementations ICLR 2021 Lucy Chai, Jonas Wulff, Phillip Isola

In this work, we investigate regression into the latent space as a probe to understand the compositional properties of GANs.

GAN inversion Image Inpainting

The Low-Rank Simplicity Bias in Deep Networks

1 code implementation18 Mar 2021 Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, Phillip Isola

Modern deep neural networks are highly over-parameterized compared to the data on which they are trained, yet they often generalize remarkably well.

Image Classification

INeRF: Inverting Neural Radiance Fields for Pose Estimation

1 code implementation10 Dec 2020 Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin

We then show that for complex real-world scenes from the LLFF dataset, iNeRF can improve NeRF by estimating the camera poses of novel images and using these images as additional training data for NeRF.

Pose Estimation

What makes fake images detectable? Understanding properties that generalize

1 code implementation ECCV 2020 Lucy Chai, David Bau, Ser-Nam Lim, Phillip Isola

The quality of image generation and manipulation is reaching impressive levels, making it increasingly difficult for a human to distinguish between what is real and what is fake.

Image Generation

Noisy Agents: Self-supervised Exploration by Predicting Auditory Events

no code implementations27 Jul 2020 Chuang Gan, Xiaoyu Chen, Phillip Isola, Antonio Torralba, Joshua B. Tenenbaum

Humans integrate multiple sensory modalities (e. g. visual and audio) to build a causal understanding of the physical world.

Atari Games

What Makes for Good Views for Contrastive Learning?

1 code implementation NeurIPS 2020 Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, Phillip Isola

Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning.

Contrastive Learning Data Augmentation +7

Supervised Contrastive Learning

13 code implementations NeurIPS 2020 Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan

Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.

Contrastive Learning Data Augmentation +3

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

1 code implementation ECCV 2020 Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, Phillip Isola

The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost.

Few-Shot Image Classification General Classification

Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks

no code implementations31 Jan 2020 Joseph Suarez, Yilun Du, Igor Mordatch, Phillip Isola

We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO.

Policy Gradient Methods

Experience-Embedded Visual Foresight

no code implementations12 Nov 2019 Lin Yen-Chen, Maria Bauza, Phillip Isola

In this paper, we tackle the generalization problem via fast adaptation, where we train a prediction model to quickly adapt to the observed visual dynamics of a novel object.

Video Prediction

On the "steerability" of generative adversarial networks

1 code implementation16 Jul 2019 Ali Jahanian, Lucy Chai, Phillip Isola

We hypothesize that the degree of distributional shift is related to the breadth of the training data distribution.

GANalyze: Toward Visual Definitions of Cognitive Image Properties

1 code implementation ICCV 2019 Authors, :, Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola

We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence.

Contrastive Multiview Coding

6 code implementations ECCV 2020 Yonglong Tian, Dilip Krishnan, Phillip Isola

We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the resulting representation captures underlying scene semantics.

Contrastive Learning Object Classification +2

Neural MMO: A massively multiplayer game environment for intelligent agents

no code implementations ICLR 2019 Joseph Suarez, Yilun Du, Phillip Isola, Igor Mordatch

We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents.

Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents

1 code implementation2 Mar 2019 Joseph Suarez, Yilun Du, Phillip Isola, Igor Mordatch

The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.

InGAN: Capturing and Remapping the "DNA" of a Natural Image

1 code implementation1 Dec 2018 Assaf Shocher, Shai Bagon, Phillip Isola, Michal Irani

In this paper we propose an "Internal GAN" (InGAN) - an image-specific GAN - which trains on a single input image and learns its internal distribution of patches.

Evolved Policy Gradients

3 code implementations NeurIPS 2018 Rein Houthooft, Richard Y. Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, Pieter Abbeel

We propose a metalearning approach for learning gradient-based reinforcement learning (RL) algorithms.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

24 code implementations CVPR 2018 Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, Oliver Wang

We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics.


3D Sketching using Multi-View Deep Volumetric Prediction

no code implementations26 Jul 2017 Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, Adrien Bousseau

The main strengths of our approach are its robustness to freehand bitmap drawings, its ability to adapt to different object categories, and the continuum it offers between single-view and multi-view sketch-based modeling.

3D Reconstruction

Real-Time User-Guided Image Colorization with Learned Deep Priors

3 code implementations8 May 2017 Richard Zhang, Jun-Yan Zhu, Phillip Isola, Xinyang Geng, Angela S. Lin, Tianhe Yu, Alexei A. Efros

The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN).


Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

167 code implementations ICCV 2017 Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs.

 Ranked #1 on Image-to-Image Translation on photo2vangogh (Frechet Inception Distance metric)

Multimodal Unsupervised Image-To-Image Translation Style Transfer +2

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

2 code implementations CVPR 2017 Richard Zhang, Phillip Isola, Alexei A. Efros

We propose split-brain autoencoders, a straightforward modification of the traditional autoencoder architecture, for unsupervised representation learning.

Transfer Learning Unsupervised Representation Learning

Colorful Image Colorization

37 code implementations28 Mar 2016 Richard Zhang, Phillip Isola, Alexei A. Efros

We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result.

Colorization Self-Supervised Image Classification

Learning Ordinal Relationships for Mid-Level Vision

no code implementations ICCV 2015 Daniel Zoran, Phillip Isola, Dilip Krishnan, William T. Freeman

We demonstrate that this frame- work works well on two important mid-level vision tasks: intrinsic image decomposition and depth from an RGB im- age.

Depth Estimation Intrinsic Image Decomposition

Learning visual groups from co-occurrences in space and time

1 code implementation21 Nov 2015 Phillip Isola, Daniel Zoran, Dilip Krishnan, Edward H. Adelson

We propose a self-supervised framework that learns to group visual entities based on their rate of co-occurrence in space and time.

Discovering States and Transformations in Image Collections

no code implementations CVPR 2015 Phillip Isola, Joseph J. Lim, Edward H. Adelson

Our system works by generalizing across object classes: states and transformations learned on one set of objects are used to interpret the image collection for an entirely new object class.

Sparkle Vision: Seeing the World through Random Specular Microfacets

no code implementations26 Dec 2014 Zhengdong Zhang, Phillip Isola, Edward H. Adelson

In this paper, we study the problem of reproducing the world lighting from a single image of an object covered with random specular microfacets on the surface.

Understanding the Intrinsic Memorability of Images

no code implementations NeurIPS 2011 Phillip Isola, Devi Parikh, Antonio Torralba, Aude Oliva

Artists, advertisers, and photographers are routinely presented with the task of creating an image that a viewer will remember.

Feature Selection

Cannot find the paper you are looking for? You can Submit a new open access paper.