Search Results for author: Phillip Isola

Found 63 papers, 39 papers with code

Procedural Image Programs for Representation Learning

1 code implementation29 Nov 2022 Manel Baradad, Chun-Fu Chen, Jonas Wulff, Tongzhou Wang, Rogerio Feris, Antonio Torralba, Phillip Isola

Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias.

Representation Learning

Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings

1 code implementation28 Nov 2022 Tongzhou Wang, Phillip Isola

Asymmetrical distance structures (quasimetrics) are ubiquitous in our lives and are gaining more attention in machine learning applications.

reinforcement-learning reinforcement Learning

Powderworld: A Platform for Understanding Generalization via Rich Task Distributions

no code implementations23 Nov 2022 Kevin Frans, Phillip Isola

Within Powderworld, two motivating challenges distributions are presented, one for world-modelling and one for reinforcement learning.

reinforcement-learning reinforcement Learning

Totems: Physical Objects for Verifying Visual Integrity

no code implementations26 Sep 2022 Jingwei Ma, Lucy Chai, Minyoung Huh, Tongzhou Wang, Ser-Nam Lim, Phillip Isola, Antonio Torralba

We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene.

Image Forensics

Semantic uncertainty intervals for disentangled latent spaces

no code implementations20 Jul 2022 Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, Phillip Isola

Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street.

Image Super-Resolution

On the Learning and Learnability of Quasimetrics

2 code implementations30 Jun 2022 Tongzhou Wang, Phillip Isola

In contrast, our proposed Poisson Quasimetric Embedding (PQE) is the first quasimetric learning formulation that both is learnable with gradient-based optimization and enjoys strong performance guarantees.

Q-Learning reinforcement Learning

Denoised MDPs: Learning World Models Better Than the World Itself

1 code implementation30 Jun 2022 Tongzhou Wang, Simon S. Du, Antonio Torralba, Phillip Isola, Amy Zhang, Yuandong Tian

The ability to separate signal from noise, and reason with clean abstractions, is critical to intelligence.

Representation Learning

MultiEarth 2022 -- Multimodal Learning for Earth and Environment Workshop and Challenge

no code implementations15 Apr 2022 Miriam Cha, Kuan Wei Huang, Morgan Schmidt, Gregory Angelides, Mark Hamilton, Sam Goldberg, Armando Cabrera, Phillip Isola, Taylor Perron, Bill Freeman, Yen-Chen Lin, Brandon Swenson, Jean Piou

The Multimodal Learning for Earth and Environment Challenge (MultiEarth 2022) will be the first competition aimed at the monitoring and analysis of deforestation in the Amazon rainforest at any time and in any weather conditions.

Image-to-Image Translation Matrix Completion +2

Any-resolution Training for High-resolution Image Synthesis

no code implementations14 Apr 2022 Lucy Chai, Michael Gharbi, Eli Shechtman, Phillip Isola, Richard Zhang

To take advantage of varied-size data, we introduce continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions.

Image Generation

Exploring Visual Prompts for Adapting Large-Scale Models

1 code implementation31 Mar 2022 Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, Phillip Isola

The surprising effectiveness of visual prompting provides a new perspective on adapting pre-trained models in vision.

Learning to generate line drawings that convey geometry and semantics

1 code implementation CVPR 2022 Caroline Chan, Fredo Durand, Phillip Isola

We introduce a geometry loss which predicts depth information from the image features of a line drawing, and a semantic loss which matches the CLIP features of a line drawing with its corresponding photograph.

Translation Unsupervised Image-To-Image Translation

NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields

no code implementations3 Mar 2022 Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Tsung-Yi Lin, Alberto Rodriguez, Phillip Isola

In particular, we demonstrate that a NeRF representation of a scene can be used to train dense object descriptors.

Contrastive Feature Loss for Image Prediction

1 code implementation12 Nov 2021 Alex Andonian, Taesung Park, Bryan Russell, Phillip Isola, Jun-Yan Zhu, Richard Zhang

Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result.

Image Generation

The Neural MMO Platform for Massively Multiagent Research

no code implementations14 Oct 2021 Joseph Suarez, Yilun Du, Clare Zhu, Igor Mordatch, Phillip Isola

Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems.

OPEn: An Open-ended Physics Environment for Learning Without a Task

no code implementations13 Oct 2021 Chuang Gan, Abhishek Bhandwaldar, Antonio Torralba, Joshua B. Tenenbaum, Phillip Isola

We test several existing RL-based exploration methods on this benchmark and find that an agent using unsupervised contrastive learning for representation learning, and impact-driven learning for exploration, achieved the best results.

Contrastive Learning Representation Learning

Adaptable Agent Populations via a Generative Model of Policies

1 code implementation NeurIPS 2021 Kenneth Derek, Phillip Isola

To this end, we introduce a generative model of policies, which maps a low-dimensional latent space to an agent policy space.

Learning to See before Learning to Act: Visual Pre-training for Manipulation

no code implementations1 Jul 2021 Lin Yen-Chen, Andy Zeng, Shuran Song, Phillip Isola, Tsung-Yi Lin

With just a small amount of robotic experience, we can further fine-tune the affordance model to achieve better results.

Transfer Learning

Learning to See by Looking at Noise

1 code implementation NeurIPS 2021 Manel Baradad, Jonas Wulff, Tongzhou Wang, Phillip Isola, Antonio Torralba

We investigate a suite of image generation models that produce images from simple random processes.

Image Generation

Generative Models as a Data Source for Multiview Representation Learning

1 code implementation ICLR 2022 Ali Jahanian, Xavier Puig, Yonglong Tian, Phillip Isola

We investigate this question in the setting of learning general-purpose visual representations from a black-box generative model rather than directly from data.

Representation Learning

Ensembling with Deep Generative Views

1 code implementation CVPR 2021 Lucy Chai, Jun-Yan Zhu, Eli Shechtman, Phillip Isola, Richard Zhang

Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification.

Image Classification

Using latent space regression to analyze and leverage compositionality in GANs

1 code implementation ICLR 2021 Lucy Chai, Jonas Wulff, Phillip Isola

In this work, we investigate regression into the latent space as a probe to understand the compositional properties of GANs.

Image Inpainting regression

The Low-Rank Simplicity Bias in Deep Networks

1 code implementation18 Mar 2021 Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, Phillip Isola

We show empirically that our claim holds true on finite width linear and non-linear models on practical learning paradigms and show that on natural data, these are often the solutions that generalize well.

Image Classification

INeRF: Inverting Neural Radiance Fields for Pose Estimation

1 code implementation10 Dec 2020 Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin

We then show that for complex real-world scenes from the LLFF dataset, iNeRF can improve NeRF by estimating the camera poses of novel images and using these images as additional training data for NeRF.

Pose Estimation

What makes fake images detectable? Understanding properties that generalize

1 code implementation ECCV 2020 Lucy Chai, David Bau, Ser-Nam Lim, Phillip Isola

The quality of image generation and manipulation is reaching impressive levels, making it increasingly difficult for a human to distinguish between what is real and what is fake.

Image Generation

Noisy Agents: Self-supervised Exploration by Predicting Auditory Events

no code implementations27 Jul 2020 Chuang Gan, Xiaoyu Chen, Phillip Isola, Antonio Torralba, Joshua B. Tenenbaum

Humans integrate multiple sensory modalities (e. g. visual and audio) to build a causal understanding of the physical world.

Atari Games reinforcement Learning

What Makes for Good Views for Contrastive Learning?

1 code implementation NeurIPS 2020 Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, Phillip Isola

Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning.

Contrastive Learning Data Augmentation +8

Supervised Contrastive Learning

20 code implementations NeurIPS 2020 Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan

Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.

class-incremental learning Contrastive Learning +4

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

3 code implementations ECCV 2020 Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, Phillip Isola

The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost.

Few-Shot Image Classification General Classification

Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks

no code implementations31 Jan 2020 Joseph Suarez, Yilun Du, Igor Mordatch, Phillip Isola

We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO.

Policy Gradient Methods

Experience-Embedded Visual Foresight

no code implementations12 Nov 2019 Lin Yen-Chen, Maria Bauza, Phillip Isola

In this paper, we tackle the generalization problem via fast adaptation, where we train a prediction model to quickly adapt to the observed visual dynamics of a novel object.

Video Prediction

Learning Good Policies By Learning Good Perceptual Models

no code implementations25 Sep 2019 Yilun Du, Phillip Isola

Reinforcement learning (RL) has led to increasingly complex looking behavior in recent years.

reinforcement-learning reinforcement Learning +1

On the "steerability" of generative adversarial networks

2 code implementations16 Jul 2019 Ali Jahanian, Lucy Chai, Phillip Isola

We hypothesize that the degree of distributional shift is related to the breadth of the training data distribution.

GANalyze: Toward Visual Definitions of Cognitive Image Properties

1 code implementation ICCV 2019 Authors, :, Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola

We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence.

Contrastive Multiview Coding

6 code implementations ECCV 2020 Yonglong Tian, Dilip Krishnan, Phillip Isola

We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the resulting representation captures underlying scene semantics.

Contrastive Learning Self-Supervised Action Recognition +1

Neural MMO: A massively multiplayer game environment for intelligent agents

no code implementations ICLR 2019 Joseph Suarez, Yilun Du, Phillip Isola, Igor Mordatch

We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents.

Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents

1 code implementation2 Mar 2019 Joseph Suarez, Yilun Du, Phillip Isola, Igor Mordatch

The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.

InGAN: Capturing and Remapping the "DNA" of a Natural Image

1 code implementation1 Dec 2018 Assaf Shocher, Shai Bagon, Phillip Isola, Michal Irani

In this paper we propose an "Internal GAN" (InGAN) - an image-specific GAN - which trains on a single input image and learns its internal distribution of patches.

3D Sketching using Multi-View Deep Volumetric Prediction

no code implementations26 Jul 2017 Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, Adrien Bousseau

The main strengths of our approach are its robustness to freehand bitmap drawings, its ability to adapt to different object categories, and the continuum it offers between single-view and multi-view sketch-based modeling.

3D Reconstruction

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

189 code implementations ICCV 2017 Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs.

 Ranked #1 on Image-to-Image Translation on zebra2horse (Frechet Inception Distance metric)

Multimodal Unsupervised Image-To-Image Translation Style Transfer +2

Colorful Image Colorization

39 code implementations28 Mar 2016 Richard Zhang, Phillip Isola, Alexei A. Efros

We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result.

Colorization Image Colorization +1

Learning Ordinal Relationships for Mid-Level Vision

no code implementations ICCV 2015 Daniel Zoran, Phillip Isola, Dilip Krishnan, William T. Freeman

We demonstrate that this frame- work works well on two important mid-level vision tasks: intrinsic image decomposition and depth from an RGB im- age.

Depth Estimation Intrinsic Image Decomposition

Learning visual groups from co-occurrences in space and time

2 code implementations21 Nov 2015 Phillip Isola, Daniel Zoran, Dilip Krishnan, Edward H. Adelson

We propose a self-supervised framework that learns to group visual entities based on their rate of co-occurrence in space and time.

Discovering States and Transformations in Image Collections

no code implementations CVPR 2015 Phillip Isola, Joseph J. Lim, Edward H. Adelson

Our system works by generalizing across object classes: states and transformations learned on one set of objects are used to interpret the image collection for an entirely new object class.

Sparkle Vision: Seeing the World through Random Specular Microfacets

no code implementations26 Dec 2014 Zhengdong Zhang, Phillip Isola, Edward H. Adelson

In this paper, we study the problem of reproducing the world lighting from a single image of an object covered with random specular microfacets on the surface.

Understanding the Intrinsic Memorability of Images

no code implementations NeurIPS 2011 Phillip Isola, Devi Parikh, Antonio Torralba, Aude Oliva

Artists, advertisers, and photographers are routinely presented with the task of creating an image that a viewer will remember.

Cannot find the paper you are looking for? You can Submit a new open access paper.