Search Results for author: Dilip Krishnan

Found 39 papers, 23 papers with code

Scaling Laws of Synthetic Images for Model Training ... for Now

1 code implementation7 Dec 2023 Lijie Fan, KaiFeng Chen, Dilip Krishnan, Dina Katabi, Phillip Isola, Yonglong Tian

Our findings also suggest that scaling synthetic data can be particularly effective in scenarios such as: (1) when there is a limited supply of real images for a supervised problem (e. g., fewer than 0. 5 million images in ImageNet), (2) when the evaluation dataset diverges significantly from the training data, indicating the out-of-distribution scenario, or (3) when synthetic data is used in conjunction with real images, as demonstrated in the training of CLIP models.

Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency

no code implementations5 Oct 2023 Tianhong Li, Sangnie Bhardwaj, Yonglong Tian, Han Zhang, Jarred Barber, Dina Katabi, Guillaume Lajoie, Huiwen Chang, Dilip Krishnan

We demonstrate image generation and captioning performance on par with state-of-the-art text-to-image and image-to-text models with orders of magnitude fewer (only 3M) paired image-text data.

Substance or Style: What Does Your Image Embedding Know?

no code implementations10 Jul 2023 Cyrus Rashtchian, Charles Herrmann, Chun-Sung Ferng, Ayan Chakrabarti, Dilip Krishnan, Deqing Sun, Da-Cheng Juan, Andrew Tomkins

We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).

Style Transfer

Improving CLIP Training with Language Rewrites

1 code implementation NeurIPS 2023 Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, Yonglong Tian

During training, LaCLIP randomly selects either the original texts or the rewritten versions as text augmentations for each image.

Steerable Equivariant Representation Learning

no code implementations22 Feb 2023 Sangnie Bhardwaj, Willie McClinton, Tongzhou Wang, Guillaume Lajoie, Chen Sun, Phillip Isola, Dilip Krishnan

In this paper, we propose a method of learning representations that are instead equivariant to data augmentations.

Image Retrieval object-detection +5

Muse: Text-To-Image Generation via Masked Generative Transformers

3 code implementations2 Jan 2023 Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, Dilip Krishnan

Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.

Language Modelling Large Language Model

A simple, efficient and scalable contrastive masked autoencoder for learning visual representations

1 code implementation30 Oct 2022 Shlok Mishra, Joshua Robinson, Huiwen Chang, David Jacobs, Aaron Sarna, Aaron Maschinot, Dilip Krishnan

Our framework is a minimal and conceptually clean synthesis of (C) contrastive learning, (A) masked autoencoders, and (N) the noise prediction approach used in diffusion models.

Contrastive Learning Self-Supervised Learning +1

Simplified Transfer Learning for Chest Radiography Models Using Less Data

1 code implementation Radiology 2022 Andrew B. Sellergren, Christina Chen, Zaid Nabulsi, Yuanzhen Li, Aaron Maschinot, Aaron Sarna, Jenny Huang, Charles Lau, Sreenivasa Raju Kalidindi, Mozziyar Etemadi, Florencia Garcia-Vicente, David Melnick, Yun Liu, Krish Eswaran, Daniel Tse, Neeral Beladia, Dilip Krishnan, Shravya Shetty

Supervised contrastive learning enabled performance comparable to state-of-the-art deep learning models in multiple clinical tasks by using as few as 45 images and is a promising method for predictive modeling with use of small data sets and for predicting outcomes in shifting patient populations.

Contrastive Learning Transfer Learning

Object-Aware Cropping for Self-Supervised Learning

1 code implementation1 Dec 2021 Shlok Mishra, Anshul Shah, Ankan Bansal, Abhyuday Jagannatha, Janit Anjaria, Abhishek Sharma, David Jacobs, Dilip Krishnan

This assumption is mostly satisfied in datasets such as ImageNet where there is a large, centered object, which is highly likely to be present in random crops of the full image.

Data Augmentation object-detection +2

Pyramid Adversarial Training Improves ViT Performance

1 code implementation CVPR 2022 Charles Herrmann, Kyle Sargent, Lu Jiang, Ramin Zabih, Huiwen Chang, Ce Liu, Dilip Krishnan, Deqing Sun

In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.

Ranked #9 on Domain Generalization on ImageNet-C (using extra training data)

Adversarial Attack Data Augmentation +2

CSI: Contrastive Data Stratification for Interaction Prediction and its Application to Compound-Protein Interaction Prediction

no code implementations18 Nov 2021 Apurva Kalia, Dilip Krishnan, Soha Hassoun

Accurately predicting the likelihood of interaction between two objects (compound-protein sequence, user-item, author-paper, etc.)

Contrastive Learning

Unsupervised Disentanglement without Autoencoding: Pitfalls and Future Directions

1 code implementation14 Aug 2021 Andrea Burns, Aaron Sarna, Dilip Krishnan, Aaron Maschinot

Disentangled visual representations have largely been studied with generative models such as Variational AutoEncoders (VAEs).

Contrastive Learning Disentanglement

Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers

no code implementations15 Mar 2021 Piotr Teterwak, Chiyuan Zhang, Dilip Krishnan, Michael C. Mozer

We use our reconstruction model as a tool for exploring the nature of representations, including: the influence of model architecture and training objectives (specifically robust losses), the forms of invariance that networks achieve, representational differences between correctly and incorrectly classified images, and the effects of manipulating logits and images.

What Makes for Good Views for Contrastive Learning?

1 code implementation NeurIPS 2020 Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, Phillip Isola

Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning.

Contrastive Learning Data Augmentation +8

Supervised Contrastive Learning

23 code implementations NeurIPS 2020 Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan

Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.

Class Incremental Learning Contrastive Learning +4

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

3 code implementations ECCV 2020 Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, Phillip Isola

The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost.

Few-Shot Image Classification Few-Shot Learning +2

Adversarial Robustness through Local Linearization

no code implementations NeurIPS 2019 Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli

Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with l-infinity adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack.

Adversarial Defense Adversarial Robustness

Contrastive Multiview Coding

8 code implementations ECCV 2020 Yonglong Tian, Dilip Krishnan, Phillip Isola

We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the resulting representation captures underlying scene semantics.

Contrastive Learning Self-Supervised Action Recognition +1

A Closed-Form Learned Pooling for Deep Classification Networks

no code implementations10 Jun 2019 Vighnesh Birodkar, Hossein Mobahi, Dilip Krishnan, Samy Bengio

This operator can learn a strict super-set of what can be learned by average pooling or convolutions.

Classification Foveation +3

Predicting the Generalization Gap in Deep Networks with Margin Distributions

2 code implementations ICLR 2019 Yiding Jiang, Dilip Krishnan, Hossein Mobahi, Samy Bengio

In this paper, we propose such a measure, and conduct extensive empirical studies on how well it can predict the generalization gap.

Synthesizing Normalized Faces from Facial Identity Features

1 code implementation CVPR 2017 Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, William T. Freeman

We present a method for synthesizing a frontal, neutral-expression image of a person's face given an input face photograph.

Domain Separation Networks

5 code implementations NeurIPS 2016 Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, Dumitru Erhan

However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain.

Domain Generalization Unsupervised Domain Adaptation

Learning Ordinal Relationships for Mid-Level Vision

no code implementations ICCV 2015 Daniel Zoran, Phillip Isola, Dilip Krishnan, William T. Freeman

We demonstrate that this frame- work works well on two important mid-level vision tasks: intrinsic image decomposition and depth from an RGB im- age.

Depth Estimation Intrinsic Image Decomposition

Learning visual groups from co-occurrences in space and time

2 code implementations21 Nov 2015 Phillip Isola, Daniel Zoran, Dilip Krishnan, Edward H. Adelson

We propose a self-supervised framework that learns to group visual entities based on their rate of co-occurrence in space and time.

Binary Classification

Reflection Removal Using Ghosting Cues

no code implementations CVPR 2015 YiChang Shih, Dilip Krishnan, Fredo Durand, William T. Freeman

For single-pane windows, ghosting cues arise from shifted reflections on the two surfaces of the glass pane.

Reflection Removal

Shape and Illumination from Shading using the Generic Viewpoint Assumption

no code implementations NeurIPS 2014 Daniel Zoran, Dilip Krishnan, José Bento, Bill Freeman

The Generic Viewpoint Assumption (GVA) states that the position of the viewer or the light in a scene is not special.

Blind Deconvolution with Non-local Sparsity Reweighting

no code implementations16 Nov 2013 Dilip Krishnan, Joan Bruna, Rob Fergus

Blind deconvolution has made significant progress in the past decade.

Fast Image Deconvolution using Hyper-Laplacian Priors

no code implementations NeurIPS 2009 Dilip Krishnan, Rob Fergus

In this paper we describe a deconvolution approach that is several orders of magnitude faster than existing techniques that use hyper-Laplacian priors.

Deblurring Denoising +2

Cannot find the paper you are looking for? You can Submit a new open access paper.