Search Results for author: Kayvon Fatahalian

Found 22 papers, 12 papers with code

Block and Detail: Scaffolding Sketch-to-Image Generation

no code implementations28 Feb 2024 Vishnu Sarukkai, Lu Yuan, Mia Tang, Maneesh Agrawala, Kayvon Fatahalian

Our tool lets users sketch blocking strokes to coarsely represent the placement and form of objects and detail strokes to refine their shape and silhouettes.

Blocking Image Generation

Learning Subject-Aware Cropping by Outpainting Professional Photos

no code implementations19 Dec 2023 James Hong, Lu Yuan, Michaël Gharbi, Matthew Fisher, Kayvon Fatahalian

The stock image collection provides diversity and its images serve as pseudo-labels for a good crop, while the text-image diffusion model is used to out-paint (i. e., outward inpainting) realistic uncropped images.

Image Cropping

Iterative Motion Editing with Natural Language

no code implementations15 Dec 2023 Purvi Goel, Kuan-Chieh Wang, C. Karen Liu, Kayvon Fatahalian

Text-to-motion diffusion models can generate realistic animations from text prompts, but do not support fine-grained motion editing controls.

Collage Diffusion

no code implementations1 Mar 2023 Vishnu Sarukkai, Linden Li, Arden Ma, Christopher Ré, Kayvon Fatahalian

We seek to give users precise control over diffusion-based image generation by modeling complex scenes as sequences of layers, which define the desired spatial arrangement and visual attributes of objects in the scene.

Conditional Image Generation Image Harmonization

Spotting Temporally Precise, Fine-Grained Events in Video

2 code implementations20 Jul 2022 James Hong, Haotian Zhang, Michaël Gharbi, Matthew Fisher, Kayvon Fatahalian

We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur).

Action Detection Action Spotting +2

Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision

1 code implementation24 Mar 2022 Mayee F. Chen, Daniel Y. Fu, Dyah Adila, Michael Zhang, Frederic Sala, Kayvon Fatahalian, Christopher Ré

Despite the black-box nature of foundation models, we prove results characterizing how our approach improves performance and show that lift scales with the smoothness of label distributions in embedding space.

The Details Matter: Preventing Class Collapse in Supervised Contrastive Learning

no code implementations29 Sep 2021 Daniel Yang Fu, Mayee F Chen, Michael Zhang, Kayvon Fatahalian, Christopher Ré

Supervised contrastive learning optimizes a loss that pushes together embeddings of points from the same class while pulling apart embeddings of points from different classes.

Contrastive Learning Transfer Learning

Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories

no code implementations ICCV 2021 Fait Poms, Vishnu Sarukkai, Ravi Teja Mullapudi, Nimit S. Sohoni, William R. Mark, Deva Ramanan, Kayvon Fatahalian

For machine learning models trained with limited labeled training data, validation stands to become the main bottleneck to reducing overall annotation costs.

Mandoline: Model Evaluation under Distribution Shift

1 code implementation1 Jul 2021 Mayee Chen, Karan Goel, Nimit S. Sohoni, Fait Poms, Kayvon Fatahalian, Christopher Ré

If an unlabeled sample from the target distribution is available, along with a labeled sample from a possibly different source distribution, standard approaches such as importance weighting can be applied to estimate performance on the target.

Density Ratio Estimation Epidemiology

Large Batch Simulation for Deep Reinforcement Learning

1 code implementation ICLR 2021 Brennan Shacklett, Erik Wijmans, Aleksei Petrenko, Manolis Savva, Dhruv Batra, Vladlen Koltun, Kayvon Fatahalian

We accelerate deep reinforcement learning-based training in visually complex 3D environments by two orders of magnitude over prior work, realizing end-to-end training speeds of over 19, 000 frames of experience per second on a single GPU and up to 72, 000 frames per second on a single eight-GPU machine.

PointGoal Navigation reinforcement-learning +1

Learning Rare Category Classifiers on a Tight Labeling Budget

no code implementations ICCV 2021 Ravi Teja Mullapudi, Fait Poms, William R. Mark, Deva Ramanan, Kayvon Fatahalian

In this paper, we consider the scenario where we start with as-little-as five labeled positives of a rare category and a large amount of unlabeled data of which 99. 9% of it is negatives.

Active Learning Representation Learning

Iterative Text-based Editing of Talking-heads Using Neural Retargeting

no code implementations21 Nov 2020 Xinwei Yao, Ohad Fried, Kayvon Fatahalian, Maneesh Agrawala

We present a text-based tool for editing talking-head video that enables an iterative editing workflow.

Background Splitting: Finding Rare Classes in a Sea of Background

1 code implementation CVPR 2021 Ravi Teja Mullapudi, Fait Poms, William R. Mark, Deva Ramanan, Kayvon Fatahalian

We focus on the real-world problem of training accurate deep models for image classification of a small number of rare categories.

Image Classification

Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods

1 code implementation ICML 2020 Daniel Y. Fu, Mayee F. Chen, Frederic Sala, Sarah M. Hooper, Kayvon Fatahalian, Christopher Ré

In this work, we show that, for a class of latent variable models highly applicable to weak supervision, we can find a closed-form solution to model parameters, obviating the need for iterative solutions like stochastic gradient descent (SGD).

Multi-Resolution Weak Supervision for Sequential Data

no code implementations NeurIPS 2019 Frederic Sala, Paroma Varma, Jason Fries, Daniel Y. Fu, Shiori Sagawa, Saelig Khattar, Ashwini Ramamoorthy, Ke Xiao, Kayvon Fatahalian, James Priest, Christopher Ré

Multi-resolution sources exacerbate this challenge due to complex correlations and sample complexity that scales in the length of the sequence.

Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels

1 code implementation7 Oct 2019 Daniel Y. Fu, Will Crichton, James Hong, Xinwei Yao, Haotian Zhang, Anh Truong, Avanika Narayan, Maneesh Agrawala, Christopher Ré, Kayvon Fatahalian

Many real-world video analysis applications require the ability to identify domain-specific events in video, such as interviews and commercials in TV news broadcasts, or action sequences in film.

Online Model Distillation for Efficient Video Inference

1 code implementation ICCV 2019 Ravi Teja Mullapudi, Steven Chen, Keyi Zhang, Deva Ramanan, Kayvon Fatahalian

Rather than learn a specialized student model on offline data from the video stream, we train the student in an online fashion on the live video, intermittently running the teacher to provide a target for learning.

Segmentation Semantic Segmentation +2

HydraNets: Specialized Dynamic Architectures for Efficient Inference

no code implementations CVPR 2018 Ravi Teja Mullapudi, William R. Mark, Noam Shazeer, Kayvon Fatahalian

On ImageNet, applying the HydraNet template improves accuracy up to 2. 5% when compared to an efficient baseline architecture with similar inference cost.

Classification Computational Efficiency +2

Scanner: Efficient Video Analysis at Scale

1 code implementation18 May 2018 Alex Poms, Will Crichton, Pat Hanrahan, Kayvon Fatahalian

The challenge is that scaling applications to operate on these datasets requires efficient systems for pixel data access and parallel processing across large numbers of machines.

Cannot find the paper you are looking for? You can Submit a new open access paper.