Search Results for author: Judy Hoffman

Found 57 papers, 25 papers with code

PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization

1 code implementation2 Dec 2022 Prithvijit Chattopadhyay, Kartik Sarangmath, Vivek Vijaykumar, Judy Hoffman

Synthetic data offers the promise of cheap and bountiful training data for settings where lots of labeled real-world data for tasks is unavailable.

Domain Generalization object-detection +3

Signed Binary Weight Networks: Improving Efficiency of Binary Weight Networks by Exploiting Sparsity

no code implementations25 Nov 2022 Sachit Kuhar, Alexey Tumanov, Judy Hoffman

We propose a new method called signed-binary networks to improve further efficiency (by exploiting both weight sparsity and weight repetition) while maintaining similar accuracy.

Binarization

Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigation

no code implementations20 Nov 2022 Chia-Wen Kuo, Chih-Yao Ma, Judy Hoffman, Zsolt Kira

In Vision-and-Language Navigation (VLN), researchers typically take an image encoder pre-trained on ImageNet without fine-tuning on the environments that the agent will be trained or tested on.

Test unseen Vision and Language Navigation

Token Merging: Your ViT But Faster

1 code implementation17 Oct 2022 Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, Judy Hoffman

Off-the-shelf, ToMe can 2x the throughput of state-of-the-art ViT-L @ 512 and ViT-H @ 518 models on images and 2. 2x the throughput of ViT-L on video with only a 0. 2-0. 3% accuracy drop in each case.

Hydra Attention: Efficient Attention with Many Heads

no code implementations15 Sep 2022 Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Judy Hoffman

While transformers have begun to dominate many tasks in vision, applying them to large images is still computationally difficult.

ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings

no code implementations24 Jun 2022 Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman, Dhruv Batra

We present a scalable approach for learning open-world object-goal navigation (ObjectNav) -- the task of asking a virtual robot (agent) to find any instance of an object in an unexplored environment (e. g., "find a sink").

Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency

1 code implementation16 Jun 2022 Viraj Prabhu, Sriram Yenamandra, Aaditya Singh, Judy Hoffman

Inspired by the design of recent SSL approaches based on learning from partial image inputs generated via masking or cropping -- either by learning to predict the missing pixels, or learning representational invariances to such augmentations -- we propose PACMAC, a simple two-stage adaptation algorithm for self-supervised ViTs.

Domain Adaptation Object Recognition +1

Can domain adaptation make object recognition work for everyone?

no code implementations23 Apr 2022 Viraj Prabhu, Ramprasaath R. Selvaraju, Judy Hoffman, Nikhil Naik

Despite the rapid progress in deep visual recognition, modern computer vision datasets significantly overrepresent the developed world and models trained on such datasets underperform on images from unseen geographies.

Object Recognition Unsupervised Domain Adaptation

ConceptEvo: Interpreting Concept Evolution in Deep Learning Training

no code implementations30 Mar 2022 Haekyu Park, Seongmin Lee, Benjamin Hoover, Austin Wright, Omar Shaikh, Rahul Duggal, Nilaksh Das, Judy Hoffman, Duen Horng Chau

Deep neural networks (DNNs) have been widely used for decision making, prompting a surge of interest in interpreting how these complex models work.

Decision Making

Scalable Diverse Model Selection for Accessible Transfer Learning

no code implementations NeurIPS 2021 Daniel Bolya, Rohit Mittapalli, Judy Hoffman

In this paper, we formalize this setting as "Scalable Diverse Model Selection" and propose several benchmarks for evaluating on this task.

Model Selection Transfer Learning

UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models

1 code implementation29 Oct 2021 Arvindkumar Krishnakumar, Viraj Prabhu, Sruthi Sudhakar, Judy Hoffman

Deep learning models have been shown to learn spurious correlations from data that sometimes lead to systematic failures for certain subpopulations.

Image Classification

AUGCO: Augmentation Consistency-guided Self-training for Source-free Domain Adaptive Semantic Segmentation

no code implementations21 Jul 2021 Viraj Prabhu, Shivam Khare, Deeksha Kartik, Judy Hoffman

Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy constraints.

Semantic Segmentation Source-Free Domain Adaptation

RobustNav: Towards Benchmarking Robustness in Embodied Navigation

1 code implementation ICCV 2021 Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, Aniruddha Kembhavi

As an attempt towards assessing the robustness of embodied navigation agents, we propose RobustNav, a framework to quantify the performance of embodied navigation agents when exposed to a wide variety of visual - affecting RGB inputs - and dynamics - affecting transition dynamics - corruptions.

Data Augmentation Visual Navigation

SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation

1 code implementation ICCV 2021 Viraj Prabhu, Shivam Khare, Deeksha Kartik, Judy Hoffman

Many existing approaches for unsupervised domain adaptation (UDA) focus on adapting under only data distribution shift and offer limited success under additional cross-domain label distribution shift.

Pseudo Label Unsupervised Domain Adaptation

Temporal Action Detection with Multi-level Supervision

no code implementations ICCV 2021 Baifeng Shi, Qi Dai, Judy Hoffman, Kate Saenko, Trevor Darrell, Huijuan Xu

We extensively benchmark against the baselines for SSAD and OSAD on our created data splits in THUMOS14 and ActivityNet1. 2, and demonstrate the effectiveness of the proposed UFA and IB methods.

Action Detection Semi-Supervised Action Detection

Auxiliary Task Reweighting for Minimum-data Learning

no code implementations NeurIPS 2020 Baifeng Shi, Judy Hoffman, Kate Saenko, Trevor Darrell, Huijuan Xu

By adjusting the auxiliary task weights to minimize the divergence between the surrogate prior and the true prior of the main task, we obtain a more accurate prior estimation, achieving the goal of minimizing the required amount of training data for the main task and avoiding a costly grid search.

Domain Adaptation Multi-Label Classification

Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents

no code implementations7 Sep 2020 Samyak Datta, Oleksandr Maksymets, Judy Hoffman, Stefan Lee, Dhruv Batra, Devi Parikh

This enables a seamless adaption to changing dynamics (a different robot or floor type) by simply re-calibrating the visual odometry model -- circumventing the expense of re-training of the navigation policy.

Navigate Robot Navigation +1

Learning to Balance Specificity and Invariance for In and Out of Domain Generalization

1 code implementation ECCV 2020 Prithvijit Chattopadhyay, Yogesh Balaji, Judy Hoffman

For domain generalization, the goal is to learn from a set of source domains to produce a single model that will best generalize to an unseen target domain.

Domain Generalization Specificity

Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses

no code implementations25 Aug 2020 Fu Lin, Rohit Mittapalli, Prithvijit Chattopadhyay, Daniel Bolya, Judy Hoffman

Convolutional Neural Networks have been shown to be vulnerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability.

Adversarial Defense Adversarial Robustness

TIDE: A General Toolbox for Identifying Object Detection Errors

1 code implementation ECCV 2020 Daniel Bolya, Sean Foley, James Hays, Judy Hoffman

We introduce TIDE, a framework and associated toolbox for analyzing the sources of error in object detection and instance segmentation algorithms.

Instance Segmentation object-detection +2

Analyzing Visual Representations in Embodied Navigation Tasks

no code implementations12 Mar 2020 Erik Wijmans, Julian Straub, Dhruv Batra, Irfan Essa, Judy Hoffman, Ari Morcos

Recent advances in deep reinforcement learning require a large amount of training data and generally result in representations that are often over specialized to the target task.

reinforcement Learning

Representation Learning Through Latent Canonicalizations

no code implementations26 Feb 2020 Or Litany, Ari Morcos, Srinath Sridhar, Leonidas Guibas, Judy Hoffman

We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.

Disentanglement

Insights on Visual Representations for Embodied Navigation Tasks

no code implementations ICLR 2020 Erik Wijmans, Julian Straub, Irfan Essa, Dhruv Batra, Judy Hoffman, Ari Morcos

Surprisingly, we find that slight differences in task have no measurable effect on the visual representation for both SqueezeNet and ResNet architectures.

Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets

1 code implementation17 Oct 2019 Yogesh Balaji, Tom Goldstein, Judy Hoffman

Adversarial training is by far the most successful strategy for improving robustness of neural networks to adversarial attacks.

Predictive Inequity in Object Detection

1 code implementation21 Feb 2019 Benjamin Wilson, Judy Hoffman, Jamie Morgenstern

In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones.

object-detection Object Detection

Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation

no code implementations26 Jun 2018 Xingchao Peng, Ben Usman, Kuniaki Saito, Neela Kaushik, Judy Hoffman, Kate Saenko

In this paper, we present a new large-scale benchmark called Syn2Real, which consists of a synthetic domain rendered from 3D object models and two real-image domains containing the same object categories.

Classification Domain Adaptation +4

Algorithms and Theory for Multiple-Source Adaptation

no code implementations NeurIPS 2018 Judy Hoffman, Mehryar Mohri, Ningshan Zhang

This work includes a number of novel contributions for the multiple-source adaptation problem.

Multiple-Source Adaptation for Regression Problems

no code implementations14 Nov 2017 Judy Hoffman, Mehryar Mohri, Ningshan Zhang

We present a detailed theoretical analysis of the problem of multiple-source adaptation in the general stochastic scenario, extending known results that assume a single target labeling function.

regression Sentiment Analysis

VisDA: The Visual Domain Adaptation Challenge

1 code implementation18 Oct 2017 Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, Kate Saenko

We present the 2017 Visual Domain Adaptation (VisDA) dataset and challenge, a large-scale testbed for unsupervised domain adaptation across visual domains.

General Classification Image Classification +3

Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approach

no code implementations ICCV 2017 Timnit Gebru, Judy Hoffman, Li Fei-Fei

While fine-grained object recognition is an important problem in computer vision, current models are unlikely to accurately classify objects in the wild.

Domain Adaptation Object Recognition

Inferring and Executing Programs for Visual Reasoning

5 code implementations ICCV 2017 Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, Ross Girshick

Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes.

Visual Question Answering Visual Reasoning

Adversarial Discriminative Domain Adaptation

18 code implementations CVPR 2017 Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell

Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains.

General Classification Unsupervised Domain Adaptation +1

FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation

4 code implementations8 Dec 2016 Judy Hoffman, Dequan Wang, Fisher Yu, Trevor Darrell

In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems.

Semantic Segmentation Synthetic-to-Real Translation

Clockwork Convnets for Video Semantic Segmentation

1 code implementation11 Aug 2016 Evan Shelhamer, Kate Rakelly, Judy Hoffman, Trevor Darrell

Recent years have seen tremendous progress in still-image segmentation; however the na\"ive application of these state-of-the-art algorithms to every video frame requires considerable computation and ignores the temporal continuity inherent in video.

Image Segmentation Scheduling +3

Learning With Side Information Through Modality Hallucination

no code implementations CVPR 2016 Judy Hoffman, Saurabh Gupta, Trevor Darrell

Thus, our method transfers information commonly extracted from depth training data to a network which can extract that information from the RGB counterpart.

object-detection Object Detection

Fine-to-coarse Knowledge Transfer For Low-Res Image Classification

no code implementations21 May 2016 Xingchao Peng, Judy Hoffman, Stella X. Yu, Kate Saenko

We address the difficult problem of distinguishing fine-grained object categories in low resolution images.

Classification General Classification +2

Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

no code implementations23 Nov 2015 Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Pieter Abbeel, Sergey Levine, Kate Saenko, Trevor Darrell

We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains.

Domain Adaptation

Spatial Semantic Regularisation for Large Scale Object Detection

no code implementations ICCV 2015 Damian Mrowca, Marcus Rohrbach, Judy Hoffman, Ronghang Hu, Kate Saenko, Trevor Darrell

Our approach proves to be especially useful in large scale settings with thousands of classes, where spatial and semantic interactions are very frequent and only weakly supervised detectors can be built due to a lack of bounding box annotations.

object-detection Object Detection

Simultaneous Deep Transfer Across Domains and Tasks

1 code implementation ICCV 2015 Eric Tzeng, Judy Hoffman, Trevor Darrell, Kate Saenko

Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias.

Domain Adaptation

Cross Modal Distillation for Supervision Transfer

1 code implementation CVPR 2016 Saurabh Gupta, Judy Hoffman, Jitendra Malik

In this work we propose a technique that transfers supervision between images from different modalities.

Optical Flow Estimation

Deep Domain Confusion: Maximizing for Domain Invariance

7 code implementations10 Dec 2014 Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell

Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark.

Domain Adaptation Model Selection

Detector Discovery in the Wild: Joint Multiple Instance and Representation Learning

no code implementations CVPR 2015 Judy Hoffman, Deepak Pathak, Trevor Darrell, Kate Saenko

We develop methods for detector learning which exploit joint training over both weak and strong labels and which transfer learned perceptual representations from strongly-labeled auxiliary tasks.

Multiple Instance Learning Representation Learning +1

Continuous Manifold Based Adaptation for Evolving Visual Domains

no code implementations CVPR 2014 Judy Hoffman, Trevor Darrell, Kate Saenko

The classic domain adaptation paradigm considers the world to be separated into stationary domains with clear boundaries between them.

Domain Adaptation

One-Shot Adaptation of Supervised Deep Convolutional Models

no code implementations21 Dec 2013 Judy Hoffman, Eric Tzeng, Jeff Donahue, Yangqing Jia, Kate Saenko, Trevor Darrell

In other words, are deep CNNs trained on large amounts of labeled data as susceptible to dataset bias as previous methods have been shown to be?

Domain Adaptation Image Classification

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

8 code implementations6 Oct 2013 Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed to novel generic tasks.

Domain Adaptation Object Recognition +1

Efficient Learning of Domain-invariant Image Representations

no code implementations15 Jan 2013 Judy Hoffman, Erik Rodner, Jeff Donahue, Trevor Darrell, Kate Saenko

We present an algorithm that learns representations which explicitly compensate for domain mismatch and which can be efficiently realized as linear classifiers.

Representation Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.