Search Results for author: James Hays

Found 41 papers, 20 papers with code

DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars

no code implementations29 Mar 2022 Amit Raj, Umar Iqbal, Koki Nagano, Sameh Khamis, Pavlo Molchanov, James Hays, Jan Kautz

In this work, we present, DRaCoN, a framework for learning full-body volumetric avatars which exploits the advantages of both the 2D and 3D neural rendering techniques.

Neural Rendering

PressureVision: Estimating Hand Pressure from a Single RGB Image

no code implementations19 Mar 2022 Patrick Grady, Chengcheng Tang, Samarth Brahmbhatt, Christopher D. Twigg, Chengde Wan, James Hays, Charles C. Kemp

We also show that the output of our model depends on the appearance of the hand and cast shadows near contact regions.

CoGS: Controllable Generation and Search from Sketch and Style

no code implementations17 Mar 2022 Cusuh Ham, Gemma Canet Tarres, Tu Bui, James Hays, Zhe Lin, John Collomosse

CoGS enables exploration of diverse appearance possibilities for a given sketched object, enabling decoupled control over the structure and the appearance of the output.

MSeg: A Composite Dataset for Multi-domain Semantic Segmentation

2 code implementations CVPR 2020 John Lambert, Zhuang Liu, Ozan Sener, James Hays, Vladlen Koltun

We adopt zero-shot cross-dataset transfer as a benchmark to systematically evaluate a model's robustness and show that MSeg training yields substantially more robust models in comparison to training on individual datasets or naive mixing of datasets without the presented contributions.

Instance Segmentation Panoptic Segmentation +1

Pixel-Aligned Volumetric Avatars

no code implementations CVPR 2021 Amit Raj, Michael Zollhofer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, Stephen Lombardi

Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.

PVA: Pixel-aligned Volumetric Avatars

no code implementations7 Jan 2021 Amit Raj, Michael Zollhoefer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, Stephen Lombardi

Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.

ANR: Articulated Neural Rendering for Virtual Avatars

no code implementations CVPR 2021 Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, Christoph Lassner

The combination of traditional rendering with neural networks in Deferred Neural Rendering (DNR) provides a compelling balance between computational complexity and realism of the resulting images.

Neural Rendering

Scene Flow from Point Clouds with or without Learning

no code implementations31 Oct 2020 Jhony Kaesemodel Pontes, James Hays, Simon Lucey

Our proposed objective function can be used with or without learning---as a self-supervisory signal to learn scene flow representations, or as a non-learning-based method in which the scene flow is optimized during runtime.

Motion Segmentation Self-Supervised Learning

3D for Free: Crossmodal Transfer Learning using HD Maps

no code implementations24 Aug 2020 Benjamin Wilson, Zsolt Kira, James Hays

In this work, we address the long-tail problem by leveraging both the large class-taxonomies of modern 2D datasets and the robustness of state-of-the-art 2D detection methods.

3D Object Detection Autonomous Driving +4

TIDE: A General Toolbox for Identifying Object Detection Errors

1 code implementation ECCV 2020 Daniel Bolya, Sean Foley, James Hays, Judy Hoffman

We introduce TIDE, a framework and associated toolbox for analyzing the sources of error in object detection and instance segmentation algorithms.

Instance Segmentation object-detection +2

Argoverse: 3D Tracking and Forecasting with Rich Maps

1 code implementation CVPR 2019 Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, James Hays

In our baseline experiments, we illustrate how detailed map information such as lane direction, driveable area, and ground height improves the accuracy of 3D object tracking and motion forecasting.

3D Object Tracking Autonomous Vehicles +3

Towards Markerless Grasp Capture

no code implementations17 Jul 2019 Samarth Brahmbhatt, Charles C. Kemp, James Hays

However, grasp capture - capturing the pose of a hand grasping an object, and orienting it w. r. t.

Hand Pose Estimation

Kernel Mean Matching for Content Addressability of GANs

1 code implementation14 May 2019 Wittawat Jitkrittum, Patsorn Sangkloy, Muhammad Waleed Gondal, Amit Raj, James Hays, Bernhard Schölkopf

We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e. g., a generative adversarial network (GAN).

Image Generation

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

2 code implementations CVPR 2019 Samarth Brahmbhatt, Cusuh Ham, Charles C. Kemp, James Hays

We present ContactDB, a novel dataset of contact maps for household objects that captures the rich hand-object contact that occurs during grasping, enabled by use of a thermal camera.

Grasp Contact Prediction Translation

ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact

4 code implementations7 Apr 2019 Samarth Brahmbhatt, Ankur Handa, James Hays, Dieter Fox

Using a dataset of contact demonstrations from humans grasping diverse household objects, we synthesize functional grasps for three hand models and two functional intents.

Let's Transfer Transformations of Shared Semantic Representations

2 code implementations2 Mar 2019 Nam Vo, Lu Jiang, James Hays

In this work we show how one can learn transformations with no training examples by learning them on another domain and then transfer to the target domain.

Image Retrieval Zero-Shot Learning

Composing Text and Image for Image Retrieval - An Empirical Odyssey

4 code implementations CVPR 2019 Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, James Hays

In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image.

Image Retrieval Image Retrieval with Multi-Modal Query

Informative Features for Model Comparison

3 code implementations NeurIPS 2018 Wittawat Jitkrittum, Heishiro Kanagawa, Patsorn Sangkloy, James Hays, Bernhard Schölkopf, Arthur Gretton

Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models.

SwapNet: Garment Transfer in Single View Images

1 code implementation ECCV 2018 Amit Raj, Patsorn Sangkloy, Huiwen Chang, Jingwan Lu, Duygu Ceylan, James Hays

Garment transfer is a challenging task that requires (i) disentangling the features of the clothing from the body pose and shape and (ii) realistic synthesis of the garment texture on the new body.

 Ranked #1 on Virtual Try-on on FashionIQ (using extra training data)

Virtual Try-on

Generalization in Metric Learning: Should the Embedding Layer be the Embedding Layer?

1 code implementation8 Mar 2018 Nam Vo, James Hays

This work studies deep metric learning under small to medium scale data as we believe that better generalization could be a contributing factor to the improvement of previous fine-grained image retrieval methods; it should be considered when designing future techniques.

Image Retrieval Metric Learning

Let's Dance: Learning From Online Dance Videos

1 code implementation23 Jan 2018 Daniel Castro, Steven Hickson, Patsorn Sangkloy, Bhavishya Mittal, Sean Dai, James Hays, Irfan Essa

We present a comparison of numerous state-of-the-art techniques on our dataset using three different representations (video, optical flow and multi-person pose data) in order to analyze these approaches.

Action Recognition Optical Flow Estimation

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

no code implementations CVPR 2018 Wengling Chen, James Hays

Synthesizing realistic images from human drawn sketches is a challenging problem in computer graphics and vision.

Data Augmentation Image Generation +1

Geometry-Aware Learning of Maps for Camera Localization

1 code implementation CVPR 2018 Samarth Brahmbhatt, Jinwei Gu, Kihwan Kim, James Hays, Jan Kautz

Maps are a key component in image-based camera localization and visual SLAM systems: they are used to establish geometric constraints between images, correct drift in relative pose estimation, and relocalize cameras after lost tracking.

Camera Localization

On Convergence and Stability of GANs

8 code implementations ICLR 2018 Naveen Kodali, Jacob Abernethy, James Hays, Zsolt Kira

We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions.

Revisiting IM2GPS in the Deep Learning Era

no code implementations ICCV 2017 Nam Vo, Nathan Jacobs, James Hays

The recent state-of-the-art approach to this problem is a deep image classification approach in which the world is spatially divided into cells and a deep network is trained to predict the correct cell for a given image.

Ranked #4 on Photo geolocation estimation on Im2GPS (using extra training data)

Classification Density Estimation +2

DeepNav: Learning to Navigate Large Cities

no code implementations CVPR 2017 Samarth Brahmbhatt, James Hays

We present DeepNav, a Convolutional Neural Network (CNN) based algorithm for navigating large cities using locally visible street-view images.

Super-resolution Using Constrained Deep Texture Synthesis

no code implementations26 Jan 2017 Libin Sun, James Hays

Hallucinating high frequency image details in single image super-resolution is a challenging task.

Image Super-Resolution Single Image Super Resolution +1

Complex Event Recognition from Images with Few Training Examples

no code implementations17 Jan 2017 Unaiza Ahsan, Chen Sun, James Hays, Irfan Essa

We propose to leverage concept-level representations for complex event recognition in photographs given limited training examples.

Scribbler: Controlling Deep Image Synthesis with Sketch and Color

1 code implementation CVPR 2017 Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, James Hays

In this paper, we propose a deep adversarial image synthesis architecture that is conditioned on sketched boundaries and sparse color strokes to generate realistic cars, bedrooms, or faces.

Colorization Image Generation

StuffNet: Using 'Stuff' to Improve Object Detection

1 code implementation19 Oct 2016 Samarth Brahmbhatt, Henrik I. Christensen, James Hays

Through experiments on Pascal VOC 2007 and 2012, we demonstrate the effectiveness of this method and show that StuffNet also significantly improves object detection performance on such datasets.

object-detection Object Detection +1

COCO Attributes: Attributes for People, Animals, and Objects

1 code implementation ECCV 2016 Genevieve Patterson, James Hays

In this paper, we discover and annotate visual attributes for the COCO dataset.

Localizing and Orienting Street Views Using Overhead Imagery

no code implementations30 Jul 2016 Nam Vo, James Hays

In this paper we aim to determine the location and orientation of a ground-level query image by matching to a reference database of overhead (e. g. satellite) images.

General Classification

Learning to Match Aerial Images With Deep Attentive Architectures

no code implementations CVPR 2016 Hani Altwaijry, Eduard Trulls, James Hays, Pascal Fua, Serge Belongie

We demonstrate that our models outperform the state-of-the-art on ultra-wide baseline matching and approach human accuracy.

Solving Small-Piece Jigsaw Puzzles by Growing Consensus

no code implementations CVPR 2016 Kilho Son, Daniel Moreno, James Hays, David B. Cooper

To reconstruct such challenging puzzles, we aim to search for piece configurations which maximize the size of consensus (i. e. grid or loop) configurations which represent a geometric consensus or agreement among pieces.

Lens Factory: Automatic Lens Generation Using Off-the-shelf Components

no code implementations30 Jun 2015 Libin Sun, Brian Guenter, Neel Joshi, Patrick Therien, James Hays

Unfortunately, custom lens design is costly (thousands to tens of thousands of dollars), time consuming (10-12 weeks typical lead time), and requires specialized optics design expertise.

Learning Deep Representations for Ground-to-Aerial Geolocalization

no code implementations CVPR 2015 Tsung-Yi Lin, Yin Cui, Serge Belongie, James Hays

Most approaches predict the location of a query image by matching to ground-level images with known locations (e. g., street-view data).

Face Verification

Microsoft COCO: Common Objects in Context

26 code implementations1 May 2014 Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding.

Instance Segmentation Object Localization +3

Cross-View Image Geolocalization

no code implementations CVPR 2013 Tsung-Yi Lin, Serge Belongie, James Hays

On the other hand, there is no shortage of visual and geographic data that densely covers the Earth we examine overhead imagery and land cover survey data but the relationship between this data and ground level query photographs is complex.

FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps

no code implementations CVPR 2013 Yinda Zhang, Jianxiong Xiao, James Hays, Ping Tan

We analyze the self-similarity of the guide image to generate a set of allowable local transformations and apply them to the input image.

Image Generation

Cannot find the paper you are looking for? You can Submit a new open access paper.