Search Results for author: Jonathan Huang

Found 36 papers, 14 papers with code

Text and Click inputs for unambiguous open vocabulary instance segmentation

1 code implementation24 Nov 2023 Nikolai Warner, Meera Hahn, Jonathan Huang, Irfan Essa, Vighnesh Birodkar

We propose a new segmentation process, Text + Click segmentation, where a model takes as input an image, a text phrase describing a class to segment, and a single foreground click specifying the instance to segment.

Instance Segmentation Segmentation +1

Optimizing ViViT Training: Time and Memory Reduction for Action Recognition

no code implementations7 Jun 2023 Shreyank N Gowda, Anurag Arnab, Jonathan Huang

In this paper, we address the challenges posed by the substantial training time and memory consumption associated with video transformers, focusing on the ViViT (Video Vision Transformer) model, in particular the Factorised Encoder version, as our baseline for action recognition tasks.

Action Recognition

The Auto Arborist Dataset: A Large-Scale Benchmark for Multiview Urban Forest Monitoring Under Domain Shift

no code implementations CVPR 2022 Sara Beery, Guanhang Wu, Trevor Edwards, Filip Pavetic, Bo Majewski, Shreyasee Mukherjee, Stanley Chan, John Morgan, Vivek Rathod, Jonathan Huang

We introduce baseline results on our dataset across modalities as well as metrics for the detailed analysis of generalization with respect to geographic distribution shifts, vital for such a system to be deployed at-scale.


The surprising impact of mask-head architecture on novel class segmentation

3 code implementations ICCV 2021 Vighnesh Birodkar, Zhichao Lu, Siyang Li, Vivek Rathod, Jonathan Huang

Under this family, we study Mask R-CNN and discover that instead of its default strategy of training the mask-head with a combination of proposals and groundtruth boxes, training the mask-head with only groundtruth boxes dramatically improves its performance on novel classes.

Instance Segmentation Segmentation +1

PERF-Net: Pose Empowered RGB-Flow Net

no code implementations28 Sep 2020 Yinxiao Li, Zhichao Lu, Xuehan Xiong, Jonathan Huang

In recent years, many works in the video action recognition literature have shown that two stream models (combining spatial and temporal input streams) are necessary for achieving state of the art performance.

Action Classification Action Recognition +1

Compact Speaker Embedding: lrx-vector

no code implementations11 Aug 2020 Munir Georges, Jonathan Huang, Tobias Bocklet

Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks.

Knowledge Distillation Speaker Recognition

RetinaTrack: Online Single Stage Joint Detection and Tracking

1 code implementation CVPR 2020 Zhichao Lu, Vivek Rathod, Ronny Votel, Jonathan Huang

Traditionally multi-object tracking and object detection are performed using separate systems with most prior works focusing exclusively on one of these aspects over the other.

Autonomous Driving Multi-Object Tracking +3

Exploring Context, Attention and Audio Features for Audio Visual Scene-Aware Dialog

no code implementations20 Dec 2019 Shachi H. Kumar, Eda Okur, Saurav Sahay, Jonathan Huang, Lama Nachman

Recent progress in visual grounding techniques and Audio Understanding are enabling machines to understand shared semantic concepts and listen to the various sensory events in the environment.

Audio Classification Visual Grounding

Leveraging Topics and Audio Features with Multimodal Attention for Audio Visual Scene-Aware Dialog

no code implementations20 Dec 2019 Shachi H. Kumar, Eda Okur, Saurav Sahay, Jonathan Huang, Lama Nachman

With the recent advancements in Artificial Intelligence (AI), Intelligent Virtual Assistants (IVA) such as Alexa, Google Home, etc., have become a ubiquitous part of many homes.

Audio Classification Response Generation

Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection

3 code implementations CVPR 2020 Sara Beery, Guanhang Wu, Vivek Rathod, Ronny Votel, Jonathan Huang

In this paper we propose a method that leverages temporal context from the unlabeled frames of a novel camera to improve performance at that camera.

object-detection Video Object Detection +1

Uncertainty aware audiovisual activity recognition using deep Bayesian variational inference

no code implementations27 Nov 2018 Mahesh Subedar, Ranganath Krishnan, Paulo Lopez Meyer, Omesh Tickoo, Jonathan Huang

In the multimodal setting, the proposed framework improved precision-recall AUC by 10. 2% on the subset of MiT dataset as compared to non-Bayesian baseline.

Bayesian Inference Multimodal Activity Recognition +1

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

1 code implementation ECCV 2018 Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, Kevin Murphy

Despite the steady progress in video analysis led by the adoption of convolutional neural networks (CNNs), the relative improvement has been less drastic as that in 2D static image classification.

Ranked #27 on Action Recognition on UCF101 (using extra training data)

Action Classification Action Detection +6

Progressive Neural Architecture Search

18 code implementations ECCV 2018 Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms.

Evolutionary Algorithms General Classification +3

Generative Models of Visually Grounded Imagination

no code implementations ICLR 2018 Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin Murphy

It is easy for people to imagine what a man with pink hair looks like, even if they have never seen such a person before.


Motion Prediction Under Multimodality with Conditional Stochastic Networks

no code implementations5 May 2017 Katerina Fragkiadaki, Jonathan Huang, Alex Alemi, Sudheendra Vijayanarasimhan, Susanna Ricco, Rahul Sukthankar

In this work, we present stochastic neural network architectures that handle such multimodality through stochasticity: future trajectories of objects, body joints or frames are represented as deep, non-linear transformations of random (as opposed to deterministic) variables.

motion prediction Optical Flow Estimation +2

Spatially Adaptive Computation Time for Residual Networks

1 code implementation CVPR 2017 Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, Ruslan Salakhutdinov

This paper proposes a deep learning architecture based on Residual Network that dynamically adjusts the number of executed layers for the regions of the image.

Classification Computational Efficiency +7

Speed/accuracy trade-offs for modern convolutional object detectors

14 code implementations CVPR 2017 Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang song, Sergio Guadarrama, Kevin Murphy

On the opposite end in which accuracy is critical, we present a detector that achieves state-of-the-art performance measured on the COCO detection task.

Ranked #209 on Object Detection on COCO test-dev (using extra training data)

Object object-detection +1

Im2Calories: Towards an Automated Mobile Vision Food Diary

no code implementations ICCV 2015 Austin Meyers, Nick Johnston, Vivek Rathod, Anoop Korattikara, Alex Gorban, Nathan Silberman, Sergio Guadarrama, George Papandreou, Jonathan Huang, Kevin P. Murphy

We present a system which can recognize the contents of your meal from a single image, and then predict its nutritional contents, such as calories.

Efficient inference in occlusion-aware generative models of images

no code implementations19 Nov 2015 Jonathan Huang, Kevin Murphy

We present a generative model of images based on layering, in which image layers are individually generated, then composited from front to back.


Detecting events and key actors in multi-person videos

no code implementations CVPR 2016 Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, Li Fei-Fei

In this paper, we propose a model which learns to detect events in such videos while automatically "attending" to the people responsible for the event.

Event Detection General Classification

Generation and Comprehension of Unambiguous Object Descriptions

1 code implementation CVPR 2016 Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, Kevin Murphy

We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described.

Image Captioning Object +1

Deep Knowledge Tracing

6 code implementations NeurIPS 2015 Chris Piech, Jonathan Spencer, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, Jascha Sohl-Dickstein

Knowledge tracing---where a machine models the knowledge of a student as they interact with coursework---is a well established problem in computer supported education.

Knowledge Tracing

Learning Program Embeddings to Propagate Feedback on Student Code

no code implementations22 May 2015 Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, Leonidas Guibas

Providing feedback, both assessing final work and giving hints to stuck students, is difficult for open-ended assignments in massive online classes which can range from thousands to millions of students.

What's Cookin'? Interpreting Cooking Videos using Text, Speech and Vision

1 code implementation5 Mar 2015 Jonathan Malmaud, Jonathan Huang, Vivek Rathod, Nick Johnston, Andrew Rabinovich, Kevin Murphy

We present a novel method for aligning a sequence of instructions to a video of someone carrying out a task.

Keyword Spotting

Riffled Independence for Efficient Inference with Partial Rankings

no code implementations23 Jan 2014 Jonathan Huang, Ashish Kapoor, Carlos Guestrin

Simultaneously addressing all of these challenges i. e., designing a compactly representable model which is amenable to efficient inference and can be learned using partial ranking data is a difficult task, but is necessary if we would like to scale to problems with nontrivial size.

Tuned Models of Peer Assessment in MOOCs

no code implementations9 Jul 2013 Chris Piech, Jonathan Huang, Zhenghao Chen, Chuong Do, Andrew Ng, Daphne Koller

In massive open online courses (MOOCs), peer grading serves as a critical tool for scaling the grading of complex, open-ended assignments to courses with tens or hundreds of thousands of students.

Probabilistic Event Cascades for Alzheimer's disease

no code implementations NeurIPS 2012 Jonathan Huang, Daniel Alexander

Accurate and detailed models of the progression of neurodegenerative diseases such as Alzheimer's (AD) are crucially important for reliable early diagnosis and the determination and deployment of effective treatments.

Riffled Independence for Ranked Data

no code implementations NeurIPS 2009 Jonathan Huang, Carlos Guestrin

Representing distributions over permutations can be a daunting task due to the fact that the number of permutations of n objects scales factorially in n. One recent way that has been used to reduce storage complexity has been to exploit probabilistic independence, but as we argue, full independence assumptions impose strong sparsity constraints on distributions and are unsuitable for modeling rankings.

Card Games

Cannot find the paper you are looking for? You can Submit a new open access paper.