Search Results for author: Kevin Murphy

Found 66 papers, 41 papers with code

SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs

no code implementations NeurIPS 2023 Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David A. Ross, Irfan Essa, Yonatan Bisk, Ming-Hsuan Yang, Kevin Murphy, Alexander G. Hauptmann, Lu Jiang

In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such as images or videos.

In-Context Learning multimodal generation

Low-rank extended Kalman filtering for online learning of neural networks from streaming data

1 code implementation31 May 2023 Peter G. Chang, Gerardo Durán-Martín, Alexander Y Shestopaloff, Matt Jones, Kevin Murphy

We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream.

Bayesian Inference Variational Inference

Muse: Text-To-Image Generation via Masked Generative Transformers

4 code implementations2 Jan 2023 Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, Dilip Krishnan

Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.

Language Modelling Large Language Model +1

Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlations

1 code implementation28 Nov 2022 Qingyao Sun, Kevin Murphy, Sayna Ebrahimi, Alexander D'Amour

However, we assume that the generative model for features $p(x|y, z)$ is invariant across domains.

Efficient Online Bayesian Inference for Neural Bandits

1 code implementation1 Dec 2021 Gerardo Duran-Martin, Aleyna Kara, Kevin Murphy

In this paper we present a new algorithm for online (sequential) inference in Bayesian neural networks, and show its suitability for tackling contextual bandit problems.

Bayesian Inference Recommendation Systems

Risk score learning for COVID-19 contact tracing apps

1 code implementation17 Apr 2021 Kevin Murphy, Abhishek Kumar, Stylianos Serghiou

Although this data is already being collected (in an aggregated, privacy-preserving way) by several health authorities, in this paper we limit ourselves to simulated data, so that we can systematically study the different factors that affect the feasibility of the approach.

Privacy Preserving

HOPPY: An open-source and low-cost kit for dynamic robotics education

1 code implementation27 Oct 2020 Joao Ramos, Yanran Ding, Young-woo Sim, Kevin Murphy, Daniel Block

This letter introduces HOPPY, an open-source, low-cost, robust, and modular kit for robotics education.

Robotics

Population-Based Black-Box Optimization for Biological Sequence Design

no code implementations ICML 2020 Christof Angermueller, David Belanger, Andreea Gane, Zelda Mariet, David Dohan, Kevin Murphy, Lucy Colwell, D. Sculley

The cost and latency of wet-lab experiments requires methods that find good sequences in few experimental rounds of large batches of sequences--a setting that off-the-shelf black-box optimization methods are ill-equipped to handle.

Machine Learning on Graphs: A Model and Comprehensive Taxonomy

1 code implementation7 May 2020 Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, Kevin Murphy

The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning.

BIG-bench Machine Learning Graph Attention +3

Towards Differentiable Resampling

no code implementations24 Apr 2020 Michael Zhu, Kevin Murphy, Rico Jonschkowski

Resampling is a key component of sample-based recursive state estimation in particle filters.

Regularized Autoencoders via Relaxed Injective Probability Flow

no code implementations20 Feb 2020 Abhishek Kumar, Ben Poole, Kevin Murphy

Invertible flow-based generative models are an effective method for learning to generate samples, while allowing for tractable likelihood computation and inference.

The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction

1 code implementation CVPR 2020 Junwei Liang, Lu Jiang, Kevin Murphy, Ting Yu, Alexander Hauptmann

The first contribution is a new dataset, created in a realistic 3D simulator, which is based on real world trajectory data, and then extrapolated by human annotators to achieve different latent goals.

Autonomous Driving Human motion prediction +5

Language as an Abstraction for Hierarchical Deep Reinforcement Learning

2 code implementations NeurIPS 2019 Yiding Jiang, Shixiang Gu, Kevin Murphy, Chelsea Finn

We find that, using our approach, agents can learn to solve to diverse, temporally-extended tasks such as object sorting and multi-object rearrangement, including from raw pixel observations.

Instruction Following Object +2

Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction

1 code implementation16 Jun 2019 Steven Hickson, Karthik Raveendran, Alireza Fathi, Kevin Murphy, Irfan Essa

We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image.

 Ranked #1 on Semantic Segmentation on ScanNetV2 (Pixel Accuracy metric)

Semantic Segmentation Surface Normals Estimation +1

Learning Video Representations using Contrastive Bidirectional Transformer

no code implementations13 Jun 2019 Chen Sun, Fabien Baradel, Kevin Murphy, Cordelia Schmid

This paper proposes a self-supervised learning approach for video features that results in significantly improved performance on downstream tasks (such as video classification, captioning and segmentation) compared to existing methods.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +5

A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization

no code implementations24 May 2019 David H. Brookes, Akosua Busia, Clara Fannjiang, Kevin Murphy, Jennifer Listgarten

We show that a large class of Estimation of Distribution Algorithms, including, but not limited to, Covariance Matrix Adaption, can be written as a Monte Carlo Expectation-Maximization algorithm, and as exact EM in the limit of infinite samples.

Stochastic Optimization

Predicting the Present and Future States of Multi-agent Systems from Partially-observed Visual Data

no code implementations ICLR 2019 Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, Kevin Murphy

We present a method which learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents.

Modeling Parts, Structure, and System Dynamics via Predictive Learning

no code implementations ICLR 2019 Zhenjia Xu, Zhijian Liu, Chen Sun, Kevin Murphy, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future.

Object

Unsupervised Discovery of Parts, Structure, and Dynamics

no code implementations12 Mar 2019 Zhenjia Xu, Zhijian Liu, Chen Sun, Kevin Murphy, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future.

Object

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

no code implementations25 Feb 2019 Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, Kevin Murphy

We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents.

NAS-Bench-101: Towards Reproducible Neural Architecture Search

4 code implementations25 Feb 2019 Chris Ying, Aaron Klein, Esteban Real, Eric Christiansen, Kevin Murphy, Frank Hutter

Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation.

Benchmarking Neural Architecture Search

Composing Text and Image for Image Retrieval - An Empirical Odyssey

4 code implementations CVPR 2019 Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, James Hays

In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image.

Image Retrieval Image Retrieval with Multi-Modal Query +1

Modeling Uncertainty with Hedged Instance Embedding

1 code implementation30 Sep 2018 Seong Joon Oh, Kevin Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, Andrew Gallagher

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering.

Clustering Metric Learning +1

Actor-Centric Relation Network

1 code implementation ECCV 2018 Chen Sun, Abhinav Shrivastava, Carl Vondrick, Kevin Murphy, Rahul Sukthankar, Cordelia Schmid

A visualization of the learned relation features confirms that our approach is able to attend to the relevant relations for each action.

Action Classification Action Detection +5

An information-theoretic analysis of deep latent-variable models

no code implementations ICLR 2018 Alex Alemi, Ben Poole, Ian Fischer, Josh Dillon, Rif A. Saurus, Kevin Murphy

We present an information-theoretic framework for understanding trade-offs in unsupervised learning of deep latent-variables models using variational inference.

Variational Inference

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

1 code implementation ECCV 2018 Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, Kevin Murphy

Despite the steady progress in video analysis led by the adoption of convolutional neural networks (CNNs), the relative improvement has been less drastic as that in 2D static image classification.

Ranked #27 on Action Recognition on UCF101 (using extra training data)

Action Classification Action Detection +6

Progressive Neural Architecture Search

17 code implementations ECCV 2018 Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms.

Evolutionary Algorithms General Classification +3

XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings

4 code implementations ICLR 2018 Amélie Royer, Konstantinos Bousmalis, Stephan Gouws, Fred Bertsch, Inbar Mosseri, Forrester Cole, Kevin Murphy

Style transfer usually refers to the task of applying color and texture information from a specific style image to a given content image while preserving the structure of the latter.

Domain Adaptation Style Transfer +2

Fixing a Broken ELBO

1 code implementation ICML 2018 Alexander A. Alemi, Ben Poole, Ian Fischer, Joshua V. Dillon, Rif A. Saurous, Kevin Murphy

Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models.

Representation Learning

Generative Models of Visually Grounded Imagination

no code implementations ICLR 2018 Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, Kevin Murphy

It is easy for people to imagine what a man with pink hair looks like, even if they have never seen such a person before.

Attribute

PixColor: Pixel Recursive Colorization

no code implementations19 May 2017 Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy

Then, given the generated low-resolution color image and the original grayscale image as inputs, we train a second CNN to generate a high-resolution colorization of an image.

Colorization

Attention-based Extraction of Structured Information from Street View Imagery

3 code implementations11 Apr 2017 Zbigniew Wojna, Alex Gorban, Dar-Shyang Lee, Kevin Murphy, Qian Yu, Yeqing Li, Julian Ibarz

We present a neural network model - based on CNNs, RNNs and a novel attention mechanism - which achieves 84. 2% accuracy on the challenging French Street Name Signs (FSNS) dataset, significantly outperforming the previous state of the art (Smith'16), which achieved 72. 46%.

Optical Character Recognition (OCR)

Deep Probabilistic Programming

no code implementations13 Jan 2017 Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei

By treating inference as a first class citizen, on a par with modeling, we show that probabilistic programming can be as flexible and computationally efficient as traditional deep learning.

Probabilistic Programming Variational Inference

Context-aware Captions from Context-agnostic Supervision

1 code implementation CVPR 2017 Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, Gal Chechik

We introduce an inference technique to produce discriminative context-aware image captions (captions that describe differences between images or visual concepts) using only generic context-agnostic training data (captions that describe a concept or an image in isolation).

Image Captioning Language Modelling

Towards Accurate Multi-person Pose Estimation in the Wild

no code implementations CVPR 2017 George Papandreou, Tyler Zhu, Nori Kanazawa, Alexander Toshev, Jonathan Tompson, Chris Bregler, Kevin Murphy

Trained on COCO data alone, our final system achieves average precision of 0. 649 on the COCO test-dev set and the 0. 643 test-standard sets, outperforming the winner of the 2016 COCO keypoints challenge and other recent state-of-art.

Human Detection Keypoint Detection +1

Deep Metric Learning via Facility Location

1 code implementation CVPR 2017 Hyun Oh Song, Stefanie Jegelka, Vivek Rathod, Kevin Murphy

Learning the representation and the similarity metric in an end-to-end fashion with deep networks have demonstrated outstanding results for clustering and retrieval.

Clustering Metric Learning +2

Deep Variational Information Bottleneck

9 code implementations1 Dec 2016 Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, Kevin Murphy

We present a variational approximation to the information bottleneck of Tishby et al. (1999).

Adversarial Attack

Improved Image Captioning via Policy Gradient optimization of SPIDEr

2 code implementations ICCV 2017 Si-Qi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, Kevin Murphy

Finally, we show that using our PG method we can optimize any of the metrics, including the proposed SPIDEr metric which results in image captions that are strongly preferred by human raters compared to captions generated by the same model but trained to optimize MLE or the COCO metrics.

Image Captioning

Speed/accuracy trade-offs for modern convolutional object detectors

14 code implementations CVPR 2017 Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang song, Sergio Guadarrama, Kevin Murphy

On the opposite end in which accuracy is critical, we present a detector that achieves state-of-the-art performance measured on the COCO detection task.

Ranked #220 on Object Detection on COCO test-dev (using extra training data)

Object object-detection +1

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

47 code implementations2 Jun 2016 Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L. Yuille

ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales.

Image Segmentation Semantic Segmentation

Efficient inference in occlusion-aware generative models of images

no code implementations19 Nov 2015 Jonathan Huang, Kevin Murphy

We present a generative model of images based on layering, in which image layers are individually generated, then composited from front to back.

Object

Detecting events and key actors in multi-person videos

no code implementations CVPR 2016 Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, Li Fei-Fei

In this paper, we propose a model which learns to detect events in such videos while automatically "attending" to the people responsible for the event.

Event Detection General Classification

Generation and Comprehension of Unambiguous Object Descriptions

1 code implementation CVPR 2016 Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, Kevin Murphy

We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described.

Image Captioning Object +1

Nonlinear functional mapping of the human brain

no code implementations8 Sep 2015 Nicholas Allgaier, Tobias Banaschewski, Gareth Barker, Arun L. W. Bokde, Josh C. Bongard, Uli Bromberg, Christian Büchel, Anna Cattrell, Patricia J. Conrod, Christopher M. Danforth, Sylvane Desrivières, Peter S. Dodds, Herta Flor, Vincent Frouin, Jürgen Gallinat, Penny Gowland, Andreas Heinz, Bernd Ittermann, Scott Mackey, Jean-Luc Martinot, Kevin Murphy, Frauke Nees, Dimitri Papadopoulos-Orfanos, Luise Poustka, Michael N. Smolka, Henrik Walter, Robert Whelan, Gunter Schumann, Hugh Garavan, IMAGEN Consortium

In the present study, we introduce just such a method, called nonlinear functional mapping (NFM), and demonstrate its application in the analysis of resting state fMRI from a 242-subject subset of the IMAGEN project, a European study of adolescents that includes longitudinal phenotypic, behavioral, genetic, and neuroimaging data.

Bayesian Dark Knowledge

1 code implementation NeurIPS 2015 Anoop Korattikara, Vivek Rathod, Kevin Murphy, Max Welling

We consider the problem of Bayesian parameter estimation for deep neural networks, which is important in problem settings where we may have little data, and/ or where we need accurate posterior predictive densities, e. g., for applications involving bandits or active learning.

Active Learning

What's Cookin'? Interpreting Cooking Videos using Text, Speech and Vision

1 code implementation5 Mar 2015 Jonathan Malmaud, Jonathan Huang, Vivek Rathod, Nick Johnston, Andrew Rabinovich, Kevin Murphy

We present a novel method for aligning a sequence of instructions to a video of someone carrying out a task.

Keyword Spotting

Probabilistic Label Relation Graphs with Ising Models

no code implementations ICCV 2015 Nan Ding, Jia Deng, Kevin Murphy, Hartmut Neven

In this paper, we extend the HEX model to allow for soft or probabilistic relations between labels, which is useful when there is uncertainty about the relationship between two labels (e. g., an antelope is "sort of" furry, but not to the same degree as a grizzly bear).

General Classification Relation

A Review of Relational Machine Learning for Knowledge Graphs

2 code implementations2 Mar 2015 Maximilian Nickel, Kevin Murphy, Volker Tresp, Evgeniy Gabrilovich

In this paper, we provide a review of how such statistical models can be "trained" on large knowledge graphs, and then used to predict new facts about the world (which is equivalent to predicting new edges in the graph).

BIG-bench Machine Learning Knowledge Graphs

Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

3 code implementations9 Feb 2015 George Papandreou, Liang-Chieh Chen, Kevin Murphy, Alan L. Yuille

Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation.

Image Segmentation Segmentation +2

Loopy Belief Propagation for Approximate Inference: An Empirical Study

1 code implementation23 Jan 2013 Kevin Murphy, Yair Weiss, Michael. I. Jordan

Recently, researchers have demonstrated that loopy belief propagation - the use of Pearls polytree algorithm IN a Bayesian network WITH loops OF error- correcting codes. The most dramatic instance OF this IS the near Shannon - limit performance OF Turbo Codes codes whose decoding algorithm IS equivalent TO loopy belief propagation IN a chain - structured Bayesian network.

Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (2012)

no code implementations19 Jan 2013 Nando de Freitas, Kevin Murphy

This is the Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, which was held on Catalina Island, CA August 14-18 2012.

Cannot find the paper you are looking for? You can Submit a new open access paper.