Search Results for author: Philip H. S. Torr

Found 190 papers, 108 papers with code

GDumb: A Simple Approach that Questions Our Progress in Continual Learning

2 code implementations ECCV 2020 Ameya Prabhu, Philip H. S. Torr, Puneet K. Dokania

We discuss a general formulation for the Continual Learning (CL) problem for classification---a learning task where a stream provides samples to a learner and the goal of the learner, depending on the samples it receives, is to continually upgrade its knowledge about the old classes and learn new ones.

Continual Learning Open Set Learning

Language Model Tokenizers Introduce Unfairness Between Languages

1 code implementation17 May 2023 Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, Adel Bibi

Recent language models have shown impressive multilingual performance, even when not explicitly trained for it.

Language Modelling

Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?

1 code implementation16 May 2023 Hasan Abed Al Kader Hammoud, Ameya Prabhu, Ser-Nam Lim, Philip H. S. Torr, Adel Bibi, Bernard Ghanem

We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy, which measures the accuracy of the model on the immediate next few samples.

Continual Learning

Certifying Ensembles: A General Certification Theory with S-Lipschitzness

no code implementations25 Apr 2023 Aleksandar Petrov, Francisco Eiras, Amartya Sanyal, Philip H. S. Torr, Adel Bibi

Improving and guaranteeing the robustness of deep learning models has been a topic of intense research.

Fairness in AI and Its Long-Term Implications on Society

no code implementations16 Apr 2023 Ondrej Bohdal, Timothy Hospedales, Philip H. S. Torr, Fazl Barez

Successful deployment of artificial intelligence (AI) in various settings has led to numerous positive outcomes for individuals and society.


Don't FREAK Out: A Frequency-Inspired Approach to Detecting Backdoor Poisoned Samples in DNNs

no code implementations23 Mar 2023 Hasan Abed Al Kader Hammoud, Adel Bibi, Philip H. S. Torr, Bernard Ghanem

In this paper we investigate the frequency sensitivity of Deep Neural Networks (DNNs) when presented with clean samples versus poisoned samples.

Computationally Budgeted Continual Learning: What Does Matter?

1 code implementation CVPR 2023 Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet Dokania, Philip H. S. Torr, Ser-Nam Lim, Bernard Ghanem, Adel Bibi

Our conclusions are consistent in a different number of stream time steps, e. g., 20 to 200, and under several computational budgets.

Continual Learning

MobileBrick: Building LEGO for 3D Reconstruction on Mobile Devices

1 code implementation CVPR 2023 Kejie Li, Jia-Wang Bian, Robert Castle, Philip H. S. Torr, Victor Adrian Prisacariu

The distinct data modality offered by high-resolution RGB images and low-resolution depth maps captured on a mobile device, when combined with precise 3D geometry annotations, presents a unique opportunity for future research on high-fidelity 3D reconstruction.

3D Object Reconstruction 3D Reconstruction

MOSE: A New Dataset for Video Object Segmentation in Complex Scenes

1 code implementation3 Feb 2023 Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip H. S. Torr, Song Bai

However, since the target objects in these existing datasets are usually relatively salient, dominant, and isolated, VOS under complex scenes has rarely been studied.

Semantic Segmentation Video Object Segmentation +1

Real-Time Evaluation in Online Continual Learning: A New Hope

1 code implementation CVPR 2023 Yasir Ghunaim, Adel Bibi, Kumail Alhamoud, Motasem Alfarra, Hasan Abed Al Kader Hammoud, Ameya Prabhu, Philip H. S. Torr, Bernard Ghanem

We show that a simple baseline outperforms state-of-the-art CL methods under this evaluation, questioning the applicability of existing methods in realistic settings.

Continual Learning

Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning

no code implementations CVPR 2023 Jishnu Mukhoti, Tsung-Yu Lin, Omid Poursaeed, Rui Wang, Ashish Shah, Philip H. S. Torr, Ser-Nam Lim

We introduce Patch Aligned Contrastive Learning (PACL), a modified compatibility function for CLIP's contrastive loss, intending to train an alignment between the patch tokens of the vision encoder and the CLS token of the text encoder.

Contrastive Learning Image Classification +4

Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs

no code implementations27 Nov 2022 Guangrun Wang, Philip H. S. Torr

Proving that classifiers have learned the data distribution and are ready for image generation has far-reaching implications, for classifiers are much easier to train than generative models like DDPMs and GANs.

Text-to-Image Generation

Structure-Preserving 3D Garment Modeling with Neural Sewing Machines

no code implementations12 Nov 2022 Xipeng Chen, Guangrun Wang, Dizhong Zhu, Xiaodan Liang, Philip H. S. Torr, Liang Lin

In this paper, we propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling, which is capable of learning representations for garments with diverse shapes and topologies and is successfully applied to 3D garment reconstruction and controllable manipulation.

Representation Learning

Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesis

1 code implementation12 Nov 2022 Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe

To further capture the change in pose of each part more precisely, we propose a novel part-aware bipartite graph reasoning (PBGR) block to decompose the task of reasoning the global structure transformation with a bipartite graph into learning different local transformations for different semantic body/face parts.

Image Generation

Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning

1 code implementation24 Oct 2022 Nan Xue, Tianfu Wu, Song Bai, Fu-Dong Wang, Gui-Song Xia, Liangpei Zhang, Philip H. S. Torr

At the core is a parsimonious representation that encodes a line segment using a closed-form 4D geometric vector, which enables lifting line segments in wireframe to an end-to-end trainable holistic attraction field that has built-in geometry-awareness, context-awareness and robustness.

Self-Supervised Learning Wireframe Parsing

Raising the Bar on the Evaluation of Out-of-Distribution Detection

no code implementations24 Sep 2022 Jishnu Mukhoti, Tsung-Yu Lin, Bor-Chun Chen, Ashish Shah, Philip H. S. Torr, Puneet K. Dokania, Ser-Nam Lim

In this paper, we define 2 categories of OoD data using the subtly different concepts of perceptual/visual and semantic similarity to in-distribution (iD) data.

Out-of-Distribution Detection Out of Distribution (OOD) Detection +2

Learn what matters: cross-domain imitation learning with task-relevant embeddings

no code implementations24 Sep 2022 Tim Franzmeyer, Philip H. S. Torr, João F. Henriques

We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent.

Imitation Learning

Dynamic Graph Message Passing Networks for Visual Recognition

2 code implementations20 Sep 2022 Li Zhang, Mohan Chen, Anurag Arnab, xiangyang xue, Philip H. S. Torr

A fully-connected graph, such as the self-attention operation in Transformers, is beneficial for such modelling, however, its computational overhead is prohibitive.

Image Classification object-detection +3

Memory-Driven Text-to-Image Generation

no code implementations15 Aug 2022 Bowen Li, Philip H. S. Torr, Thomas Lukasiewicz

We introduce a memory-driven semi-parametric approach to text-to-image generation, which is based on both parametric and non-parametric techniques.

Text-to-Image Generation

An Impartial Take to the CNN vs Transformer Robustness Contest

no code implementations22 Jul 2022 Francesco Pinto, Philip H. S. Torr, Puneet K. Dokania

Following the surge of popularity of Transformers in Computer Vision, several studies have attempted to determine whether they could be more robust to distribution shifts and provide better uncertainty estimates than Convolutional Neural Networks (CNNs).

Sample-dependent Adaptive Temperature Scaling for Improved Calibration

1 code implementation13 Jul 2022 Tom Joy, Francesco Pinto, Ser-Nam Lim, Philip H. S. Torr, Puneet K. Dokania

The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the confidences of the predictions on any input by scaling the logits by a fixed value.

Out of Distribution (OOD) Detection

SiamMask: A Framework for Fast Online Object Tracking and Segmentation

no code implementations5 Jul 2022 Weiming Hu, Qiang Wang, Li Zhang, Luca Bertinetto, Philip H. S. Torr

In this paper we introduce SiamMask, a framework to perform both visual object tracking and video object segmentation, in real-time, with the same simple method.

Multiple Object Tracking Semantic Segmentation +3

RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness

1 code implementation29 Jun 2022 Francesco Pinto, Harry Yang, Ser-Nam Lim, Philip H. S. Torr, Puneet K. Dokania

We show that the effectiveness of the well celebrated Mixup [Zhang et al., 2018] can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss.

Out-of-Distribution Detection

How Robust is Unsupervised Representation Learning to Distribution Shift?

no code implementations17 Jun 2022 Yuge Shi, Imant Daunhawer, Julia E. Vogt, Philip H. S. Torr, Amartya Sanyal

As such, there is a lack of insight on the robustness of the representations learned from unsupervised methods, such as self-supervised learning (SSL) and auto-encoder based algorithms (AE), to distribution shift.

Representation Learning Self-Supervised Learning

Catastrophic overfitting is a bug but also a feature

1 code implementation16 Jun 2022 Guillermo Ortiz-Jiménez, Pau de Jorge, Amartya Sanyal, Adel Bibi, Puneet K. Dokania, Pascal Frossard, Gregory Rogéz, Philip H. S. Torr

Despite clear computational advantages in building robust neural networks, adversarial training (AT) using single-step methods is unstable as it suffers from catastrophic overfitting (CO): Networks gain non-trivial robustness during the first stages of adversarial training, but suddenly reach a breaking point where they quickly lose all robustness in just a few iterations.

Zero-Shot Logit Adjustment

1 code implementation25 Apr 2022 Dubing Chen, Yuming Shen, Haofeng Zhang, Philip H. S. Torr

As a consequence of our derivation, the aforementioned two properties are incorporated into the classifier training as seen-unseen priors via logit adjustment.

 Ranked #1 on Generalized Zero-Shot Learning on AwA2 (Accuracy Unseen metric)

Bayesian Inference Generalized Zero-Shot Learning +1

Deconstructed Generation-Based Zero-Shot Model

1 code implementation24 Apr 2022 Dubing Chen, Yuming Shen, Haofeng Zhang, Philip H. S. Torr

Recent research on Generalized Zero-Shot Learning (GZSL) has focused primarily on generation-based methods.

Generalized Zero-Shot Learning

Local and Global GANs with Semantic-Aware Upsampling for Image Generation

1 code implementation28 Feb 2022 Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe

To learn more discriminative class-specific feature representations for the local generation, we also propose a novel classification module.

Image Generation

Make Some Noise: Reliable and Efficient Single-Step Adversarial Training

1 code implementation2 Feb 2022 Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip H. S. Torr, Grégory Rogez, Puneet K. Dokania

Recently, Wong et al. showed that adversarial training with single-step FGSM leads to a characteristic failure mode named Catastrophic Overfitting (CO), in which a model becomes suddenly vulnerable to multi-step attacks.

Learning to Hash Naturally Sorts

no code implementations31 Jan 2022 Jiaguo Yu, Yuming Shen, Menghan Wang, Haofeng Zhang, Philip H. S. Torr

In this paper, we tackle this problem by introducing Naturally-Sorted Hashing (NSH).

Contrastive Learning Retrieval

On the Robustness of Quality Measures for GANs

1 code implementation31 Jan 2022 Motasem Alfarra, Juan C. Pérez, Anna Frühstück, Philip H. S. Torr, Peter Wonka, Bernard Ghanem

Finally, we show that the FID can be robustified by simply replacing the standard Inception with a robust Inception.

Adversarial Masking for Self-Supervised Learning

1 code implementation31 Jan 2022 Yuge Shi, N. Siddharth, Philip H. S. Torr, Adam R. Kosiorek

We propose ADIOS, a masked image model (MIM) framework for self-supervised learning, which simultaneously learns a masking function and an image encoder using an adversarial objective.

Representation Learning Self-Supervised Learning +1

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

1 code implementation CVPR 2022 Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, Philip H. S. Torr

Referring image segmentation is a fundamental vision-language task that aims to segment out an object referred to by a natural language expression from an image.

Image Segmentation Referring Expression +2

Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS

no code implementations23 Nov 2021 Christian Schroeder de Witt, Yongchao Huang, Philip H. S. Torr, Martin Strohmeier

We then argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions, and introduce a temporally extended multi-agent reinforcement learning framework in which the resultant dynamics can be studied.

Continual Learning Multi-agent Reinforcement Learning +2

Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge

no code implementations15 Nov 2021 Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge Belongie, Alan Yuille, Philip H. S. Torr, Song Bai

To promote the development of occlusion understanding, we collect a large-scale dataset called OVIS for video instance segmentation in the occluded scenario.

Instance Segmentation Object Recognition +3

Deep Deterministic Uncertainty for Semantic Segmentation

no code implementations29 Oct 2021 Jishnu Mukhoti, Joost van Amersfoort, Philip H. S. Torr, Yarin Gal

We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation.

Semantic Segmentation

Detecting and Quantifying Malicious Activity with Simulation-based Inference

no code implementations6 Oct 2021 Andrew Gambardella, Bogdan State, Naeemullah Khan, Leo Tsourides, Philip H. S. Torr, Atılım Güneş Baydin

We propose the use of probabilistic programming techniques to tackle the malicious user identification problem in a recommendation algorithm.

Probabilistic Programming

RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

1 code implementation11 Sep 2021 Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H. S. Torr, DaCheng Tao

Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).

Adversarial Robustness Benchmarking +2

ANCER: Anisotropic Certification via Sample-wise Volume Maximization

1 code implementation9 Jul 2021 Francisco Eiras, Motasem Alfarra, M. Pawan Kumar, Philip H. S. Torr, Puneet K. Dokania, Bernard Ghanem, Adel Bibi

Randomized smoothing has recently emerged as an effective tool that enables certification of deep neural network classifiers at scale.

Do Different Tracking Tasks Require Different Appearance Models?

1 code implementation NeurIPS 2021 Zhongdao Wang, Hengshuang Zhao, Ya-Li Li, Shengjin Wang, Philip H. S. Torr, Luca Bertinetto

We show how most tracking tasks can be solved within this framework, and that the same appearance model can be successfully used to obtain results that are competitive against specialised methods for most of the tasks considered.

Multi-Object Tracking Multi-Object Tracking and Segmentation +10

DeformRS: Certifying Input Deformations with Randomized Smoothing

2 code implementations2 Jul 2021 Motasem Alfarra, Adel Bibi, Naeemullah Khan, Philip H. S. Torr, Bernard Ghanem

Deep neural networks are vulnerable to input deformations in the form of vector fields of pixel displacements and to other parameterized geometric deformations e. g. translations, rotations, etc.

Learning Multimodal VAEs through Mutual Supervision

1 code implementation ICLR 2022 Tom Joy, Yuge Shi, Philip H. S. Torr, Tom Rainforth, Sebastian M. Schmon, N. Siddharth

Here we introduce a novel alternative, the MEME, that avoids such explicit combinations by repurposing semi-supervised VAEs to combine information between modalities implicitly through mutual supervision.

KL Guided Domain Adaptation

1 code implementation ICLR 2022 A. Tuan Nguyen, Toan Tran, Yarin Gal, Philip H. S. Torr, Atılım Güneş Baydin

A common approach in the domain adaptation literature is to learn a representation of the input that has the same (marginal) distribution over the source and the target domain.

Domain Adaptation

You Never Cluster Alone

no code implementations NeurIPS 2021 Yuming Shen, Ziyi Shen, Menghan Wang, Jie Qin, Philip H. S. Torr, Ling Shao

On one hand, with the corresponding assignment variables being the weight, a weighted aggregation along the data points implements the set representation of a cluster.

Contrastive Learning Self-Supervised Learning

Gradient Matching for Domain Generalization

2 code implementations ICLR 2022 Yuge Shi, Jeffrey Seely, Philip H. S. Torr, N. Siddharth, Awni Hannun, Nicolas Usunier, Gabriel Synnaeve

We perform experiments on both the Wilds benchmark, which captures distribution shift in the real world, as well as datasets in DomainBed benchmark that focuses more on synthetic-to-real transfer.

Domain Generalization

Solving Inefficiency of Self-supervised Representation Learning

1 code implementation ICCV 2021 Guangrun Wang, Keze Wang, Guangcong Wang, Philip H. S. Torr, Liang Lin

In this paper, we reveal two contradictory phenomena in contrastive learning that we call under-clustering and over-clustering problems, which are major obstacles to learning efficiency.

Contrastive Learning Representation Learning +3

Cloth Interactive Transformer for Virtual Try-On

1 code implementation12 Apr 2021 Bin Ren, Hao Tang, Fanyang Meng, Runwei Ding, Ling Shao, Philip H. S. Torr, Nicu Sebe

2D image-based virtual try-on has attracted increased attention from the multimedia and computer vision communities.

Virtual Try-on

Deep Deterministic Uncertainty: A Simple Baseline

4 code implementations23 Feb 2021 Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip H. S. Torr, Yarin Gal

Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.

Active Learning

Shape-Tailored Deep Neural Networks

no code implementations16 Feb 2021 Naeemullah Khan, Angira Sharma, Ganesh Sundaramoorthi, Philip H. S. Torr

We stack multiple PDE layers to generalize a deep CNN to arbitrary regions, and apply it to segmentation.

Occluded Video Instance Segmentation: A Benchmark

1 code implementation2 Feb 2021 Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge Belongie, Alan Yuille, Philip H. S. Torr, Song Bai

On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16. 3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario.

Instance Segmentation Semantic Segmentation +2

GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation

2 code implementations13 Dec 2020 Xiaojuan Qi, Zhengzhe Liu, Renjie Liao, Philip H. S. Torr, Raquel Urtasun, Jiaya Jia

Note that GeoNet++ is generic and can be used in other depth/normal prediction frameworks to improve the quality of 3D reconstruction and pixel-wise accuracy of depth and surface normals.

3D Reconstruction Depth Estimation +2

Data-Dependent Randomized Smoothing

no code implementations8 Dec 2020 Motasem Alfarra, Adel Bibi, Philip H. S. Torr, Bernard Ghanem

In this work, we revisit Gaussian randomized smoothing and show that the variance of the Gaussian distribution can be optimized at each input so as to maximize the certification radius for the construction of the smooth classifier.

Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?

4 code implementations18 Nov 2020 Christian Schroeder de Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip H. S. Torr, Mingfei Sun, Shimon Whiteson

Most recently developed approaches to cooperative multi-agent reinforcement learning in the \emph{centralized training with decentralized execution} setting involve estimating a centralized, joint value function.

reinforcement-learning Reinforcement Learning (RL) +2

Lightweight Generative Adversarial Networks for Text-Guided Image Manipulation

1 code implementation NeurIPS 2020 Bowen Li, Xiaojuan Qi, Philip H. S. Torr, Thomas Lukasiewicz

To achieve this, a new word-level discriminator is proposed, which provides the generator with fine-grained training feedback at word-level, to facilitate training a lightweight generator that has a small number of parameters, but can still correctly focus on specific visual attributes of an image, and then edit them without affecting other contents that are not described in the text.

Image Manipulation Text-to-Image Generation

Continual Learning in Low-rank Orthogonal Subspaces

1 code implementation NeurIPS 2020 Arslan Chaudhry, Naeemullah Khan, Puneet K. Dokania, Philip H. S. Torr

In continual learning (CL), a learner is faced with a sequence of tasks, arriving one after the other, and the goal is to remember all the tasks once the continual learning experience is finished.

Continual Learning

Bipartite Graph Reasoning GANs for Person Image Generation

1 code implementation10 Aug 2020 Hao Tang, Song Bai, Philip H. S. Torr, Nicu Sebe

We present a novel Bipartite Graph Reasoning GAN (BiGraphGAN) for the challenging person image generation task.

 Ranked #1 on Pose Transfer on Market-1501 (PCKh metric)

Pose Transfer

XingGAN for Person Image Generation

2 code implementations ECCV 2020 Hao Tang, Song Bai, Li Zhang, Philip H. S. Torr, Nicu Sebe

We propose a novel Generative Adversarial Network (XingGAN or CrossingGAN) for person image generation tasks, i. e., translating the pose of a given person to a desired one.

 Ranked #1 on Pose Transfer on Market-1501 (IS metric)

Pose Transfer

WordCraft: An Environment for Benchmarking Commonsense Agents

1 code implementation ICML Workshop LaReL 2020 Minqi Jiang, Jelena Luketina, Nantas Nardelli, Pasquale Minervini, Philip H. S. Torr, Shimon Whiteson, Tim Rocktäschel

This is partly due to the lack of lightweight simulation environments that sufficiently reflect the semantics of the real world and provide knowledge sources grounded with respect to observations in an RL environment.

Benchmarking Knowledge Graphs +2

How benign is benign overfitting?

no code implementations8 Jul 2020 Amartya Sanyal, Puneet K. Dokania, Varun Kanade, Philip H. S. Torr

We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models.

Adversarial Robustness Representation Learning

Relating by Contrasting: A Data-efficient Framework for Multimodal Generative Models

no code implementations ICLR 2021 Yuge Shi, Brooks Paige, Philip H. S. Torr, N. Siddharth

Multimodal learning for generative models often refers to the learning of abstract concepts from the commonality of information in multiple modalities, such as vision and language.

Capturing Label Characteristics in VAEs

2 code implementations ICLR 2021 Tom Joy, Sebastian M. Schmon, Philip H. S. Torr, N. Siddharth, Tom Rainforth

We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels.

Progressive Skeletonization: Trimming more fat from a network at initialization

1 code implementation ICLR 2021 Pau de Jorge, Amartya Sanyal, Harkirat S. Behl, Philip H. S. Torr, Gregory Rogez, Puneet K. Dokania

Recent studies have shown that skeletonization (pruning parameters) of networks \textit{at initialization} provides all the practical benefits of sparsity both at inference and training time, while only marginally degrading their performance.

A Revised Generative Evaluation of Visual Dialogue

1 code implementation20 Apr 2020 Daniela Massiceti, Viveka Kulharia, Puneet K. Dokania, N. Siddharth, Philip H. S. Torr

Evaluating Visual Dialogue, the task of answering a sequence of questions relating to a visual input, remains an open research challenge.

Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training

no code implementations ICLR 2021 Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr, Martin Jaggi

As a result, we find across various workloads of data set, network model, and optimization algorithm that there exists a general scaling trend between batch size and number of training steps to convergence for the effect of data parallelism, and further, difficulty of training under sparsity.

Network Pruning

Cross-modal Deep Face Normals with Deactivable Skip Connections

1 code implementation CVPR 2020 Victoria Fernandez Abrevaya, Adnane Boukhayma, Philip H. S. Torr, Edmond Boyer

Core to our approach is a novel module that we call deactivable skip connections, which allows integrating both the auto-encoded and image-to-normal branches within the same architecture that can be trained end-to-end.

3D Face Reconstruction

FACMAC: Factored Multi-Agent Centralised Policy Gradients

3 code implementations NeurIPS 2021 Bei Peng, Tabish Rashid, Christian A. Schroeder de Witt, Pierre-Alexandre Kamienny, Philip H. S. Torr, Wendelin Böhmer, Shimon Whiteson

We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces.

Q-Learning SMAC +2

Holistically-Attracted Wireframe Parsing

1 code implementation CVPR 2020 Nan Xue, Tianfu Wu, Song Bai, Fu-Dong Wang, Gui-Song Xia, Liangpei Zhang, Philip H. S. Torr

For computing line segment proposals, a novel exact dual representation is proposed which exploits a parsimonious geometric reparameterization for line segments and forms a holistic 4-dimensional attraction field map for an input image.

Line Segment Detection Wireframe Parsing

Lagrangian Decomposition for Neural Network Verification

2 code implementations24 Feb 2020 Rudy Bunel, Alessandro De Palma, Alban Desmaison, Krishnamurthy Dvijotham, Pushmeet Kohli, Philip H. S. Torr, M. Pawan Kumar

Both the algorithms offer three advantages: (i) they yield bounds that are provably at least as tight as previous dual algorithms relying on Lagrangian relaxations; (ii) they are based on operations analogous to forward/backward pass of neural networks layers and are therefore easily parallelizable, amenable to GPU implementation and able to take advantage of the convolutional structure of problems; and (iii) they allow for anytime stopping while still providing valid bounds.

Calibrating Deep Neural Networks using Focal Loss

2 code implementations NeurIPS 2020 Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, Puneet K. Dokania

To facilitate the use of focal loss in practice, we also provide a principled approach to automatically select the hyperparameter involved in the loss function.

Image-to-Image Translation with Text Guidance

no code implementations12 Feb 2020 Bowen Li, Xiaojuan Qi, Philip H. S. Torr, Thomas Lukasiewicz

The goal of this paper is to embed controllable factors, i. e., natural language descriptions, into image-to-image translation with generative adversarial networks, which allows text descriptions to determine the visual attributes of synthetic images.

Image-to-Image Translation Part-Of-Speech Tagging +1

Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation

1 code implementation3 Feb 2020 Hao Tang, Philip H. S. Torr, Nicu Sebe

In the first stage, the input image and the conditional semantic guidance are fed into a cycled semantic-guided generation network to produce initial coarse results.

Image-to-Image Translation Translation

Unifying Training and Inference for Panoptic Segmentation

no code implementations CVPR 2020 Qizhu Li, Xiaojuan Qi, Philip H. S. Torr

This panoptic submodule gives rise to a novel propagation mechanism for panoptic logits and enables the network to output a coherent panoptic segmentation map for both "stuff" and "thing" classes, without any post-processing.

Panoptic Segmentation

Few-shot Action Recognition with Permutation-invariant Attention

1 code implementation ECCV 2020 Hongguang Zhang, Li Zhang, Xiaojuan Qi, Hongdong Li, Philip H. S. Torr, Piotr Koniusz

Such encoded blocks are aggregated by permutation-invariant pooling to make our approach robust to varying action lengths and long-range temporal dependencies whose patterns are unlikely to repeat even in clips of the same class.

Few-Shot action recognition Few Shot Action Recognition +2

Improving Few-shot Learning by Spatially-aware Matching and CrossTransformer

no code implementations6 Jan 2020 Hongguang Zhang, Philip H. S. Torr, Piotr Koniusz

In this paper, we study the impact of scale and location mismatch in the few-shot learning scenario, and propose a novel Spatially-aware Matching (SM) scheme to effectively perform matching across multiple scales and locations, and learn image relations by giving the highest weights to the best matching pairs.

Deblurring Few-Shot Learning +2

Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

2 code implementations CVPR 2020 Hao Tang, Dan Xu, Yan Yan, Philip H. S. Torr, Nicu Sebe

To tackle this issue, in this work we consider learning the scene generation in a local context, and correspondingly design a local class-specific generative network with semantic maps as a guidance, which separately constructs and learns sub-generators concentrating on the generation of different classes, and is able to provide more scene details.

Image Generation Scene Generation

Learning Regional Attraction for Line Segment Detection

no code implementations18 Dec 2019 Nan Xue, Song Bai, Fu-Dong Wang, Gui-Song Xia, Tianfu Wu, Liangpei Zhang, Philip H. S. Torr

Given a line segment map, the proposed regional attraction first establishes the relationship between line segments and regions in the image lattice.

Line Segment Detection

Lessons from reinforcement learning for biological representations of space

no code implementations13 Dec 2019 Alex Muryy, Siddharth Narayanaswamy, Nantas Nardelli, Andrew Glennerster, Philip H. S. Torr

Neuroscientists postulate 3D representations in the brain in a variety of different coordinate frames (e. g. 'head-centred', 'hand-centred' and 'world-based').

reinforcement-learning Reinforcement Learning (RL)

ManiGAN: Text-Guided Image Manipulation

3 code implementations12 Dec 2019 Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, Philip H. S. Torr

The goal of our paper is to semantically edit parts of an image matching a given text that describes desired attributes (e. g., texture, colour, and background), while preserving other contents that are irrelevant to the text.

Image Manipulation Text-to-Image Generation

Transflow Learning: Repurposing Flow Models Without Retraining

no code implementations29 Nov 2019 Andrew Gambardella, Atılım Güneş Baydin, Philip H. S. Torr

It is well known that deep generative models have a rich latent space, and that it is possible to smoothly manipulate their outputs by traversing this latent space.

Bayesian Inference Style Transfer

Siam R-CNN: Visual Tracking by Re-Detection

1 code implementation CVPR 2020 Paul Voigtlaender, Jonathon Luiten, Philip H. S. Torr, Bastian Leibe

We present Siam R-CNN, a Siamese re-detection architecture which unleashes the full power of two-stage object detection approaches for visual object tracking.

object-detection Object Detection +3

AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks

2 code implementations27 Nov 2019 Hao Tang, Hong Liu, Dan Xu, Philip H. S. Torr, Nicu Sebe

State-of-the-art methods in image-to-image translation are capable of learning a mapping from a source domain to a target domain with unpaired image data.

Image-to-Image Translation Translation

Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models

2 code implementations NeurIPS 2019 Yuge Shi, N. Siddharth, Brooks Paige, Philip H. S. Torr

In this work, we characterise successful learning of such models as the fulfillment of four criteria: i) implicit latent decomposition into shared and private subspaces, ii) coherent joint generation over all modalities, iii) coherent cross-generation across individual modalities, and iv) improved model learning for individual modalities through multi-modal integration.

Mirror Descent View for Neural Network Quantization

1 code implementation18 Oct 2019 Thalaiyasingam Ajanthan, Kartik Gupta, Philip H. S. Torr, Richard Hartley, Puneet K. Dokania

Quantizing large Neural Networks (NN) while maintaining the performance is highly desirable for resource-limited devices due to reduced memory and time complexity.


Controllable Text-to-Image Generation

2 code implementations NeurIPS 2019 Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, Philip H. S. Torr

In this paper, we propose a novel controllable text-to-image generative adversarial network (ControlGAN), which can effectively synthesise high-quality images and also control parts of the image generation according to natural language descriptions.

Text-to-Image Generation

Branch and Bound for Piecewise Linear Neural Network Verification

no code implementations14 Sep 2019 Rudy Bunel, Jingyue Lu, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, M. Pawan Kumar

We use the data sets to conduct a thorough experimental comparison of existing and new algorithms and to provide an inclusive analysis of the factors impacting the hardness of verification problems.

Dual Graph Convolutional Network for Semantic Segmentation

5 code implementations13 Sep 2019 Li Zhang, Xiangtai Li, Anurag Arnab, Kuiyuan Yang, Yunhai Tong, Philip H. S. Torr

Exploiting long-range contextual information is key for pixel-wise prediction tasks such as semantic segmentation.

Semantic Segmentation

Dynamic Graph Message Passing Networks

1 code implementation CVPR 2020 Li Zhang, Dan Xu, Anurag Arnab, Philip H. S. Torr

We propose a dynamic graph message passing network, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph.

Image Classification object-detection +3

Real-Time Highly Accurate Dense Depth on a Power Budget using an FPGA-CPU Hybrid SoC

no code implementations17 Jul 2019 Oscar Rahnama, Tommaso Cavallari, Stuart Golodetz, Alessio Tonioni, Thomas Joy, Luigi Di Stefano, Simon Walker, Philip H. S. Torr

Obtaining highly accurate depth from stereo images in real time has many applications across computer vision and robotics, but in some contexts, upper bounds on power consumption constrain the feasible hardware to embedded platforms such as FPGAs.

A Signal Propagation Perspective for Pruning Neural Networks at Initialization

1 code implementation ICLR 2020 Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, Philip H. S. Torr

Alternatively, a recent approach shows that pruning can be done at initialization prior to training, based on a saliency criterion called connection sensitivity.

Image Classification Network Pruning

Stable Rank Normalization for Improved Generalization in Neural Networks and GANs

no code implementations ICLR 2020 Amartya Sanyal, Philip H. S. Torr, Puneet K. Dokania

Exciting new work on the generalization bounds for neural networks (NN) given by Neyshabur et al. , Bartlett et al. closely depend on two parameter-depenedent quantities: the Lipschitz constant upper-bound and the stable rank (a softer version of the rank operator).

Generalization Bounds Image Generation +1

Hijacking Malaria Simulators with Probabilistic Programming

no code implementations29 May 2019 Bradley Gram-Hansen, Christian Schröder de Witt, Tom Rainforth, Philip H. S. Torr, Yee Whye Teh, Atılım Güneş Baydin

Epidemiology simulations have become a fundamental tool in the fight against the epidemics of various infectious diseases like AIDS and malaria.

Epidemiology Probabilistic Programming

Straight to Shapes++: Real-time Instance Segmentation Made More Accurate

1 code implementation27 May 2019 Laurynas Miksys, Saumya Jetley, Michael Sapienza, Stuart Golodetz, Philip H. S. Torr

The STS model can run at 35 FPS on a high-end desktop, but its accuracy is significantly worse than that of offline state-of-the-art methods.

Autonomous Driving Data Augmentation +4

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

no code implementations17 May 2019 Harkirat Singh Behl, Atılım Güneş Baydin, Philip H. S. Torr

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks.

Few-Shot Learning General Classification

GA-Net: Guided Aggregation Net for End-to-end Stereo Matching

3 code implementations CVPR 2019 Feihu Zhang, Victor Prisacariu, Ruigang Yang, Philip H. S. Torr

In the stereo matching task, matching cost aggregation is crucial in both traditional methods and deep neural network models in order to accurately estimate disparities.

Stereo Matching

Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks

no code implementations CVPR 2019 Eunwoo Kim, Chanho Ahn, Philip H. S. Torr, Songhwai Oh

To this end, we propose a novel network architecture producing multiple networks of different configurations, termed deep virtual networks (DVNs), for different tasks.

Learning to Adapt for Stereo

1 code implementation CVPR 2019 Alessio Tonioni, Oscar Rahnama, Thomas Joy, Luigi Di Stefano, Thalaiyasingam Ajanthan, Philip H. S. Torr

Real world applications of stereo depth estimation require models that are robust to dynamic variations in the environment.

Autonomous Driving Stereo Depth Estimation

Domain Partitioning Network

no code implementations21 Feb 2019 Botos Csaba, Adnane Boukhayma, Viveka Kulharia, András Horváth, Philip H. S. Torr

Standard adversarial training involves two agents, namely a generator and a discriminator, playing a mini-max game.

3D Hand Shape and Pose from Images in the Wild

2 code implementations CVPR 2019 Adnane Boukhayma, Rodrigo de Bem, Philip H. S. Torr

We present in this work the first end-to-end deep learning based method that predicts both 3D hand shape and pose from RGB images in the wild.

Ranked #10 on 3D Hand Pose Estimation on FreiHAND (PA-MPVPE metric)

3D Hand Pose Estimation Pose Prediction

Adversarial Metric Attack and Defense for Person Re-identification

1 code implementation30 Jan 2019 Song Bai, Yingwei Li, Yuyin Zhou, Qizhu Li, Philip H. S. Torr

However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images.

Adversarial Attack Benchmarking +2

Hypergraph Convolution and Hypergraph Attention

1 code implementation23 Jan 2019 Song Bai, Feihu Zhang, Philip H. S. Torr

To efficiently learn deep embeddings on the high-order graph-structured data, we introduce two end-to-end trainable operators to the family of graph neural networks, i. e., hypergraph convolution and hypergraph attention.

Node Classification Representation Learning

Learn to Interpret Atari Agents

1 code implementation29 Dec 2018 Zhao Yang, Song Bai, Li Zhang, Philip H. S. Torr

Deep reinforcement learning (DeepRL) agents surpass human-level performance in many tasks.

Decision Making

Visual Dialogue without Vision or Dialogue

2 code implementations16 Dec 2018 Daniela Massiceti, Puneet K. Dokania, N. Siddharth, Philip H. S. Torr

We characterise some of the quirks and shortcomings in the exploration of Visual Dialogue - a sequential question-answering task where the questions and corresponding answers are related through given visual stimuli.

Question Answering Visual Dialog

Fast Online Object Tracking and Segmentation: A Unifying Approach

3 code implementations CVPR 2019 Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, Philip H. S. Torr

In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach.

Real-Time Visual Tracking Semi-Supervised Semantic Segmentation +2

Proximal Mean-field for Neural Network Quantization

1 code implementation ICCV 2019 Thalaiyasingam Ajanthan, Puneet K. Dokania, Richard Hartley, Philip H. S. Torr

Compressing large Neural Networks (NN) by quantizing the parameters, while maintaining the performance is highly desirable due to reduced memory and time complexity.

Image Classification Quantization

Deeper Interpretability of Deep Networks

no code implementations19 Nov 2018 Tian Xu, Jiayu Zhan, Oliver G. B. Garrod, Philip H. S. Torr, Song-Chun Zhu, Robin A. A. Ince, Philippe G. Schyns

However, understanding the information represented and processed in CNNs remains in most cases challenging.

Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade

1 code implementation29 Oct 2018 Tommaso Cavallari, Stuart Golodetz, Nicholas A. Lord, Julien Valentin, Victor A. Prisacariu, Luigi Di Stefano, Philip H. S. Torr

The adapted forests achieved relocalisation performance that was on par with that of offline forests, and our approach was able to estimate the camera pose in close to real time.

Pose Estimation

SNIP: Single-shot Network Pruning based on Connection Sensitivity

8 code implementations ICLR 2019 Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr

To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task.

Image Classification Network Pruning +1

Weakly- and Semi-Supervised Panoptic Segmentation

1 code implementation ECCV 2018 Qizhu Li, Anurag Arnab, Philip H. S. Torr

We present a weakly supervised model that jointly performs both semantic- and instance-segmentation -- a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks.

Instance Segmentation Panoptic Segmentation +3

With Friends Like These, Who Needs Adversaries?

1 code implementation NeurIPS 2018 Saumya Jetley, Nicholas A. Lord, Philip H. S. Torr

Via a novel experimental analysis, we illustrate some facts about deep convolutional networks for image classification that shed new light on their behaviour and how it connects to the problem of adversaries.

Adversarial Attack Classification +2

Value Propagation Networks

no code implementations ICLR 2018 Nantas Nardelli, Gabriel Synnaeve, Zeming Lin, Pushmeet Kohli, Philip H. S. Torr, Nicolas Usunier

We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments.

Navigate reinforcement-learning +2

Meta-learning with differentiable closed-form solvers

5 code implementations ICLR 2019 Luca Bertinetto, João F. Henriques, Philip H. S. Torr, Andrea Vedaldi

The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data.

BIG-bench Machine Learning Few-Shot Learning +1

Robustness via Deep Low-Rank Representations

no code implementations ICLR 2019 Amartya Sanyal, Varun Kanade, Philip H. S. Torr, Puneet K. Dokania

To achieve low dimensionality of learned representations, we propose an easy-to-use, end-to-end trainable, low-rank regularizer (LR) that can be applied to any intermediate layer representation of a DNN.

General Classification Image Classification +1

Learn To Pay Attention

4 code implementations ICLR 2018 Saumya Jetley, Nicholas A. Lord, Namhoon Lee, Philip H. S. Torr

We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification.

Adversarial Attack General Classification +3

Three Birds One Stone: A General Architecture for Salient Object Segmentation, Edge Detection and Skeleton Extraction

no code implementations27 Mar 2018 Qibin Hou, Jiang-Jiang Liu, Ming-Ming Cheng, Ali Borji, Philip H. S. Torr

Although these tasks are inherently very different, we show that our unified approach performs very well on all of them and works far better than current single-purpose state-of-the-art methods.

Edge Detection Semantic Segmentation

WebSeg: Learning Semantic Segmentation from Web Searches

no code implementations27 Mar 2018 Qibin Hou, Ming-Ming Cheng, Jiang-Jiang Liu, Philip H. S. Torr

In this paper, we improve semantic segmentation by automatically learning from Flickr images associated with a particular keyword, without relying on any explicit user annotations, thus substantially alleviating the dependence on accurate annotations when compared to previous weakly supervised methods.

Semantic Segmentation

Real-Time Dense Stereo Matching With ELAS on FPGA Accelerated Embedded Devices

1 code implementation20 Feb 2018 Oscar Rahnama, Duncan Frost, Ondrej Miksik, Philip H. S. Torr

For many applications in low-power real-time robotics, stereo cameras are the sensors of choice for depth perception as they are typically cheaper and more versatile than their active counterparts.

Stereo Matching Stereo Matching Hand

Devon: Deformable Volume Network for Learning Optical Flow

no code implementations20 Feb 2018 Yao Lu, Jack Valmadre, Heng Wang, Juho Kannala, Mehrtash Harandi, Philip H. S. Torr

State-of-the-art neural network models estimate large displacement optical flow in multi-resolution and use warping to propagate the estimation between two resolutions.

Optical Flow Estimation

FlipDial: A Generative Model for Two-Way Visual Dialogue

no code implementations CVPR 2018 Daniela Massiceti, N. Siddharth, Puneet K. Dokania, Philip H. S. Torr

We are the first to extend this paradigm to full two-way visual dialogue, where our model is capable of generating both questions and answers in sequence based on a visual input, for which we propose a set of novel evaluation measures and metrics.

Visual Dialog Vocal Bursts Valence Prediction

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

1 code implementation ECCV 2018 Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, Philip H. S. Torr

We observe that, in addition to forgetting, a known issue while preserving knowledge, IL also suffers from a problem we call intransigence, inability of a model to update its knowledge.

Incremental Learning

Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation

no code implementations25 Jan 2018 Stuart Golodetz, Tommaso Cavallari, Nicholas A. Lord, Victor A. Prisacariu, David W. Murray, Philip H. S. Torr

Reconstructing dense, volumetric models of real-world 3D scenes is important for many tasks, but capturing large scenes can take significant time, and the risk of transient changes to the scene goes up as the capture time increases.

3D Reconstruction

Piecewise Linear Neural Networks verification: A comparative study

no code implementations ICLR 2018 Rudy Bunel, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, M. Pawan Kumar

Motivated by the need of accelerating progress in this very important area, we investigate the trade-offs of a number of different approaches based on Mixed Integer Programming, Satisfiability Modulo Theory, as well as a novel method based on the Branch-and-Bound framework.

On the Robustness of Semantic Segmentation Models to Adversarial Attacks

1 code implementation CVPR 2018 Anurag Arnab, Ondrej Miksik, Philip H. S. Torr

Deep Neural Networks (DNNs) have demonstrated exceptional performance on most recognition tasks such as image classification and segmentation.

General Classification Image Classification +2

Learning to Compare: Relation Network for Few-Shot Learning

12 code implementations CVPR 2018 Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, Timothy M. Hospedales

Once trained, a RN is able to classify images of new classes by computing relation scores between query images and the few examples of each new class without further updating the network.

Few-Shot Image Classification Few-Shot Learning +1

A Unified View of Piecewise Linear Neural Network Verification

2 code implementations NeurIPS 2018 Rudy Bunel, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, M. Pawan Kumar

The success of Deep Learning and its potential use in many safety-critical applications has motivated research on formal verification of Neural Network (NN) models.

Holistic, Instance-Level Human Parsing

1 code implementation11 Sep 2017 Qizhu Li, Anurag Arnab, Philip H. S. Torr

We address this problem by segmenting the parts of objects at an instance-level, such that each pixel in the image is assigned a part label, as well as the identity of the object it belongs to.

Human Detection Multi-Human Parsing

InfiniTAM v3: A Framework for Large-Scale 3D Reconstruction with Loop Closure

1 code implementation2 Aug 2017 Victor Adrian Prisacariu, Olaf Kähler, Stuart Golodetz, Michael Sapienza, Tommaso Cavallari, Philip H. S. Torr, David W. Murray

Representing the reconstruction volumetrically as a TSDF leads to most of the simplicity and efficiency that can be achieved with GPU implementations of these systems.

3D Reconstruction Simultaneous Localization and Mapping

Spatio-temporal Human Action Localisation and Instance Segmentation in Temporally Untrimmed Videos

no code implementations22 Jul 2017 Suman Saha, Gurkirt Singh, Michael Sapienza, Philip H. S. Torr, Fabio Cuzzolin

Current state-of-the-art human action recognition is focused on the classification of temporally trimmed videos in which only one action occurs per frame.

Action Recognition Instance Segmentation +2

Learning Disentangled Representations with Semi-Supervised Deep Generative Models

1 code implementation NeurIPS 2017 N. Siddharth, Brooks Paige, Jan-Willem van de Meent, Alban Desmaison, Noah D. Goodman, Pushmeet Kohli, Frank Wood, Philip H. S. Torr

We propose to learn such representations using model architectures that generalise from standard VAEs, employing a general graphical model structure in the encoder and decoder.

Representation Learning

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

3 code implementations CVPR 2017 Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B. Choy, Philip H. S. Torr, Manmohan Chandraker

DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i. e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents.

Future prediction Multi Future Trajectory Prediction +1

Multi-Agent Diverse Generative Adversarial Networks

1 code implementation CVPR 2018 Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Philip H. S. Torr, Puneet K. Dokania

Second, to enforce that different generators capture diverse high probability modes, the discriminator of MAD-GAN is designed such that along with finding the real and fake samples, it is also required to identify the generator that generated the given fake sample.

Face Generation Image-to-Image Translation +1

Pixelwise Instance Segmentation with a Dynamically Instantiated Network

1 code implementation CVPR 2017 Anurag Arnab, Philip H. S. Torr

This subnetwork uses the initial category-level segmentation, along with cues from the output of an object detector, within an end-to-end CRF to predict instances.

Instance Segmentation object-detection +2

Incremental Tube Construction for Human Action Detection

1 code implementation5 Apr 2017 Harkirat Singh Behl, Michael Sapienza, Gurkirt Singh, Suman Saha, Fabio Cuzzolin, Philip H. S. Torr

In this work, we introduce a real-time and online joint-labelling and association algorithm for action detection that can incrementally construct space-time action tubes on the most challenging action videos in which different action categories occur concurrently.

Action Detection

On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation

no code implementations CVPR 2017 Tommaso Cavallari, Stuart Golodetz, Nicholas A. Lord, Julien Valentin, Luigi Di Stefano, Philip H. S. Torr

Camera relocalisation is an important problem in computer vision, with applications in simultaneous localisation and mapping, virtual/augmented reality and navigation.

Camera Relocalization regression

ROAM: a Rich Object Appearance Model with Application to Rotoscoping

no code implementations CVPR 2017 Ondrej Miksik, Juan-Manuel Pérez-Rúa, Philip H. S. Torr, Patrick Pérez

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines.

Playing Doom with SLAM-Augmented Deep Reinforcement Learning

1 code implementation1 Dec 2016 Shehroze Bhatti, Alban Desmaison, Ondrej Miksik, Nantas Nardelli, N. Siddharth, Philip H. S. Torr

A number of recent approaches to policy learning in 2D game domains have been successful going directly from raw input images to actions.

object-detection Object Detection +3