Search Results for author: Tao Xiang

Found 135 papers, 49 papers with code

SceneTrilogy: On Scene Sketches and its Relationship with Text and Photo

no code implementations25 Apr 2022 Pinaki Nath Chowdhury, Ayan Kumar Bhunia, Tao Xiang, Yi-Zhe Song

We spell out a few insights on the complementarity of each modality for scene understanding, and study for the first time a series of scene-specific applications like joint sketch- and text-based image retrieval, sketch captioning.

Image Retrieval Scene Understanding

UIGR: Unified Interactive Garment Retrieval

1 code implementation6 Apr 2022 Xiao Han, Sen He, Li Zhang, Yi-Zhe Song, Tao Xiang

In this paper, we propose a Unified Interactive Garment Retrieval (UIGR) framework to unify TGR and VCR.

Style-Based Global Appearance Flow for Virtual Try-On

1 code implementation CVPR 2022 Sen He, Yi-Zhe Song, Tao Xiang

To achieve this, a key step is garment warping which spatially aligns the target garment with the corresponding body parts in the person image.

Virtual Try-on

Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches

no code implementations CVPR 2022 Ayan Kumar Bhunia, Viswanatha Reddy Gajjala, Subhadeep Koley, Rohit Kundu, Aneeshan Sain, Tao Xiang, Yi-Zhe Song

In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints).

class-incremental learning Graph Attention +2

Sketch3T: Test-Time Training for Zero-Shot SBIR

no code implementations CVPR 2022 Aneeshan Sain, Ayan Kumar Bhunia, Vaishnav Potlapalli, Pinaki Nath Chowdhury, Tao Xiang, Yi-Zhe Song

In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i. e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result.

Meta-Learning Sketch-Based Image Retrieval

Dynamic Instance Domain Adaptation

1 code implementation9 Mar 2022 Zhongying Deng, Kaiyang Zhou, Da Li, Junjun He, Yi-Zhe Song, Tao Xiang

In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain.

Unsupervised Domain Adaptation

FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context

no code implementations4 Mar 2022 Pinaki Nath Chowdhury, Aneeshan Sain, Yulia Gryaditskaya, Ayan Kumar Bhunia, Tao Xiang, Yi-Zhe Song

In addition, we propose new solutions enabled by our dataset (i) We adopt meta-learning to show how the retrieval model can be fine-tuned to a new user style given just a small set of sketches, (ii) We extend a popular vector sketch LSTM-based encoder to handle sketches with larger complexity than was supported by previous work.

Image Captioning Image Retrieval +1

One Sketch for All: One-Shot Personalized Sketch Segmentation

no code implementations20 Dec 2021 Anran Qi, Yulia Gryaditskaya, Tao Xiang, Yi-Zhe Song

We aim to segment all sketches belonging to the same category provisioned with a single sketch with a given part annotation while (i) preserving the parts semantics embedded in the exemplar, and (ii) being robust to input style and abstraction.

Hybrid Graph Neural Networks for Few-Shot Learning

no code implementations13 Dec 2021 Tianyuan Yu, Sen He, Yi-Zhe Song, Tao Xiang

This is because they use an instance GNN as a label propagation/classification module, which is jointly meta-learned with a feature embedding network.

Few-Shot Learning

Domain Attention Consistency for Multi-Source Domain Adaptation

1 code implementation6 Nov 2021 Zhongying Deng, Kaiyang Zhou, Yongxin Yang, Tao Xiang

Importantly, the attention module is supervised by a consistency loss, which is imposed on the distributions of channel attention weights between source and target domains.

Domain Adaptation

Towards artificial general intelligence via a multimodal foundation model

1 code implementation27 Oct 2021 Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun, Ji-Rong Wen

To overcome this limitation and take a solid step towards artificial general intelligence (AGI), we develop a foundation model pre-trained with huge multimodal data, which can be quickly adapted for various downstream cognitive tasks.

Image Classification Reading Comprehension +2

SOFT: Softmax-free Transformer with Linear Complexity

no code implementations NeurIPS 2021 Jiachen Lu, Jinghan Yao, Junge Zhang, Xiatian Zhu, Hang Xu, Weiguo Gao, Chunjing Xu, Tao Xiang, Li Zhang

Crucially, with a linear complexity, much longer token sequences are permitted in SOFT, resulting in superior trade-off between accuracy and complexity.

Natural Language Processing

Few-Shot Temporal Action Localization with Query Adaptive Transformer

1 code implementation20 Oct 2021 Sauradip Nag, Xiatian Zhu, Tao Xiang

Further, a novel FS-TAL model is proposed which maximizes the knowledge transfer from training classes whilst enabling the model to be dynamically adapted to both the new class and each video of that class simultaneously.

Action Segmentation Few Shot Temporal Action Localization +4

Text-Based Person Search with Limited Data

1 code implementation20 Oct 2021 Xiao Han, Sen He, Li Zhang, Tao Xiang

Firstly, to fully utilize the existing small-scale benchmarking datasets for more discriminative feature learning, we introduce a cross-modal momentum contrastive learning framework to enrich the training data for a given mini-batch.

 Ranked #1 on Text based Person Retrieval on CUHK-PEDES (using extra training data)

Contrastive Learning Cross-Modal Retrieval +4

SketchODE: Learning neural sketch representation in continuous time

no code implementations ICLR 2022 Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song

Learning meaningful representations for chirographic drawing data such as sketches, handwriting, and flowcharts is a gateway for understanding and emulating human creative expression.

Data Augmentation

Temporal Action Localization with Global Segmentation Mask Transformers

no code implementations29 Sep 2021 Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, Tao Xiang

In this paper, to address the above two challenges, a novel {\em Global Segmentation Mask Transformer} (GSMT) is proposed.

object-detection Object Detection +1

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer

1 code implementation ICCV 2021 Zhihe Lu, Sen He, Xiatian Zhu, Li Zhang, Yi-Zhe Song, Tao Xiang

A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels).

Few-Shot Semantic Segmentation Meta-Learning +1

Disentangled Lifespan Face Synthesis

no code implementations ICCV 2021 Sen He, Wentong Liao, Michael Ying Yang, Yi-Zhe Song, Bodo Rosenhahn, Tao Xiang

The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving.

Face Generation

Global Aggregation then Local Distribution for Scene Parsing

1 code implementation28 Jul 2021 Xiangtai Li, Li Zhang, Guangliang Cheng, Kuiyuan Yang, Yunhai Tong, Xiatian Zhu, Tao Xiang

Modelling long-range contextual relationships is critical for pixel-wise prediction tasks such as semantic segmentation.

Scene Parsing Semantic Segmentation

MixStyle Neural Networks for Domain Generalization and Adaptation

2 code implementations5 Jul 2021 Kaiyang Zhou, Yongxin Yang, Yu Qiao, Tao Xiang

In this work, we address domain generalization with MixStyle, a plug-and-play, parameter-free module that is simply inserted to shallow CNN layers and requires no modification to training objectives.

Domain Generalization Object Recognition +2

L2M-GAN: Learning To Manipulate Latent Space Semantics for Facial Attribute Editing

2 code implementations CVPR 2021 Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu, Tao Xiang

To overcome these limitations, we propose a novel latent space factorization model, called L2M-GAN, which is learned end-to-end and effective for editing both local and global attributes.


Domain Generalization with MixStyle

3 code implementations ICLR 2021 Kaiyang Zhou, Yongxin Yang, Yu Qiao, Tao Xiang

Our method, termed MixStyle, is motivated by the observation that visual domain is closely related to image style (e. g., photo vs.~sketch images).

Domain Generalization

Cloud2Curve: Generation and Vectorization of Parametric Sketches

1 code implementation CVPR 2021 Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song

Analysis of human sketches in deep learning has advanced immensely through the use of waypoint-sequences rather than raster-graphic representations.

StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval

no code implementations CVPR 2021 Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song

With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.

Disentanglement Meta-Learning +1

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

1 code implementation CVPR 2021 Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Yongxin Yang, Tao Xiang, Yi-Zhe Song

A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.

Cross-Modal Retrieval Semi-Supervised Sketch Based Image Retrieval +1

Context-Aware Layout to Image Generation with Enhanced Object Appearance

1 code implementation CVPR 2021 Sen He, Wentong Liao, Michael Ying Yang, Yongxin Yang, Yi-Zhe Song, Bodo Rosenhahn, Tao Xiang

We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.

Layout-to-Image Generation

Domain Generalization: A Survey

2 code implementations3 Mar 2021 Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, Chen Change Loy

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.

Action Recognition Data Augmentation +8

Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning

no code implementations23 Jan 2021 Yizhao Gao, Nanyi Fei, Guangzhen Liu, Zhiwu Lu, Tao Xiang, Songfang Huang

First, data augmentations are introduced to both the support and query sets with each sample now being represented as an augmented embedding (AE) composed of concatenated embeddings of both the original and augmented versions.

Few-Shot Learning

Few-shot Action Recognition with Prototype-centered Attentive Learning

1 code implementation20 Jan 2021 Xiatian Zhu, Antoine Toisoul, Juan-Manuel Perez-Rua, Li Zhang, Brais Martinez, Tao Xiang

Extensive experiments on four standard few-shot action benchmarks show that our method clearly outperforms previous state-of-the-art methods, with the improvement particularly significant (10+\%) on the most challenging fine-grained action recognition benchmark.

Contrastive Learning Few Shot Action Recognition +2

Local Black-box Adversarial Attacks: A Query Efficient Approach

no code implementations4 Jan 2021 Tao Xiang, Hangcheng Liu, Shangwei Guo, Tianwei Zhang, Xiaofeng Liao

Based on this property, we identify the discriminative areas of a given clean example easily for local perturbations.

Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw

no code implementations1 Jan 2021 Yuqi Huo, Mingyu Ding, Haoyu Lu, Zhiwu Lu, Tao Xiang, Ji-Rong Wen, Ziyuan Huang, Jianwen Jiang, Shiwei Zhang, Mingqian Tang, Songfang Huang, Ping Luo

With the constrained jigsaw puzzles, instead of solving them directly, which could still be extremely hard, we carefully design four surrogate tasks that are more solvable but meanwhile still ensure that the learned representation is sensitive to spatiotemporal continuity at both the local and global levels.

Representation Learning

IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning

1 code implementation ICLR 2021 Manli Zhang, Jianhong Zhang, Zhiwu Lu, Tao Xiang, Mingyu Ding, Songfang Huang

Importantly, at the episode-level, two SSL-FSL hybrid learning objectives are devised: (1) The consistency across the predictions of an FSL classifier from different extended episodes is maximized as an episode-level pretext task.

Few-Shot Learning Self-Supervised Learning +1

MELR: Meta-Learning via Modeling Episode-Level Relationships for Few-Shot Learning

no code implementations ICLR 2021 Nanyi Fei, Zhiwu Lu, Tao Xiang, Songfang Huang

Most recent few-shot learning (FSL) approaches are based on episodic training whereby each episode samples few training instances (shots) per class to imitate the test condition.

Few-Shot Learning

Z-Score Normalization, Hubness, and Few-Shot Learning

no code implementations ICCV 2021 Nanyi Fei, Yizhao Gao, Zhiwu Lu, Tao Xiang

This means that these methods are prone to the hubness problem, that is, a certain class prototype becomes the nearest neighbor of many test instances regardless which classes they belong to.

Few-Shot Learning

Margin-Based Transfer Bounds for Meta Learning with Deep Feature Embedding

no code implementations2 Dec 2020 Jiechao Guan, Zhiwu Lu, Tao Xiang, Timothy Hospedales

By transferring knowledge learned from seen/previous tasks, meta learning aims to generalize well to unseen/future tasks.

Classification General Classification +2

Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval

1 code implementation29 Jul 2020 Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song

In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.

Sketch-Based Image Retrieval

On Learning Semantic Representations for Million-Scale Free-Hand Sketches

1 code implementation7 Jul 2020 Peng Xu, Yongye Huang, Tongtong Yuan, Tao Xiang, Timothy M. Hospedales, Yi-Zhe Song, Liang Wang

Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches.

Learning Semantic Representations Zero-Shot Learning

BézierSketch: A generative model for scalable vector sketches

1 code implementation ECCV 2020 Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song

The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.

Image Generation

Egocentric Action Recognition by Video Attention and Temporal Context

no code implementations3 Jul 2020 Juan-Manuel Perez-Rua, Antoine Toisoul, Brais Martinez, Victor Escorcia, Li Zhang, Xiatian Zhu, Tao Xiang

In this challenge, action recognition is posed as the problem of simultaneously predicting a single `verb' and `noun' class label given an input trimmed video clip.

Action Recognition

Topology-aware Differential Privacy for Decentralized Image Classification

no code implementations14 Jun 2020 Shangwei Guo, Tianwei Zhang, Guowen Xu, Han Yu, Tao Xiang, Yang Liu

In this paper, we design Top-DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems.

Classification Image Classification

Long-Term Cloth-Changing Person Re-identification

no code implementations26 May 2020 Xuelin Qian, Wenxuan Wang, Li Zhang, Fangrui Zhu, Yanwei Fu, Tao Xiang, Yu-Gang Jiang, xiangyang xue

Specifically, we consider that under cloth-changes, soft-biometrics such as body shape would be more reliable.

Person Re-Identification

Domain-Adaptive Few-Shot Learning

1 code implementation19 Mar 2020 An Zhao, Mingyu Ding, Zhiwu Lu, Tao Xiang, Yulei Niu, Jiechao Guan, Ji-Rong Wen, Ping Luo

Existing few-shot learning (FSL) methods make the implicit assumption that the few target class samples are from the same domain as the source class samples.

Domain Adaptation Few-Shot Learning

Domain Adaptive Ensemble Learning

2 code implementations16 Mar 2020 Kaiyang Zhou, Yongxin Yang, Yu Qiao, Tao Xiang

Each such classifier is an expert to its own domain and a non-expert to others.

Domain Generalization Ensemble Learning +2

Deep Domain-Adversarial Image Generation for Domain Generalisation

no code implementations12 Mar 2020 Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, Tao Xiang

This is achieved by having a learning objective formulated to ensure that the generated data can be correctly classified by the label classifier while fooling the domain classifier.

Domain Generalization Image Generation

Incremental Few-Shot Object Detection

no code implementations CVPR 2020 Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy Hospedales, Tao Xiang

To this end we propose OpeN-ended Centre nEt (ONCE), a detector designed for incrementally learning to detect novel class objects with few examples.

Few-Shot Learning Few-Shot Object Detection +2

AdarGCN: Adaptive Aggregation GCN for Few-Shot Learning

no code implementations28 Feb 2020 Jianhong Zhang, Manli Zhang, Zhiwu Lu, Tao Xiang, Ji-Rong Wen

To address this problem, we propose a graph convolutional network (GCN)-based label denoising (LDN) method to remove the irrelevant images.

Denoising Few-Shot Learning +1

Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval

1 code implementation24 Feb 2020 Ayan Kumar Bhunia, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, Yi-Zhe Song

Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.

Cross-Modal Retrieval On-the-Fly Sketch Based Image Retrieval

Byzantine-resilient Decentralized Stochastic Gradient Descent

no code implementations20 Feb 2020 Shangwei Guo, Tianwei Zhang, Han Yu, Xiaofei Xie, Lei Ma, Tao Xiang, Yang Liu

It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes.

Edge-computing Image Classification

Meta-Learning across Meta-Tasks for Few-Shot Learning

no code implementations11 Feb 2020 Nanyi Fei, Zhiwu Lu, Yizhao Gao, Jia Tian, Tao Xiang, Ji-Rong Wen

In this paper, we argue that the inter-meta-task relationships should be exploited and those tasks are sampled strategically to assist in meta-learning.

Domain Adaptation Few-Shot Learning +1

Few-Shot Learning as Domain Adaptation: Algorithm and Analysis

no code implementations6 Feb 2020 Jiechao Guan, Zhiwu Lu, Tao Xiang, Ji-Rong Wen

Specifically, armed with a set transformer based attention module, we construct each episode with two sub-episodes without class overlap on the seen classes to simulate the domain shift between the seen and unseen classes.

Domain Adaptation Few-Shot Image Classification

Deep Learning for Person Re-identification: A Survey and Outlook

4 code implementations13 Jan 2020 Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven C. H. Hoi

The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets.

Cross-Modal Person Re-Identification Metric Learning +2

Deep Learning for Free-Hand Sketch: A Survey

2 code implementations8 Jan 2020 Peng Xu, Timothy M. Hospedales, Qiyue Yin, Yi-Zhe Song, Tao Xiang, Liang Wang

Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present.

Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch

2 code implementations22 Oct 2019 Kaiyang Zhou, Tao Xiang

Person re-identification (re-ID), which aims to re-identify people across different camera views, has been significantly advanced by deep learning in recent years, particularly with convolutional neural networks (CNNs).

Person Re-Identification

Learning Generalisable Omni-Scale Representations for Person Re-Identification

2 code implementations15 Oct 2019 Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, Tao Xiang

An effective person re-identification (re-ID) model should learn feature representations that are both discriminative, for distinguishing similar-looking people, and generalisable, for deployment across datasets without any adaptation.

Unsupervised Domain Adaptation Unsupervised Person Re-Identification

Simple and Effective Stochastic Neural Networks

no code implementations25 Sep 2019 Tianyuan Yu, Yongxin Yang, Da Li, Timothy Hospedales, Tao Xiang

Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian neural networks, variational information bottleneck (VIB) and noise regularized learning.

Adversarial Attack Adversarial Defense

Few-Shot Learning with Global Class Representations

2 code implementations ICCV 2019 Tiange Luo, Aoxue Li, Tao Xiang, Weiran Huang, Li-Wei Wang

In this paper, we propose to tackle the challenging few-shot learning (FSL) problem by learning global class representations using both base and novel class training samples.

Generalized Few-Shot Classification

Goal-Driven Sequential Data Abstraction

no code implementations ICCV 2019 Umar Riaz Muhammad, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, Yi-Zhe Song

In the former one asks whether a machine can `understand' enough about the meaning of input data to produce a meaningful but more compact abstraction.

General Reinforcement Learning reinforcement-learning

Omni-Scale Feature Learning for Person Re-Identification

10 code implementations ICCV 2019 Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, Tao Xiang

As an instance-level recognition problem, person re-identification (ReID) relies on discriminative features, which not only capture different spatial scales but also encapsulate an arbitrary combination of multiple scales.

Person Re-Identification

Compressing deep neural networks by matrix product operators

1 code implementation11 Apr 2019 Ze-Feng Gao, Song Cheng, Rong-Qiang He, Z. Y. Xie, Hui-Hai Zhao, Zhong-Yi Lu, Tao Xiang

A deep neural network is a parametrization of a multilayer mapping of signals in terms of many alternatively arranged linear and nonlinear transformations.

Differentiable Programming Tensor Networks

3 code implementations22 Mar 2019 Hai-Jun Liao, Jin-Guo Liu, Lei Wang, Tao Xiang

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation (AD).

Strongly Correlated Electrons Quantum Physics

Tree Tensor Networks for Generative Modeling

no code implementations8 Jan 2019 Song Cheng, Lei Wang, Tao Xiang, Pan Zhang

Matrix product states (MPS), a tensor network designed for one-dimensional quantum systems, has been recently proposed for generative modeling of natural data (such as images) in terms of `Born machine'.

Tensor Networks

Zero-Shot Learning with Sparse Attribute Propagation

no code implementations11 Dec 2018 Nanyi Fei, Jiechao Guan, Zhiwu Lu, Tao Xiang, Ji-Rong Wen

The standard approach to ZSL requires a set of training images annotated with seen class labels and a semantic descriptor for seen/unseen classes (attribute vector is the most widely used).

Image Retrieval Zero-Shot Learning

Disjoint Label Space Transfer Learning with Common Factorised Space

no code implementations6 Dec 2018 Xiaobin Chang, Yongxin Yang, Tao Xiang, Timothy M. Hospedales

In this paper, a unified approach is presented to transfer learning that addresses several source and target domain label-space and annotation assumptions with a single model.

Transfer Learning Unsupervised Domain Adaptation

Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning

no code implementations19 Oct 2018 Aoxue Li, Zhiwu Lu, Jiechao Guan, Tao Xiang, Li-Wei Wang, Ji-Rong Wen

Inspired by the fact that an unseen class is not exactly `unseen' if it belongs to the same superclass as a seen class, we propose a novel inductive ZSL model that leverages superclasses as the bridge between seen and unseen classes to narrow the domain gap.

Few-Shot Learning Zero-Shot Learning

Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning

no code implementations19 Oct 2018 Zhiwu Lu, Jiechao Guan, Aoxue Li, Tao Xiang, An Zhao, Ji-Rong Wen

Specifically, we assume that each synthesised data point can belong to any unseen class; and the most likely two class candidates are exploited to learn a robust projection function in a competitive fashion.

Few-Shot Learning Zero-Shot Learning

Deep Factorised Inverse-Sketching

no code implementations ECCV 2018 Kaiyue Pang, Da Li, Jifei Song, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales

Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included.

Sketch-Based Image Retrieval Style Transfer

SketchyScene: Richly-Annotated Scene Sketches

2 code implementations ECCV 2018 Changqing Zou, Qian Yu, Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengying Gao, Baoquan Chen, Hao Zhang

We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.

Colorization Image Retrieval +1

Person Re-Identification in Identity Regression Space

no code implementations25 Jun 2018 Hanxiao Wang, Xiatian Zhu, Shaogang Gong, Tao Xiang

Most existing person re-identification (re-id) methods are unsuitable for real-world deployment due to two reasons: Unscalability to large population size, and Inadaptability over time.

Incremental Learning Person Re-Identification

Learning to Sketch with Shortcut Cycle Consistency

no code implementations CVPR 2018 Jifei Song, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process.

Multi-Task Learning Sketch-Based Image Retrieval

Learning Deep Sketch Abstraction

no code implementations CVPR 2018 Umar Riaz Muhammad, Yongxin Yang, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales

Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR).

Sketch-Based Image Retrieval Sketch Recognition

SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval

1 code implementation CVPR 2018 Peng Xu, Yongye Huang, Tongtong Yuan, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, Zhanyu Ma, Jun Guo

Key to our network design is the embedding of unique characteristics of human sketch, where (i) a two-branch CNN-RNN architecture is adapted to explore the temporal ordering of strokes, and (ii) a novel hashing loss is specifically designed to accommodate both the temporal and abstract traits of sketches.

Sketch Recognition

Multi-Level Factorisation Net for Person Re-Identification

no code implementations CVPR 2018 Xiaobin Chang, Timothy M. Hospedales, Tao Xiang

Key to effective person re-identification (Re-ID) is modelling discriminative and view-invariant factors of person appearance at both high and low semantic levels.

Person Re-Identification

Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward

6 code implementations29 Dec 2017 Kaiyang Zhou, Yu Qiao, Tao Xiang

Video summarization aims to facilitate large-scale video browsing by producing short, concise summaries that are diverse and representative of original videos.

Decision Making reinforcement-learning +2

Pose-Normalized Image Generation for Person Re-identification

2 code implementations ECCV 2018 Xuelin Qian, Yanwei Fu, Tao Xiang, Wenxuan Wang, Jie Qiu, Yang Wu, Yu-Gang Jiang, xiangyang xue

Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations.

Image Generation Person Re-Identification +1

Learning to Compare: Relation Network for Few-Shot Learning

9 code implementations CVPR 2018 Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, Timothy M. Hospedales

Once trained, a RN is able to classify images of new classes by computing relation scores between query images and the few examples of each new class without further updating the network.

Few-Shot Image Classification Zero-Shot Learning

Recent Advances in Zero-shot Recognition

no code implementations13 Oct 2017 Yanwei Fu, Tao Xiang, Yu-Gang Jiang, xiangyang xue, Leonid Sigal, Shaogang Gong

With the recent renaissance of deep convolution neural networks, encouraging breakthroughs have been achieved on the supervised recognition tasks, where each class has sufficient training data and fully annotated training data.

Open Set Learning Zero-Shot Learning

Multi-scale Deep Learning Architectures for Person Re-identification

no code implementations ICCV 2017 Xuelin Qian, Yanwei Fu, Yu-Gang Jiang, Tao Xiang, xiangyang xue

Our model is able to learn deep discriminative feature representations at different scales and automatically determine the most suitable scales for matching.

Person Re-Identification

Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation

no code implementations5 Sep 2017 Yulei Niu, Zhiwu Lu, Ji-Rong Wen, Tao Xiang, Shih-Fu Chang

In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept; 2) how to annotate an image with the optimal number of class labels.

Weakly Supervised Image Annotation and Segmentation with Objects and Attributes

no code implementations8 Aug 2017 Zhiyuan Shi, Yongxin Yang, Timothy M. Hospedales, Tao Xiang

We propose to model complex visual scenes using a non-parametric Bayesian model learned from weakly labelled images abundant on media sharing sites such as Flickr.

object-detection Object Detection +2

Scalable and Effective Deep CCA via Soft Decorrelation

no code implementations CVPR 2018 Xiaobin Chang, Tao Xiang, Timothy M. Hospedales

Specifically, exact decorrelation is replaced by soft decorrelation via a mini-batch based Stochastic Decorrelation Loss (SDL) to be optimised jointly with the other training objectives.


Zero-Shot Fine-Grained Classification by Deep Feature Learning with Semantics

no code implementations4 Jul 2017 Aoxue Li, Zhiwu Lu, Li-Wei Wang, Tao Xiang, Xinqi Li, Ji-Rong Wen

In this paper, to address the two issues, we propose a two-phase framework for recognizing images from unseen fine-grained classes, i. e. zero-shot fine-grained classification.

Domain Adaptation Fine-Grained Image Classification +2

Actor-Critic Sequence Training for Image Captioning

no code implementations29 Jun 2017 Li Zhang, Flood Sung, Feng Liu, Tao Xiang, Shaogang Gong, Yongxin Yang, Timothy M. Hospedales

Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing.

Image Captioning reinforcement-learning

Bayesian Joint Modelling for Object Localisation in Weakly Labelled Images

no code implementations19 Jun 2017 Zhiyuan Shi, Timothy M. Hospedales, Tao Xiang

We address the problem of localisation of objects as bounding boxes in images and videos with weak labels.

Domain Adaptation Transfer Learning

Transferring a Semantic Representation for Person Re-Identification and Search

no code implementations CVPR 2015 Zhiyuan Shi, Timothy M. Hospedales, Tao Xiang

Learning semantic attributes for person re-identification and description-based person search has gained increasing interest due to attributes' great potential as a pose and view-invariant representation.

Person Re-Identification Person Search

Deep Mutual Learning

8 code implementations CVPR 2018 Ying Zhang, Tao Xiang, Timothy M. Hospedales, Huchuan Lu

Model distillation is an effective and widely used technique to transfer knowledge from a teacher to a student network.

Person Re-Identification

Bayesian Joint Topic Modelling for Weakly Supervised Object Localisation

no code implementations9 May 2017 Zhiyuan Shi, Timothy M. Hospedales, Tao Xiang

(3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning.

Transfer Learning by Ranking for Weakly Supervised Object Annotation

no code implementations2 May 2017 Zhiyuan Shi, Parthipan Siva, Tao Xiang

Most existing approaches to training object detectors rely on fully supervised learning, which requires the tedious manual annotation of object location in a training set.

Learning-To-Rank Transfer Learning

Semantic Autoencoder for Zero-Shot Learning

4 code implementations CVPR 2017 Elyor Kodirov, Tao Xiang, Shaogang Gong

We show that with this additional reconstruction constraint, the learned projection function from the seen classes is able to generalise better to the new unseen classes.

Zero-Shot Learning

Equivalence of restricted Boltzmann machines and tensor network states

1 code implementation17 Jan 2017 Jing Chen, Song Cheng, Haidong Xie, Lei Wang, Tao Xiang

Conversely, we give sufficient and necessary conditions to determine whether a TNS can be transformed into an RBM of given architectures.

Recommendation Systems

Human-In-The-Loop Person Re-Identification

no code implementations5 Dec 2016 Hanxiao Wang, Shaogang Gong, Xiatian Zhu, Tao Xiang

Current person re-identification (re-id) methods assume that (1) pre-labelled training data are available for every camera pair, (2) the gallery size for re-identification is moderate.

Ensemble Learning Incremental Learning +1

Highly Efficient Regression for Scalable Person Re-Identification

no code implementations5 Dec 2016 Hanxiao Wang, Shaogang Gong, Tao Xiang

Existing person re-identification models are poor for scaling up to large data required in real-world applications due to: (1) Complexity: They employ complex models for optimal performance resulting in high computational cost for training at a large scale; (2) Inadaptability: Once trained, they are unsuitable for incremental update to incorporate any new data available.

Active Learning Person Re-Identification

Deep Transfer Learning for Person Re-identification

1 code implementation16 Nov 2016 Mengyue Geng, Yao-Wei Wang, Tao Xiang, Yonghong Tian

Second, a two-stepped fine-tuning strategy is developed to transfer knowledge from auxiliary datasets.

General Classification Image Classification +2

Sketch Me That Shoe

no code implementations CVPR 2016 Qian Yu, Feng Liu, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, Chen-Change Loy

We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images.

Data Augmentation Sketch-Based Image Retrieval

Unsupervised Cross-Dataset Transfer Learning for Person Re-Identification

no code implementations CVPR 2016 Peixi Peng, Tao Xiang, Yao-Wei Wang, Massimiliano Pontil, Shaogang Gong, Tiejun Huang, Yonghong Tian

Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.

Dictionary Learning Person Re-Identification +1

Partial Person Re-Identification

no code implementations ICCV 2015 Wei-Shi Zheng, Xiang Li, Tao Xiang, Shengcai Liao, Jian-Huang Lai, Shaogang Gong

We address a new partial person re-identification (re-id) problem, where only a partial observation of a person is available for matching across different non-overlapping camera views.

Person Re-Identification

Unsupervised Domain Adaptation for Zero-Shot Learning

no code implementations ICCV 2015 Elyor Kodirov, Tao Xiang, Zhen-Yong Fu, Shaogang Gong

Zero-shot learning (ZSL) can be considered as a special case of transfer learning where the source and target domains have different tasks/label spaces and the target domain is unlabelled, providing little guidance for the knowledge transfer.

Action Recognition Transfer Learning +1

Multi-Scale Learning for Low-Resolution Person Re-Identification

no code implementations ICCV 2015 Xiang Li, Wei-Shi Zheng, Xiaojuan Wang, Tao Xiang, Shaogang Gong

In real world person re-identification (re-id), images of people captured at very different resolutions from different locations need be matched.

Person Re-Identification

Making Better Use of Edges via Perceptual Grouping

no code implementations CVPR 2015 Yonggang Qi, Yi-Zhe Song, Tao Xiang, Honggang Zhang, Timothy Hospedales, Yi Li, Jun Guo

We propose a perceptual grouping framework that organizes image edges into meaningful structures and demonstrate its usefulness on various computer vision tasks.

Learning-To-Rank Sketch-Based Image Retrieval

Zero-Shot Object Recognition by Semantic Manifold Distance

no code implementations CVPR 2015 Zhenyong Fu, Tao Xiang, Elyor Kodirov, Shaogang Gong

The semantic manifold structure is used to redefine the distance metric in the semantic embedding space for more effective ZSL.

Object Recognition Transfer Learning +1

Weakly Supervised Learning of Objects, Attributes and their Associations

no code implementations31 Mar 2015 Zhiyuan Shi, Yongxin Yang, Timothy M. Hospedales, Tao Xiang

When humans describe images they tend to use combinations of nouns and adjectives, corresponding to objects and their associated attributes respectively.

Transductive Multi-label Zero-shot Learning

no code implementations26 Mar 2015 Yanwei Fu, Yongxin Yang, Tim Hospedales, Tao Xiang, Shaogang Gong

Zero-shot learning has received increasing interest as a means to alleviate the often prohibitive expense of annotating training data for large scale recognition problems.

Multi-label zero-shot learning

Sketch-a-Net that Beats Humans

1 code implementation30 Jan 2015 Qian Yu, Yongxin Yang, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans.

Sketch Recognition

Robust Subjective Visual Property Prediction from Crowdsourced Pairwise Labels

no code implementations25 Jan 2015 Yanwei Fu, Timothy M. Hospedales, Tao Xiang, Jiechao Xiong, Shaogang Gong, Yizhou Wang, Yuan YAO

In this paper, we propose a more principled way to identify annotation outliers by formulating the subjective visual property prediction task as a unified robust learning to rank problem, tackling both the outlier detection and learning to rank jointly.

Learning-To-Rank Outlier Detection

Transductive Multi-view Zero-Shot Learning

no code implementations19 Jan 2015 Yanwei Fu, Timothy M. Hospedales, Tao Xiang, Shaogang Gong

A projection from a low-level feature space to the semantic representation space is learned from the auxiliary dataset and is applied without adaptation to the target dataset.

Transfer Learning Zero-Shot Learning

Semantic Graph for Zero-Shot Learning

no code implementations16 Jun 2014 Zhen-Yong Fu, Tao Xiang, Shaogang Gong

Specifically, in contrast to previous work which ignores the semantic relationships between seen classes and focus merely on those between seen and unseen classes, in this paper a novel approach based on a semantic graph is proposed to represent the relationships between all the seen and unseen class in a semantic word space.

Transfer Learning Zero-Shot Learning

Can Image-Level Labels Replace Pixel-Level Labels for Image Parsing

no code implementations7 Mar 2014 Zhiwu Lu, Zhen-Yong Fu, Tao Xiang, Li-Wei Wang, Ji-Rong Wen

By oversegmenting all the images into regions, we formulate noisily tagged image parsing as a weakly supervised sparse learning problem over all the regions, where the initial labels of each region are inferred from image-level labels.

Sparse Learning

Cumulative Attribute Space for Age and Crowd Density Estimation

no code implementations CVPR 2013 Ke Chen, Shaogang Gong, Tao Xiang, Chen Change Loy

A number of computer vision problems such as human age estimation, crowd density estimation and body/face pose (view angle) estimation can be formulated as a regression problem by learning a mapping function between a high dimensional vector-formed feature input and a scalarvalued output.

Age Estimation Crowd Counting +1

Cannot find the paper you are looking for? You can Submit a new open access paper.