Search Results for author: Yi-Zhe Song

Found 54 papers, 23 papers with code

A Tree-Structured Decoder for Image-to-Markup Generation

no code implementations ICML 2020 Jianshu Zhang, Jun Du, Yongxin Yang, Yi-Zhe Song, Si Wei, Li-Rong Dai

Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup.

SketchLattice: Latticed Representation for Sketch Manipulation

no code implementations ICCV 2021 Yonggang Qi, Guoyao Su, Pinaki Nath Chowdhury, Mingkang Li, Yi-Zhe Song

The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches.

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer

1 code implementation ICCV 2021 Zhihe Lu, Sen He, Xiatian Zhu, Li Zhang, Yi-Zhe Song, Tao Xiang

A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels).

Few-Shot Semantic Segmentation Meta-Learning +1

Disentangled Lifespan Face Synthesis

no code implementations ICCV 2021 Sen He, Wentong Liao, Michael Ying Yang, Yi-Zhe Song, Bodo Rosenhahn, Tao Xiang

The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving.

Face Generation

Towards the Unseen: Iterative Text Recognition by Distilling from Errors

no code implementations ICCV 2021 Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Yi-Zhe Song

Our framework is iterative in nature, in that it utilises predicted knowledge of character sequences from a previous iteration, to augment the main network in improving the next prediction.

Affine Transformation

Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation

no code implementations ICCV 2021 Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury, Yi-Zhe Song

In this paper, for the first time, we argue for their unification -- we aim for a single model that can compete favourably with two separate state-of-the-art STR and HTR models.

Handwriting Recognition Knowledge Distillation +2

PQA: Perceptual Question Answering

1 code implementation CVPR 2021 Yonggang Qi, Kai Zhang, Aneeshan Sain, Yi-Zhe Song

Perceptual organization remains one of the very few established theories on the human visual system.

Question Answering

MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition

no code implementations CVPR 2021 Ayan Kumar Bhunia, Shuvozit Ghose, Amandeep Kumar, Pinaki Nath Chowdhury, Aneeshan Sain, Yi-Zhe Song

In this paper, we take a completely different perspective -- we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation.

Meta-Learning

StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval

no code implementations CVPR 2021 Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song

With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.

Meta-Learning Sketch-Based Image Retrieval

Cloud2Curve: Generation and Vectorization of Parametric Sketches

no code implementations CVPR 2021 Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song

Analysis of human sketches in deep learning has advanced immensely through the use of waypoint-sequences rather than raster-graphic representations.

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

1 code implementation CVPR 2021 Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Yongxin Yang, Tao Xiang, Yi-Zhe Song

A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.

Cross-Modal Retrieval Semi-Supervised Sketch Based Image Retrieval +1

Context-Aware Layout to Image Generation with Enhanced Object Appearance

1 code implementation CVPR 2021 Sen He, Wentong Liao, Michael Ying Yang, Yongxin Yang, Yi-Zhe Song, Bodo Rosenhahn, Tao Xiang

We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.

Layout-to-Image Generation

SketchAA: Abstract Representation for Abstract Sketches

no code implementations ICCV 2021 Lan Yang, Kaiyue Pang, Honggang Zhang, Yi-Zhe Song

The superiority of explicitly abstracting sketch representation is empirically validated on a number of sketch analysis tasks, including sketch recognition, fine-grained sketch-based image retrieval, and generative sketch healing.

Sketch-Based Image Retrieval Sketch Recognition

Your "Flamingo" is My "Bird": Fine-Grained, or Not

1 code implementation CVPR 2021 Dongliang Chang, Kaiyue Pang, Yixiao Zheng, Zhanyu Ma, Yi-Zhe Song, Jun Guo

For that, we re-envisage the traditional setting of FGVC, from single-label classification, to that of top-down traversal of a pre-defined coarse-to-fine label hierarchy -- so that our answer becomes "bird"-->"Phoenicopteriformes"-->"Phoenicopteridae"-->"flamingo".

Fine-Grained Image Classification General Classification

Deep Sketch-Based Modeling: Tips and Tricks

no code implementations12 Nov 2020 Yue Zhong, Yulia Gryaditskaya, Honggang Zhang, Yi-Zhe Song

Deep image-based modeling received lots of attention in recent years, yet the parallel problem of sketch-based modeling has only been briefly studied, often as a potential application.

Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval

1 code implementation29 Jul 2020 Aneeshan Sain, Ayan Kumar Bhunia, Yongxin Yang, Tao Xiang, Yi-Zhe Song

In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.

Hierarchical structure Sketch-Based Image Retrieval

On Learning Semantic Representations for Million-Scale Free-Hand Sketches

1 code implementation7 Jul 2020 Peng Xu, Yongye Huang, Tongtong Yuan, Tao Xiang, Timothy M. Hospedales, Yi-Zhe Song, Liang Wang

Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches.

Learning Semantic Representations Zero-Shot Learning

BézierSketch: A generative model for scalable vector sketches

1 code implementation ECCV 2020 Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song

The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.

Image Generation

Sequential Learning for Domain Generalization

no code implementations3 Apr 2020 Da Li, Yongxin Yang, Yi-Zhe Song, Timothy Hospedales

In DG this means encountering a sequence of domains and at each step training to maximise performance on the next domain.

Domain Generalization Meta-Learning

Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation

2 code implementations8 Mar 2020 Dongliang Chang, Aneeshan Sain, Zhanyu Ma, Yi-Zhe Song, Jun Guo

The key insight lies with how we exploit the mutually beneficial information between two networks; (a) to separate samples of known and unknown classes, (b) to maximize the domain confusion between source and target domain without the influence of unknown samples.

Unsupervised Domain Adaptation

Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval

1 code implementation24 Feb 2020 Ayan Kumar Bhunia, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, Yi-Zhe Song

Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.

Cross-Modal Retrieval On-the-Fly Sketch Based Image Retrieval

Deep Self-Supervised Representation Learning for Free-Hand Sketch

1 code implementation3 Feb 2020 Peng Xu, Zeyu Song, Qiyue Yin, Yi-Zhe Song, Liang Wang

In this paper, we tackle for the first time, the problem of self-supervised representation learning for free-hand sketches.

Self-Supervised Learning Unsupervised Representation Learning

SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence

no code implementations16 Jan 2020 Deng Yu, Lei LI, Youyi Zheng, Manfred Lau, Yi-Zhe Song, Chiew-Lan Tai, Hongbo Fu

In this paper, we study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object and predict as output the semantic correspondence among the sketches.

Semantic correspondence

Deep Learning for Free-Hand Sketch: A Survey and A Toolbox

2 code implementations8 Jan 2020 Peng Xu, Timothy M. Hospedales, Qiyue Yin, Yi-Zhe Song, Tao Xiang, Liang Wang

Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present.

Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval

no code implementations10 Nov 2019 Jianjun Lei, Yuxin Song, Bo Peng, Zhanyu Ma, Ling Shao, Yi-Zhe Song

How to align abstract sketches and natural images into a common high-level semantic space remains a key problem in SBIR.

Sketch-Based Image Retrieval

Goal-Driven Sequential Data Abstraction

no code implementations ICCV 2019 Umar Riaz Muhammad, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, Yi-Zhe Song

In the former one asks whether a machine can `understand' enough about the meaning of input data to produce a meaningful but more compact abstraction.

General Reinforcement Learning

Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval

1 code implementation CVPR 2019 Sounak Dey, Pau Riba, Anjan Dutta, Josep Llados, Yi-Zhe Song

Highly abstract amateur human sketches are purposefully sourced to maximize the domain gap, instead of ones included in existing datasets that can often be semi-photorealistic.

Sketch-Based Image Retrieval

Episodic Training for Domain Generalization

2 code implementations ICCV 2019 Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, Timothy M. Hospedales

In this paper, we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime.

Domain Generalization

Deep Factorised Inverse-Sketching

no code implementations ECCV 2018 Kaiyue Pang, Da Li, Jifei Song, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales

Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included.

Sketch-Based Image Retrieval Style Transfer

SketchyScene: Richly-Annotated Scene Sketches

1 code implementation ECCV 2018 Changqing Zou, Qian Yu, Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengying Gao, Baoquan Chen, Hao Zhang

We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.

Colorization Image Retrieval +1

Learning to Sketch with Shortcut Cycle Consistency

no code implementations CVPR 2018 Jifei Song, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process.

Multi-Task Learning Sketch-Based Image Retrieval

Learning Deep Sketch Abstraction

no code implementations CVPR 2018 Umar Riaz Muhammad, Yongxin Yang, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales

Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR).

Sketch-Based Image Retrieval Sketch Recognition

SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval

1 code implementation CVPR 2018 Peng Xu, Yongye Huang, Tongtong Yuan, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, Zhanyu Ma, Jun Guo

Key to our network design is the embedding of unique characteristics of human sketch, where (i) a two-branch CNN-RNN architecture is adapted to explore the temporal ordering of strokes, and (ii) a novel hashing loss is specifically designed to accommodate both the temporal and abstract traits of sketches.

Sketch Recognition

Deeper, Broader and Artier Domain Generalization

2 code implementations ICCV 2017 Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales

In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning.

Domain Generalization

Sketch Me That Shoe

no code implementations CVPR 2016 Qian Yu, Feng Liu, Yi-Zhe Song, Tao Xiang, Timothy M. Hospedales, Chen-Change Loy

We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images.

Data Augmentation Sketch-Based Image Retrieval

Free-hand Sketch Synthesis with Deformable Stroke Models

no code implementations9 Oct 2015 Yi Li, Yi-Zhe Song, Timothy Hospedales, Shaogang Gong

We present a generative model which can automatically summarize the stroke composition of free-hand sketches of a given category.

Making Better Use of Edges via Perceptual Grouping

no code implementations CVPR 2015 Yonggang Qi, Yi-Zhe Song, Tao Xiang, Honggang Zhang, Timothy Hospedales, Yi Li, Jun Guo

We propose a perceptual grouping framework that organizes image edges into meaningful structures and demonstrate its usefulness on various computer vision tasks.

Learning-To-Rank Sketch-Based Image Retrieval

Sketch-a-Net that Beats Humans

1 code implementation30 Jan 2015 Qian Yu, Yongxin Yang, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans.

Sketch Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.