Search Results for author:
Found 72 papers, 34 papers with code
A Tree-Structured Decoder for Image-to-Markup Generation
Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup.
Adaptive Fine-Grained Sketch-Based Image Retrieval
To solve this new problem, we introduce a novel model-agnostic meta-learning (MAML) based framework with several key modifications: (1) As a retrieval task with a margin-based contrastive loss, we simplify the MAML training in the inner loop to make it more stable and tractable.
SceneTrilogy: On Scene Sketches and its Relationship with Text and Photo
We spell out a few insights on the complementarity of each modality for scene understanding, and study for the first time a series of scene-specific applications like joint sketch- and text-based image retrieval, sketch captioning.
UIGR: Unified Interactive Garment Retrieval
In this paper, we propose a Unified Interactive Garment Retrieval (UIGR) framework to unify TGR and VCR.
Style-Based Global Appearance Flow for Virtual Try-On
To achieve this, a key step is garment warping which spatially aligns the target garment with the corresponding body parts in the person image.
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches
In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints).
Sketch3T: Test-Time Training for Zero-Shot SBIR
In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i. e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result.
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial".
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval
We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch".
Dynamic Instance Domain Adaptation
In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain.
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context
In addition, we propose new solutions enabled by our dataset (i) We adopt meta-learning to show how the retrieval model can be fine-tuned to a new user style given just a small set of sketches, (ii) We extend a popular vector sketch LSTM-based encoder to handle sketches with larger complexity than was supported by previous work.
Finding Badly Drawn Bunnies
Our key discovery lies in exploiting the magnitude (L2 norm) of a sketch feature as a quantitative quality metric.
One Sketch for All: One-Shot Personalized Sketch Segmentation
We aim to segment all sketches belonging to the same category provisioned with a single sketch with a given part annotation while (i) preserving the parts semantics embedded in the exemplar, and (ii) being robust to input style and abstraction.
Hybrid Graph Neural Networks for Few-Shot Learning
This is because they use an instance GNN as a label propagation/classification module, which is jointly meta-learned with a feature embedding network.
Clue Me In: Semi-Supervised FGVC with Out-of-Distribution Data
Despite great strides made on fine-grained visual classification (FGVC), current methods are still heavily reliant on fully-supervised paradigms where ample expert labels are called for.
Fine-Grained Image Analysis with Deep Learning: A Survey
Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications.
One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective
In this work, we propose a novel deep hashing model with only a single learning objective.
SketchODE: Learning neural sketch representation in continuous time
Learning meaningful representations for chirographic drawing data such as sketches, handwriting, and flowcharts is a gateway for understanding and emulating human creative expression.
Temporal Action Localization with Global Segmentation Mask Transformers
In this paper, to address the above two challenges, a novel {\em Global Segmentation Mask Transformer} (GSMT) is proposed.
SketchLattice: Latticed Representation for Sketch Manipulation
The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches.
Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels).
Disentangled Lifespan Face Synthesis
The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving.
Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation
In this paper, for the first time, we argue for their unification -- we aim for a single model that can compete favourably with two separate state-of-the-art STR and HTR models.
Towards the Unseen: Iterative Text Recognition by Distilling from Errors
Our framework is iterative in nature, in that it utilises predicted knowledge of character sequences from a previous iteration, to augment the main network in improving the next prediction.
Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
In this paper, we argue that semantic information offers a complementary role in addition to visual only.
Towards Unsupervised Sketch-based Image Retrieval
Current supervised sketch-based image retrieval (SBIR) methods achieve excellent performance.
PQA: Perceptual Question Answering
Perceptual organization remains one of the very few established theories on the human visual system.
MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition
In this paper, we take a completely different perspective -- we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation.
Cloud2Curve: Generation and Vectorization of Parametric Sketches
Analysis of human sketches in deep learning has advanced immensely through the use of waypoint-sequences rather than raster-graphic representations.
StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.
Cross-Modal Retrieval
Semi-Supervised Sketch Based Image Retrieval
+1
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
This data is uniquely characterised by its existence in dual modalities of rasterized images and vector coordinate sequences.
Context-Aware Layout to Image Generation with Enhanced Object Appearance
We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.
Ranked #1 on
Layout-to-Image Generation
on COCO-Stuff 128x128
SketchAA: Abstract Representation for Abstract Sketches
The superiority of explicitly abstracting sketch representation is empirically validated on a number of sketch analysis tasks, including sketch recognition, fine-grained sketch-based image retrieval, and generative sketch healing.
Your "Flamingo" is My "Bird": Fine-Grained, or Not
For that, we re-envisage the traditional setting of FGVC, from single-label classification, to that of top-down traversal of a pre-defined coarse-to-fine label hierarchy -- so that our answer becomes "bird"-->"Phoenicopteriformes"-->"Phoenicopteridae"-->"flamingo".
Ranked #9 on
Fine-Grained Image Classification
on FGVC Aircraft
Deep Sketch-Based Modeling: Tips and Tricks
Deep image-based modeling received lots of attention in recent years, yet the parallel problem of sketch-based modeling has only been briefly studied, often as a potential application.
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.
On Learning Semantic Representations for Million-Scale Free-Hand Sketches
Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches.
BézierSketch: A generative model for scalable vector sketches
The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.
Sequential Learning for Domain Generalization
In DG this means encountering a sequence of domains and at each step training to maximise performance on the next domain.
Ranked #52 on
Domain Generalization
on PACS
Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation
The key insight lies with how we exploit the mutually beneficial information between two networks; (a) to separate samples of known and unknown classes, (b) to maximize the domain confusion between source and target domain without the influence of unknown samples.
Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches
In this work, we propose a novel framework for fine-grained visual classification to tackle these problems.
Ranked #11 on
Fine-Grained Image Classification
on Stanford Cars
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.
Cross-Modal Retrieval
On-the-Fly Sketch Based Image Retrieval
Fine-Grained Instance-Level Sketch-Based Video Retrieval
Existing sketch-analysis work studies sketches depicting static objects or scenes.
The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component.
Ranked #20 on
Fine-Grained Image Classification
on FGVC Aircraft
Deep Self-Supervised Representation Learning for Free-Hand Sketch
In this paper, we tackle for the first time, the problem of self-supervised representation learning for free-hand sketches.
SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
In this paper, we study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object and predict as output the semantic correspondence among the sketches.
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present.
Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval
How to align abstract sketches and natural images into a common high-level semantic space remains a key problem in SBIR.
Goal-Driven Sequential Data Abstraction
In the former one asks whether a machine can `understand' enough about the meaning of input data to produce a meaningful but more compact abstraction.
Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
Highly abstract amateur human sketches are purposefully sourced to maximize the domain gap, instead of ones included in existing datasets that can often be semi-photorealistic.
Episodic Training for Domain Generalization
In this paper, we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime.
Ranked #52 on
Domain Generalization
on PACS
Deep Factorised Inverse-Sketching
Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included.
SketchyScene: Richly-Annotated Scene Sketches
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.
Learning to Sketch with Shortcut Cycle Consistency
In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process.
Sketch-a-Classifier: Sketch-based Photo Classifier Generation
Contemporary deep learning techniques have made image recognition a reasonably reliable technology.
Learning Deep Sketch Abstraction
Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR).
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval
Key to our network design is the embedding of unique characteristics of human sketch, where (i) a two-branch CNN-RNN architecture is adapted to explore the temporal ordering of strokes, and (ii) a novel hashing loss is specifically designed to accommodate both the temporal and abstract traits of sketches.
The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching
Many vision problems require matching images of object instances across different domains.
Learning to Generalize: Meta-Learning for Domain Generalization
We propose a novel {meta-learning} method for domain generalization.
Ranked #88 on
Domain Generalization
on PACS
Deeper, Broader and Artier Domain Generalization
In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning.
Ranked #91 on
Domain Generalization
on PACS
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval
Human sketches are unique in being able to capture both the spatial topology of a visual object, as well as its subtle appearance details.
Ranked #2 on
Sketch-Based Image Retrieval
on Chairs
Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
Sketch-based image retrieval (SBIR) is challenging due to the inherent domain-gap between sketch and photo.
Ranked #5 on
Sketch-Based Image Retrieval
on Chairs
Sketch Me That Shoe
We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images.
Ranked #3 on
Sketch-Based Image Retrieval
on Chairs
ForgetMeNot: Memory-Aware Forensic Facial Sketch Matching
Based on this database we build a model to reverse the forgetting process.
Free-hand Sketch Synthesis with Deformable Stroke Models
We present a generative model which can automatically summarize the stroke composition of free-hand sketches of a given category.
Making Better Use of Edges via Perceptual Grouping
We propose a perceptual grouping framework that organizes image edges into meaningful structures and demonstrate its usefulness on various computer vision tasks.
Sketch-a-Net that Beats Humans
We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans.
A Survey on Heterogeneous Face Recognition: Sketch, Infra-red, 3D and Low-resolution
Heterogeneous face recognition (HFR) refers to matching face imagery across different domains.
