Search Results for author:
Found 34 papers, 11 papers with code
SketchINR: A First Look into Sketches as Implicit Neural Representations
(ii) SketchINR's auto-decoder provides a much higher-fidelity representation than other learned vector sketch representations, and is uniquely able to scale to complex vector sketches such as FS-COCO.
What Sketch Explainability Really Means for Downstream Tasks
In this paper, we explore the unique modality of sketch for explainability, emphasising the profound impact of human strokes compared to conventional pixel-oriented studies.
You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval
Two primary input modalities prevail in image retrieval: sketch and text.
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models
This paper unravels the potential of sketches for diffusion models, addressing the deceptive promise of direct sketch control in generative AI.
Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers
This paper, for the first time, explores text-to-image diffusion models for Zero-Shot Sketch-based Image Retrieval (ZS-SBIR).
How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?
@q loss to inject that understanding into the system.
DemoCaricature: Democratising Caricature Generation with a Rough Sketch
In this paper, we democratise caricature generation, empowering individuals to effortlessly craft personalised caricatures with just a photo and a conceptual sketch.
Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes
In this paper, we democratise 3D content creation, enabling precise generation of 3D shapes from abstract sketches while overcoming limitations tied to drawing skills.
What Can Human Sketches Do for Object Detection?
In particular, we first perform independent prompting on both sketch and photo branches of an SBIR model to build highly generalisable sketch and photo encoders on the back of the generalisation ability of CLIP.
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR
This paper advances the fine-grained sketch-based image retrieval (FG-SBIR) literature by putting forward a strong baseline that overshoots prior state-of-the-arts by ~11%.
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not
At the very core of our solution is a prompt learning setup.
Picture that Sketch: Photorealistic Image Generation from Abstract Sketches
We further introduce specific designs to tackle the abstract nature of human sketches, including a fine-grained discriminative loss on the back of a trained sketch-photo retrieval model, and a partial-aware sketch augmentation strategy.
Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings
Human sketch has already proved its worth in various visual understanding tasks (e. g., retrieval, segmentation, image-captioning, etc).
Democratising 2D Sketch to 3D Shape Retrieval Through Pivoting
We perform pivoting on two existing datasets, each from a distant research domain to the other: 2D sketch and photo pairs from the sketch-based image retrieval field (SBIR), and 3D shapes from ShapeNet.
Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classification
Conventional few-shot learning methods however cannot be naively adopted for this fine-grained setting -- a quick pilot study reveals that they in fact push for the opposite (i. e., lower inter-class variations and higher intra-class variations).
Adaptive Fine-Grained Sketch-Based Image Retrieval
To solve this new problem, we introduce a novel model-agnostic meta-learning (MAML) based framework with several key modifications: (1) As a retrieval task with a margin-based contrastive loss, we simplify the MAML training in the inner loop to make it more stable and tractable.
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text
In this paper, we extend scene understanding to include that of human sketch.
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial".
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches
In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints).
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval
We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch".
Sketch3T: Test-Time Training for Zero-Shot SBIR
In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i. e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result.
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context
We advance sketch research to scenes with the first dataset of freehand scene sketches, FS-COCO.
Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
In this paper, we argue that semantic information offers a complementary role in addition to visual only.
Towards the Unseen: Iterative Text Recognition by Distilling from Errors
Our framework is iterative in nature, in that it utilises predicted knowledge of character sequences from a previous iteration, to augment the main network in improving the next prediction.
Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation
In this paper, for the first time, we argue for their unification -- we aim for a single model that can compete favourably with two separate state-of-the-art STR and HTR models.
PQA: Perceptual Question Answering
Perceptual organization remains one of the very few established theories on the human visual system.
MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition
In this paper, we take a completely different perspective -- we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation.
StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.
Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation
The key insight lies with how we exploit the mutually beneficial information between two networks; (a) to separate samples of known and unknown classes, (b) to maximize the domain confusion between source and target domain without the influence of unknown samples.
Improving Document Binarization via Adversarial Noise-Texture Augmentation
By jointly training the two networks we can increase the adversarial robustness of our system.
Indic Handwritten Script Identification using Offline-Online Multimodal Deep Network
In this paper, we propose a novel approach of word-level Indic script identification using only character-level data in training stage.
Multi-Oriented Text Detection and Verification in Video Frames and Scene Images
Until now only a few methods have been proposed that look into curved text detection in video frames, wherein lies our novelty.
