Search Results for author:
Found 121 papers, 62 papers with code
A Survey on Heterogeneous Face Recognition: Sketch, Infra-red, 3D and Low-resolution
Heterogeneous face recognition (HFR) refers to matching face imagery across different domains.
Sketch-a-Net that Beats Humans
We propose a multi-scale multi-channel deep neural network framework that, for the first time, yields sketch recognition performance surpassing that of humans.
Making Better Use of Edges via Perceptual Grouping
We propose a perceptual grouping framework that organizes image edges into meaningful structures and demonstrate its usefulness on various computer vision tasks.
Free-hand Sketch Synthesis with Deformable Stroke Models
We present a generative model which can automatically summarize the stroke composition of free-hand sketches of a given category.
Sketch Me That Shoe
We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images.
Ranked #3 on
Sketch-Based Image Retrieval
on Chairs
ForgetMeNot: Memory-Aware Forensic Facial Sketch Matching
Based on this database we build a model to reverse the forgetting process.
Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
Sketch-based image retrieval (SBIR) is challenging due to the inherent domain-gap between sketch and photo.
Ranked #5 on
Sketch-Based Image Retrieval
on Chairs
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval
Human sketches are unique in being able to capture both the spatial topology of a visual object, as well as its subtle appearance details.
Ranked #2 on
Sketch-Based Image Retrieval
on Handbags
Deeper, Broader and Artier Domain Generalization
In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning.
Ranked #117 on
Domain Generalization
on PACS
Learning to Generalize: Meta-Learning for Domain Generalization
We propose a novel {meta-learning} method for domain generalization.
Ranked #114 on
Domain Generalization
on PACS
The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching
Many vision problems require matching images of object instances across different domains.
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval
Key to our network design is the embedding of unique characteristics of human sketch, where (i) a two-branch CNN-RNN architecture is adapted to explore the temporal ordering of strokes, and (ii) a novel hashing loss is specifically designed to accommodate both the temporal and abstract traits of sketches.
Learning Deep Sketch Abstraction
Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR).
Sketch-a-Classifier: Sketch-based Photo Classifier Generation
Contemporary deep learning techniques have made image recognition a reasonably reliable technology.
Learning to Sketch with Shortcut Cycle Consistency
In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process.
SketchyScene: Richly-Annotated Scene Sketches
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.
Deep Factorised Inverse-Sketching
Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included.
Episodic Training for Domain Generalization
In this paper, we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime.
Ranked #76 on
Domain Generalization
on PACS
Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
Highly abstract amateur human sketches are purposefully sourced to maximize the domain gap, instead of ones included in existing datasets that can often be semi-photorealistic.
Goal-Driven Sequential Data Abstraction
In the former one asks whether a machine can `understand' enough about the meaning of input data to produce a meaningful but more compact abstraction.
Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval
How to align abstract sketches and natural images into a common high-level semantic space remains a key problem in SBIR.
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present.
SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
In this paper, we study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object and predict as output the semantic correspondence among the sketches.
Deep Self-Supervised Representation Learning for Free-Hand Sketch
In this paper, we tackle for the first time, the problem of self-supervised representation learning for free-hand sketches.
The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component.
Ranked #29 on
Fine-Grained Image Classification
on FGVC Aircraft
Fine-Grained Instance-Level Sketch-Based Video Retrieval
Existing sketch-analysis work studies sketches depicting static objects or scenes.
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.
Cross-Modal Retrieval
On-the-Fly Sketch Based Image Retrieval
+1
Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation
The key insight lies with how we exploit the mutually beneficial information between two networks; (a) to separate samples of known and unknown classes, (b) to maximize the domain confusion between source and target domain without the influence of unknown samples.
Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches
In this work, we propose a novel framework for fine-grained visual classification to tackle these problems.
Ranked #17 on
Fine-Grained Image Classification
on Stanford Cars
Sequential Learning for Domain Generalization
In DG this means encountering a sequence of domains and at each step training to maximise performance on the next domain.
Ranked #76 on
Domain Generalization
on PACS
BézierSketch: A generative model for scalable vector sketches
The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.
On Learning Semantic Representations for Million-Scale Free-Hand Sketches
Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches.
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.
Deep Sketch-Based Modeling: Tips and Tricks
Deep image-based modeling received lots of attention in recent years, yet the parallel problem of sketch-based modeling has only been briefly studied, often as a potential application.
Your "Flamingo" is My "Bird": Fine-Grained, or Not
For that, we re-envisage the traditional setting of FGVC, from single-label classification, to that of top-down traversal of a pre-defined coarse-to-fine label hierarchy -- so that our answer becomes "bird"-->"Phoenicopteriformes"-->"Phoenicopteridae"-->"flamingo".
Ranked #16 on
Fine-Grained Image Classification
on FGVC Aircraft
SketchAA: Abstract Representation for Abstract Sketches
The superiority of explicitly abstracting sketch representation is empirically validated on a number of sketch analysis tasks, including sketch recognition, fine-grained sketch-based image retrieval, and generative sketch healing.
Context-Aware Layout to Image Generation with Enhanced Object Appearance
We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.
Ranked #1 on
Layout-to-Image Generation
on COCO-Stuff 128x128
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
This data is uniquely characterised by its existence in dual modalities of rasterized images and vector coordinate sequences.
Cloud2Curve: Generation and Vectorization of Parametric Sketches
Analysis of human sketches in deep learning has advanced immensely through the use of waypoint-sequences rather than raster-graphic representations.
StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.
MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition
In this paper, we take a completely different perspective -- we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation.
PQA: Perceptual Question Answering
Perceptual organization remains one of the very few established theories on the human visual system.
Towards Unsupervised Sketch-based Image Retrieval
The practical value of existing supervised sketch-based image retrieval (SBIR) algorithms is largely limited by the requirement for intensive data collection and labeling.
Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation
In this paper, for the first time, we argue for their unification -- we aim for a single model that can compete favourably with two separate state-of-the-art STR and HTR models.
Towards the Unseen: Iterative Text Recognition by Distilling from Errors
Our framework is iterative in nature, in that it utilises predicted knowledge of character sequences from a previous iteration, to augment the main network in improving the next prediction.
Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
In this paper, we argue that semantic information offers a complementary role in addition to visual only.
Disentangled Lifespan Face Synthesis
The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving.
Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels).
SketchLattice: Latticed Representation for Sketch Manipulation
The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches.
One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective
In this work, we propose a novel deep hashing model with only a single learning objective.
SketchODE: Learning neural sketch representation in continuous time
Learning meaningful representations for chirographic drawing data such as sketches, handwriting, and flowcharts is a gateway for understanding and emulating human creative expression.
Temporal Action Localization with Global Segmentation Mask Transformers
In this paper, to address the above two challenges, a novel {\em Global Segmentation Mask Transformer} (GSMT) is proposed.
Fine-Grained Image Analysis with Deep Learning: A Survey
Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications.
Clue Me In: Semi-Supervised FGVC with Out-of-Distribution Data
Despite great strides made on fine-grained visual classification (FGVC), current methods are still heavily reliant on fully-supervised paradigms where ample expert labels are called for.
Hybrid Graph Neural Networks for Few-Shot Learning
This is because they use an instance GNN as a label propagation/classification module, which is jointly meta-learned with a feature embedding network.
One Sketch for All: One-Shot Personalized Sketch Segmentation
We aim to segment all sketches belonging to the same category provisioned with a single sketch with a given part annotation while (i) preserving the parts semantics embedded in the exemplar, and (ii) being robust to input style and abstraction.
Finding Badly Drawn Bunnies
Our key discovery lies in exploiting the magnitude (L2 norm) of a sketch feature as a quantitative quality metric.
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context
We advance sketch research to scenes with the first dataset of freehand scene sketches, FS-COCO.
Dynamic Instance Domain Adaptation
In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain.
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches
In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints).
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval
We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch".
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial".
Sketch3T: Test-Time Training for Zero-Shot SBIR
In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i. e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result.
Style-Based Global Appearance Flow for Virtual Try-On
To achieve this, a key step is garment warping which spatially aligns the target garment with the corresponding body parts in the person image.
Ranked #1 on
Virtual Try-on
on VITON
UIGR: Unified Interactive Garment Retrieval
In this paper, we propose a Unified Interactive Garment Retrieval (UIGR) framework to unify TGR and VCR.
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text
In this paper, we extend scene understanding to include that of human sketch.
Adaptive Fine-Grained Sketch-Based Image Retrieval
To solve this new problem, we introduce a novel model-agnostic meta-learning (MAML) based framework with several key modifications: (1) As a retrieval task with a margin-based contrastive loss, we simplify the MAML training in the inner loop to make it more stable and tractable.
Semi-Supervised Temporal Action Detection with Proposal-Free Masking
Such a novel design effectively eliminates the dependence between localization and classification by cutting off the route for error propagation in-between.
Ranked #1 on
Semi-Supervised Action Detection
on ActivityNet-1.3
Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning
Existing temporal action detection (TAD) methods rely on generating an overwhelmingly large number of proposals per video.
Ranked #15 on
Temporal Action Localization
on ActivityNet-1.3
FashionViL: Fashion-Focused Vision-and-Language Representation Learning
We thus propose a Multi-View Contrastive Learning task for pulling closer the visual representation of one image to the compositional multimodal representation of another image+text.
Zero-Shot Temporal Action Detection via Vision-Language Prompting
Such a novel design effectively eliminates the dependence between localization and classification by breaking the route for error propagation in-between.
Ranked #1 on
Zero-Shot Action Detection
on THUMOS' 14
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling
Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape.
DifferSketching: How Differently Do People Sketch 3D Objects?
We invited 70 novice users and 38 expert users to sketch 136 3D objects, which were presented as 362 images rendered from multiple views.
Structure-Aware 3D VR Sketch to 3D Shape Retrieval
In particular, we propose to use a triplet loss with an adaptive margin value driven by a "fitting gap", which is the similarity of two shapes under structure-preserving deformations.
Towards 3D VR-Sketch to 3D Shape Retrieval
In this paper, we offer a different perspective towards answering these questions -- we study the use of 3D sketches as an input modality and advocate a VR-scenario where retrieval is conducted.
Fine-Grained VR Sketching: Dataset and Insights
We then, for the first time, study the scenario of fine-grained 3D VR sketch to 3D shape retrieval, as a novel VR sketching application and a proving ground to drive out generic insights to inform future research.
Robust Target Training for Multi-Source Domain Adaptation
Given any existing fully-trained one-step MSDA model, BORT$^2$ turns it to a labeling function to generate pseudo-labels for the target data and trains a target model using pseudo-labeled target data only.
Prediction Calibration for Generalized Few-shot Semantic Segmentation
To ensure that the fused scores are not biased to either the base or novel classes, a new Transformer-based calibration module is introduced.
Generalized Few-Shot Semantic Segmentation
Semantic Segmentation
Learning to Augment via Implicit Differentiation for Domain Generalization
Machine learning models are intrinsically vulnerable to domain shift between training and testing data, resulting in poor performance in novel domains.
Single Stage Multi-Pose Virtual Try-On
Key to our model is a parallel flow estimation module that predicts the flow fields for both person and garment images conditioned on the target pose.
Post-Processing Temporal Action Detection
To address this problem, in this work we introduce a novel model-agnostic post-processing method without model redesign and retraining.
Multi-Modal Few-Shot Temporal Action Detection
In this work, we introduce a new multi-modality few-shot (MMFS) TAD problem, which can be considered as a marriage of FS-TAD and ZS-TAD by leveraging few-shot support videos and new class names jointly.
Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classification
Conventional few-shot learning methods however cannot be naively adopted for this fine-grained setting -- a quick pilot study reveals that they in fact push for the opposite (i. e., lower inter-class variations and higher intra-class variations).
An Erudite Fine-Grained Visual Classification Model
Therefore, we first propose a feature disentanglement module and a feature re-fusion module to reduce negative transfer and boost positive transfer between different datasets.
Controllable Person Image Synthesis with Pose-Constrained Latent Diffusion
Controllable person image synthesis aims at rendering a source image based on user-specified changes in body pose or appearance.
Photo Pre-Training, but for Sketch
This lack of sketch data has imposed on the community a few "peculiar" design choices -- the most representative of them all is perhaps the coerced utilisation of photo-based pre-training (i. e., no sketch), for many core tasks that otherwise dictates specific sketch understanding.
Democratising 2D Sketch to 3D Shape Retrieval Through Pivoting
We perform pivoting on two existing datasets, each from a distant research domain to the other: 2D sketch and photo pairs from the sketch-based image retrieval field (SBIR), and 3D shapes from ShapeNet.
On-the-Fly Category Discovery
Our code is available at https://github. com/PRIS-CV/On-the-fly-Category-Discovery.
Task-aware Adaptive Learning for Cross-domain Few-shot Learning
In this paper, we first observe the dependence of task-specific parameter configuration on the target task.
Unsupervised Hashing with Similarity Distribution Calibration
However, these methods often overlook the fact that the similarity between data points in the continuous feature space may not be preserved in the discrete hash code space, due to the limited similarity range of hash codes.
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
In the fashion domain, there exists a variety of vision-and-language (V+L) tasks, including cross-modal retrieval, text-guided image retrieval, multi-modal classification, and image captioning.
Generative Model Based Noise Robust Training for Unsupervised Domain Adaptation
D-CFA minimizes the domain gap by augmenting the source data with distribution-sampled target features, and trains a noise-robust discriminative classifier by using target domain knowledge from the generative models.
Data-Free Sketch-Based Image Retrieval
For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm.
Picture that Sketch: Photorealistic Image Generation from Abstract Sketches
We further introduce specific designs to tackle the abstract nature of human sketches, including a fine-grained discriminative loss on the back of a trained sketch-photo retrieval model, and a partial-aware sketch augmentation strategy.
Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings
Human sketch has already proved its worth in various visual understanding tasks (e. g., retrieval, segmentation, image-captioning, etc).
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not
At the very core of our solution is a prompt learning setup.
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR
This paper advances the fine-grained sketch-based image retrieval (FG-SBIR) literature by putting forward a strong baseline that overshoots prior state-of-the-arts by ~11%.
Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style
This paper studies the problem of zero-short sketch-based image retrieval (ZS-SBIR), however with two significant differentiators to prior art (i) we tackle all variants (inter-category, intra-category, and cross datasets) of ZS-SBIR with just one network (``everything''), and (ii) we would really like to understand how this sketch-photo matching operates (``explainable'').
What Can Human Sketches Do for Object Detection?
In particular, we first perform independent prompting on both sketch and photo branches of an SBIR model to build highly generalisable sketch and photo encoders on the back of the generalisation ability of CLIP.
DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion
Concretely, we establish the denoising process in the Transformer decoder (e. g., DETR) by introducing a temporal location query design with faster convergence in training.
ChiroDiff: Modelling chirographic data with Diffusion Models
Such strictly-ordered discrete factorization however falls short of capturing key properties of chirographic data -- it fails to build holistic understanding of the temporal concept due to one-way visibility (causality).
SketchXAI: A First Look at Explainability for Human Sketches
Following this, we design a simple explainability-friendly sketch encoder that accommodates the intrinsic properties of strokes: shape, location, and order.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
3D VR Sketch Guided 3D Shape Prototyping and Exploration
3D shape modeling is labor-intensive, time-consuming, and requires years of expertise.
SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation
Through this work, we hope to aspire the way we create visual content, democratise the creative process, and inspire further research in enhancing human creativity in AIGC.
Sketch-based Video Object Segmentation: Benchmark and Analysis
Reference-based video object segmentation is an emerging topic which aims to segment the corresponding target object in each video frame referred by a given reference, such as a language expression or a photo mask.
DemoFusion: Democratising High-Resolution Image Generation With No $$$
High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls.
Wired Perspectives: Multi-View Wire Art Embraces Generative AI
Creating multi-view wire art (MVWA), a static 3D sculpture with diverse interpretations from different viewpoints, is a complex task even for skilled artists.
DreamCreature: Crafting Photorealistic Virtual Creatures from Imagination
To bridge this gap, we introduce a novel task, Virtual Creatures Generation: Given a set of unlabeled images of the target concepts (e. g., 200 bird species), we aim to train a T2I model capable of creating new, hybrid concepts within diverse backgrounds and contexts.
Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes
In this paper, we democratise 3D content creation, enabling precise generation of 3D shapes from abstract sketches while overcoming limitations tied to drawing skills.
DemoCaricature: Democratising Caricature Generation with a Rough Sketch
In this paper, we democratise caricature generation, empowering individuals to effortlessly craft personalised caricatures with just a photo and a conceptual sketch.
How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?
@q loss to inject that understanding into the system.
Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers
This paper, for the first time, explores text-to-image diffusion models for Zero-Shot Sketch-based Image Retrieval (ZS-SBIR).
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models
This paper unravels the potential of sketches for diffusion models, addressing the deceptive promise of direct sketch control in generative AI.
You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval
Two primary input modalities prevail in image retrieval: sketch and text.
What Sketch Explainability Really Means for Downstream Tasks
In this paper, we explore the unique modality of sketch for explainability, emphasising the profound impact of human strokes compared to conventional pixel-oriented studies.
SketchINR: A First Look into Sketches as Implicit Neural Representations
(ii) SketchINR's auto-decoder provides a much higher-fidelity representation than other learned vector sketch representations, and is uniquely able to scale to complex vector sketches such as FS-COCO.
A Tree-Structured Decoder for Image-to-Markup Generation
Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup.
