Search Results for author:
Found 212 papers, 95 papers with code
MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism
Based on analyzing the character of cascaded decoder architecture commonly adopted in existing DETR-like models, this paper proposes a new decoder architecture.
Ranked #2 on
Object Detection
on COCO 2017 val
Semantic Gaussian Mixture Variational Autoencoder for Sequential Recommendation
SIGMA assumes that the prior of sequence representation conforms to a Gaussian mixture distribution, where each component of the distribution semantically corresponds to one of multiple interests.
Freestyle Sketch-in-the-Loop Image Segmentation
In this paper, we expand the domain of sketch research into the field of image segmentation, aiming to establish freehand sketches as a query modality for subjective image segmentation.
Chirpy3D: Continuous Part Latents for Creative 3D Bird Generation
In this paper, we push the boundaries of fine-grained 3D generation into truly creative territory.
Preventing Non-intrusive Load Monitoring Privacy Invasion: A Precise Adversarial Attack Scheme for Networked Smart Meters
The scheme effectively prevents NILM models from violating appliance-level privacy, while also ensuring accurate billing calculation for users.
PIGUIQA: A Physical Imaging Guided Perceptual Framework for Underwater Image Quality Assessment
This module effectively captures subtle features in images, thereby enhancing the adaptive perception of distortions on the basis of local information.
Learning Flow Fields in Attention for Controllable Person Image Generation
Additionally, we show that our loss is model-agnostic and can be used to improve the performance of other diffusion models.
Ranked #1 on
Pose Transfer
on Deep-Fashion
(FID metric)
On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance
To our knowledge, this paper proposes the first on-road object importance estimation model that fuses multi-fold top-down guidance factors with bottom-up feature.
MarDini: Masked Autoregressive Diffusion for Video Generation at Scale
We introduce MarDini, a new family of video diffusion models that integrate the advantages of masked auto-regression (MAR) into a unified diffusion model (DM) framework.
DI-MaskDINO: A Joint Object Detection and Instance Segmentation Model
This paper is motivated by an interesting phenomenon: the performance of object detection lags behind that of instance segmentation (i. e., performance imbalance) when investigating the intermediate results from the beginning transformer decoder layer of MaskDINO (i. e., the SOTA model for joint detection and segmentation).
An Unbiased Risk Estimator for Partial Label Learning with Augmented Classes
To address this issue, in this paper, we focus on the problem of Partial Label Learning with Augmented Class (PLLAC), where one or more augmented classes are not visible in the training stage but appear in the inference stage.
AsyCo: An Asymmetric Dual-task Co-training Model for Partial-label Learning
Specifically, the disambiguation network is trained with self-training PLL task to learn label confidence, while the auxiliary network is trained in a supervised learning paradigm to learn from the noisy pairwise similarity labels that are constructed according to the learned label confidence.
Orthogonal Hyper-category Guided Multi-interest Elicitation for Micro-video Matching
Moreover, OPAL employs a two-stage training strategy, in which the pre-train is to generate soft interests from historical interactions under the guidance of orthogonal hyper-categories of micro-videos and the fine-tune is to reinforce the degree of disentanglement among the interests and learn the temporal evolution of each interest of each user.
PartCraft: Crafting Creative Objects by Parts
This paper propels creative control in generative visual AI by allowing users to "select".
Do Generalised Classifiers really work on Human Drawn Sketches?
This generalisation happens on two fronts: (i) generalisation across unknown categories (i. e., open-set), and (ii) generalisation traversing abstraction levels (i. e., good and bad sketches), both being timely challenges that remain unsolved in the sketch literature.
ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery
Code at: https://github. com/kamwoh/concepthash.
EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection
To tackle this issue, we dive into the fundamental mechanism of backdoor attacks on FSSL, proposing the Embedding Inspector (EmInspector) that detects malicious clients by inspecting the embedding space of local models.
Move Anything with Layered Scene Diffusion
Diffusion models generate images with an unprecedented level of quality, but how can we freely rearrange image layouts?
Faster Diffusion via Temporal Attention Decomposition
We explore the role of attention mechanism during inference in text-conditional diffusion models.
SketchINR: A First Look into Sketches as Implicit Neural Representations
(ii) SketchINR's auto-decoder provides a much higher-fidelity representation than other learned vector sketch representations, and is uniquely able to scale to complex vector sketches such as FS-COCO.
What Sketch Explainability Really Means for Downstream Tasks
In this paper, we explore the unique modality of sketch for explainability, emphasising the profound impact of human strokes compared to conventional pixel-oriented studies.
Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers
This paper, for the first time, explores text-to-image diffusion models for Zero-Shot Sketch-based Image Retrieval (ZS-SBIR).
You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval
Two primary input modalities prevail in image retrieval: sketch and text.
How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?
@q loss to inject that understanding into the system.
Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability
In this paper, we propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain.
IPR-NeRF: Ownership Verification meets Neural Radiance Field
Neural Radiance Field (NeRF) models have gained significant attention in the computer vision community in the recent past with state-of-the-art visual quality and produced impressive demonstrations.
What Sketch Explainability Really Means for Downstream Tasks?
In this paper we explore the unique modality of sketch for explainability emphasising the profound impact of human strokes compared to conventional pixel-oriented studies.
Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks
Solving image-to-3D from a single view is an ill-posed problem, and current neural reconstruction methods addressing it through diffusion models still rely on scene-specific optimization, constraining their generalization capability.
GenTron: Diffusion Transformers for Image and Video Generation
In this study, we explore Transformer-based diffusion models for image and video generation.
Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes
In this paper, we democratise 3D content creation, enabling precise generation of 3D shapes from abstract sketches while overcoming limitations tied to drawing skills.
Refine, Discriminate and Align: Stealing Encoders via Sample-Wise Prototypes and Multi-Relational Extraction
This paper introduces RDA, a pioneering approach designed to address two primary deficiencies prevalent in previous endeavors aiming at stealing pre-trained encoders: (1) suboptimal performances attributed to biased optimization objectives, and (2) elevated query costs stemming from the end-to-end paradigm that necessitates querying the target encoder every epoch.
DreamCreature: Crafting Photorealistic Virtual Creatures from Imagination
To bridge this gap, we introduce a novel task, Virtual Creatures Generation: Given a set of unlabeled images of the target concepts (e. g., 200 bird species), we aim to train a T2I model capable of creating new, hybrid concepts within diverse backgrounds and contexts.
Wired Perspectives: Multi-View Wire Art Embraces Generative AI
Creating multi-view wire art (MVWA), a static 3D sculpture with diverse interpretations from different viewpoints, is a complex task even for skilled artists.
FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing
In this paper, for the first time, we introduce optical flow into the attention module in the diffusion model's U-Net to address the inconsistency issue for text-to-video editing.
SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation
Through this work, we hope to aspire the way we create visual content, democratise the creative process, and inspire further research in enhancing human creativity in AIGC.
Mitigating Cross-client GANs-based Attack in Federated Learning
Since a benign client's data can be leaked to the adversary, this attack brings the risk of local data leakage for clients in many security-critical FL applications.
3D VR Sketch Guided 3D Shape Prototyping and Exploration
3D shape modeling is labor-intensive, time-consuming, and requires years of expertise.
A Generalized Unbiased Risk Estimator for Learning with Augmented Classes
In this paper, we propose a generalized URE that can be equipped with arbitrary loss functions while maintaining the theoretical guarantees, given unlabeled data for LAC.
G$^2$uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering
Through empirical evaluation, comparing G$^2$uardFL with cutting-edge defenses, such as FLAME (USENIX Security 2022) [28] and DeepSight (NDSS 2022) [36], against various backdoor attacks including 3DFed (SP 2023) [20], our results demonstrate its significant effectiveness in mitigating backdoor attacks while having a negligible impact on the aggregated model's performance on benign samples (i. e., the primary task performance).
SketchXAI: A First Look at Explainability for Human Sketches
Following this, we design a simple explainability-friendly sketch encoder that accommodates the intrinsic properties of strokes: shape, location, and order.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
ChiroDiff: Modelling chirographic data with Diffusion Models
Such strictly-ordered discrete factorization however falls short of capturing key properties of chirographic data -- it fails to build holistic understanding of the temporal concept due to one-way visibility (causality).
Learning Garment DensePose for Robust Warping in Virtual Try-On
In this work, we propose a robust warping method for virtual try-on based on a learned garment DensePose which has a direct correspondence with the person's DensePose.
What Can Human Sketches Do for Object Detection?
In particular, we first perform independent prompting on both sketch and photo branches of an SBIR model to build highly generalisable sketch and photo encoders on the back of the generalisation ability of CLIP.
DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion
Concretely, we establish the denoising process in the Transformer decoder (e. g., DETR) by introducing a temporal location query design with faster convergence in training.
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR
This paper advances the fine-grained sketch-based image retrieval (FG-SBIR) literature by putting forward a strong baseline that overshoots prior state-of-the-arts by ~11%.
Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings
Human sketch has already proved its worth in various visual understanding tasks (e. g., retrieval, segmentation, image-captioning, etc).
Picture that Sketch: Photorealistic Image Generation from Abstract Sketches
We further introduce specific designs to tackle the abstract nature of human sketches, including a fine-grained discriminative loss on the back of a trained sketch-photo retrieval model, and a partial-aware sketch augmentation strategy.
BCGS: Blockchain-assisted privacy-preserving cross-domain authentication for VANETs
Vehicular Ad-Hoc Networks (VANETs) have significantly enhanced driving safety and comfort by leveraging vehicular wireless communication technology.
Generative Model Based Noise Robust Training for Unsupervised Domain Adaptation
D-CFA minimizes the domain gap by augmenting the source data with distribution-sampled target features, and trains a noise-robust discriminative classifier by using target domain knowledge from the generative models.
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
In the fashion domain, there exists a variety of vision-and-language (V+L) tasks, including cross-modal retrieval, text-guided image retrieval, multi-modal classification, and image captioning.
A Survey of Secure Computation Using Trusted Execution Environments
To the best of our knowledge, this article is the first survey to review TEE-based secure computation protocols and the comprehensive comparison can serve as a guideline for selecting suitable protocols for deployment in practice.
Unsupervised Hashing with Similarity Distribution Calibration
However, these methods often overlook the fact that the similarity between data points in the continuous feature space may not be preserved in the discrete hash code space, due to the limited similarity range of hash codes.
Representing Noisy Image Without Denoising
In this paper, we explore a non-learning paradigm that aims to derive robust representation directly from noisy images, without the denoising as pre-processing.
Controllable Person Image Synthesis with Pose-Constrained Latent Diffusion
Controllable person image synthesis aims at rendering a source image based on user-specified changes in body pose or appearance.
Democratising 2D Sketch to 3D Shape Retrieval Through Pivoting
We perform pivoting on two existing datasets, each from a distant research domain to the other: 2D sketch and photo pairs from the sketch-based image retrieval field (SBIR), and 3D shapes from ShapeNet.
FBLNet: FeedBack Loop Network for Driver Attention Prediction
However, the nonobjective driving experience is difficult to model, so a mechanism simulating the driver experience accumulation procedure is absent in existing methods, and the current methods usually follow the technique line of saliency prediction methods to predict driver attention.
Multi-Modal Few-Shot Temporal Action Detection
In this work, we introduce a new multi-modality few-shot (MMFS) TAD problem, which can be considered as a marriage of FS-TAD and ZS-TAD by leveraging few-shot support videos and new class names jointly.
Post-Processing Temporal Action Detection
To address this problem, in this work we introduce a novel model-agnostic post-processing method without model redesign and retraining.
Single Stage Multi-Pose Virtual Try-On
Key to our model is a parallel flow estimation module that predicts the flow fields for both person and garment images conditioned on the target pose.
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
Our experiments show the proposed adaptations improve egocentric query detection, leading to a better visual query localization system in both 2D and 3D configurations.
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning
Additionally, these works have mainly been restricted to multimodal understanding tasks.
Learning to Augment via Implicit Differentiation for Domain Generalization
Machine learning models are intrinsically vulnerable to domain shift between training and testing data, resulting in poor performance in novel domains.
Prediction Calibration for Generalized Few-shot Semantic Segmentation
To ensure that the fused scores are not biased to either the base or novel classes, a new Transformer-based calibration module is introduced.
Robust Target Training for Multi-Source Domain Adaptation
Given any existing fully-trained one-step MSDA model, BORT$^2$ turns it to a labeling function to generate pseudo-labels for the target data and trains a target model using pseudo-labeled target data only.
Fine-Grained VR Sketching: Dataset and Insights
We then, for the first time, study the scenario of fine-grained 3D VR sketch to 3D shape retrieval, as a novel VR sketching application and a proving ground to drive out generic insights to inform future research.
Towards 3D VR-Sketch to 3D Shape Retrieval
In this paper, we offer a different perspective towards answering these questions -- we study the use of 3D sketches as an input modality and advocate a VR-scenario where retrieval is conducted.
Structure-Aware 3D VR Sketch to 3D Shape Retrieval
In particular, we propose to use a triplet loss with an adaptive margin value driven by a "fitting gap", which is the similarity of two shapes under structure-preserving deformations.
Negative Frames Matter in Egocentric Visual Query 2D Localization
The repeated gradient computation of the same object lead to an inefficient training; (2) The false positive rate is high on background frames.
Vision Transformers: From Semantic Segmentation to Dense Prediction
Extensive experiments show that our methods achieve appealing performance on a variety of dense prediction tasks (e. g., object detection and instance segmentation and semantic segmentation) as well as image classification.
Zero-Shot Temporal Action Detection via Vision-Language Prompting
Such a novel design effectively eliminates the dependence between localization and classification by breaking the route for error propagation in-between.
Ranked #1 on
Zero-Shot Action Detection
on THUMOS' 14
FashionViL: Fashion-Focused Vision-and-Language Representation Learning
We thus propose a Multi-View Contrastive Learning task for pulling closer the visual representation of one image to the compositional multimodal representation of another image+text.
Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning
Existing temporal action detection (TAD) methods rely on generating an overwhelmingly large number of proposals per video.
Ranked #16 on
Temporal Action Localization
on ActivityNet-1.3
Semi-Supervised Temporal Action Detection with Proposal-Free Masking
Such a novel design effectively eliminates the dependence between localization and classification by cutting off the route for error propagation in-between.
Ranked #1 on
Semi-Supervised Action Detection
on ActivityNet-1.3
Softmax-free Linear Transformers
With linear complexity, much longer token sequences are permitted by SOFT, resulting in superior trade-off between accuracy and complexity.
Adaptive Fine-Grained Sketch-Based Image Retrieval
To solve this new problem, we introduce a novel model-agnostic meta-learning (MAML) based framework with several key modifications: (1) As a retrieval task with a margin-based contrastive loss, we simplify the MAML training in the inner loop to make it more stable and tractable.
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text
In this paper, we extend scene understanding to include that of human sketch.
UIGR: Unified Interactive Garment Retrieval
In this paper, we propose a Unified Interactive Garment Retrieval (UIGR) framework to unify TGR and VCR.
Style-Based Global Appearance Flow for Virtual Try-On
To achieve this, a key step is garment warping which spatially aligns the target garment with the corresponding body parts in the person image.
Ranked #1 on
Virtual Try-on
on VITON
Sketch3T: Test-Time Training for Zero-Shot SBIR
In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i. e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result.
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial".
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval
We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch".
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches
In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints).
class-incremental learning
Few-Shot Class-Incremental Learning
+3
Dynamic Instance Domain Adaptation
In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain.
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context
We advance sketch research to scenes with the first dataset of freehand scene sketches, FS-COCO.
One Sketch for All: One-Shot Personalized Sketch Segmentation
We aim to segment all sketches belonging to the same category provisioned with a single sketch with a given part annotation while (i) preserving the parts semantics embedded in the exemplar, and (ii) being robust to input style and abstraction.
Hybrid Graph Neural Networks for Few-Shot Learning
This is because they use an instance GNN as a label propagation/classification module, which is jointly meta-learned with a feature embedding network.
Domain Attention Consistency for Multi-Source Domain Adaptation
Importantly, the attention module is supervised by a consistency loss, which is imposed on the distributions of channel attention weights between source and target domains.
Towards artificial general intelligence via a multimodal foundation model
To overcome this limitation and take a solid step towards artificial general intelligence (AGI), we develop a foundation model pre-trained with huge multimodal data, which can be quickly adapted for various downstream cognitive tasks.
SOFT: Softmax-free Transformer with Linear Complexity
Crucially, with a linear complexity, much longer token sequences are permitted in SOFT, resulting in superior trade-off between accuracy and complexity.
Text-Based Person Search with Limited Data
Firstly, to fully utilize the existing small-scale benchmarking datasets for more discriminative feature learning, we introduce a cross-modal momentum contrastive learning framework to enrich the training data for a given mini-batch.
Ranked #12 on
Text based Person Retrieval
on CUHK-PEDES
(using extra training data)
Few-Shot Temporal Action Localization with Query Adaptive Transformer
Further, a novel FS-TAL model is proposed which maximizes the knowledge transfer from training classes whilst enabling the model to be dynamically adapted to both the new class and each video of that class simultaneously.
Ranked #1 on
Few Shot Temporal Action Localization
on THUMOS14
Action Segmentation
Few Shot Temporal Action Localization
+4
Temporal Action Localization with Global Segmentation Mask Transformers
In this paper, to address the above two challenges, a novel {\em Global Segmentation Mask Transformer} (GSMT) is proposed.
SketchODE: Learning neural sketch representation in continuous time
Learning meaningful representations for chirographic drawing data such as sketches, handwriting, and flowcharts is a gateway for understanding and emulating human creative expression.
One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective
In this work, we propose a novel deep hashing model with only a single learning objective.
Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels).
Disentangled Lifespan Face Synthesis
The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving.
Global Aggregation then Local Distribution for Scene Parsing
Modelling long-range contextual relationships is critical for pixel-wise prediction tasks such as semantic segmentation.
MixStyle Neural Networks for Domain Generalization and Adaptation
MixStyle is easy to implement with a few lines of code, does not require modification to training objectives, and can fit a variety of learning paradigms including supervised domain generalization, semi-supervised domain generalization, and unsupervised domain adaptation.
L2M-GAN: Learning To Manipulate Latent Space Semantics for Facial Attribute Editing
To overcome these limitations, we propose a novel latent space factorization model, called L2M-GAN, which is learned end-to-end and effective for editing both local and global attributes.
Domain Generalization with MixStyle
Our method, termed MixStyle, is motivated by the observation that visual domain is closely related to image style (e. g., photo vs.~sketch images).
Ranked #66 on
Domain Generalization
on PACS
Cloud2Curve: Generation and Vectorization of Parametric Sketches
Analysis of human sketches in deep learning has advanced immensely through the use of waypoint-sequences rather than raster-graphic representations.
StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
With this meta-learning framework, our model can not only disentangle the cross-modal shared semantic content for SBIR, but can adapt the disentanglement to any unseen user style as well, making the SBIR model truly style-agnostic.
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
This data is uniquely characterised by its existence in dual modalities of rasterized images and vector coordinate sequences.
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
A fundamental challenge faced by existing Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) models is the data scarcity -- model performances are largely bottlenecked by the lack of sketch-photo pairs.
Context-Aware Layout to Image Generation with Enhanced Object Appearance
We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.
Ranked #1 on
Layout-to-Image Generation
on COCO-Stuff 128x128
Enhancement of Superconductivity Linked with Linear-in-Temperature/Field Resistivity in Ion-Gated FeSe Films
Iron selenide (FeSe) - the structurally simplest iron-based superconductor, has attracted tremendous interest in the past years.
Superconductivity
Domain Generalization: A Survey
Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.
Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning
First, data augmentations are introduced to both the support and query sets with each sample now being represented as an augmented embedding (AE) composed of concatenated embeddings of both the original and augmented versions.
Few-shot Action Recognition with Prototype-centered Attentive Learning
Extensive experiments on four standard few-shot action benchmarks show that our method clearly outperforms previous state-of-the-art methods, with the improvement particularly significant (10+\%) on the most challenging fine-grained action recognition benchmark.
Local Black-box Adversarial Attacks: A Query Efficient Approach
Based on this property, we identify the discriminative areas of a given clean example easily for local perturbations.
Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw
With the constrained jigsaw puzzles, instead of solving them directly, which could still be extremely hard, we carefully design four surrogate tasks that are more solvable but meanwhile still ensure that the learned representation is sensitive to spatiotemporal continuity at both the local and global levels.
MELR: Meta-Learning via Modeling Episode-Level Relationships for Few-Shot Learning
Most recent few-shot learning (FSL) approaches are based on episodic training whereby each episode samples few training instances (shots) per class to imitate the test condition.
IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning
Importantly, at the episode-level, two SSL-FSL hybrid learning objectives are devised: (1) The consistency across the predictions of an FSL classifier from different extended episodes is maximized as an episode-level pretext task.
Z-Score Normalization, Hubness, and Few-Shot Learning
This means that these methods are prone to the hubness problem, that is, a certain class prototype becomes the nearest neighbor of many test instances regardless which classes they belong to.
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
In this paper, we aim to provide an alternative perspective by treating semantic segmentation as a sequence-to-sequence prediction task.
Ranked #2 on
Semantic Segmentation
on FoodSeg103
(using extra training data)
Margin-Based Transfer Bounds for Meta Learning with Deep Feature Embedding
By transferring knowledge learned from seen/previous tasks, meta learning aims to generalize well to unseen/future tasks.
Boundary-sensitive Pre-training for Temporal Localization in Videos
However, most existing models developed for these tasks are pre-trained on general video action classification tasks.
Ranked #24 on
Temporal Action Localization
on ActivityNet-1.3
Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models
In this paper, we propose a novel watermark removal attack from a different perspective.
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
In this paper, we study a further trait of sketches that has been overlooked to date, that is, they are hierarchical in terms of the levels of detail -- a person typically sketches up to various extents of detail to depict an object.
On Learning Semantic Representations for Million-Scale Free-Hand Sketches
Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches.
Learning to Generate Novel Domains for Domain Generalization
This explicitly increases the diversity of available training domains and leads to a more generalizable model.
Ranked #74 on
Domain Generalization
on PACS
BézierSketch: A generative model for scalable vector sketches
The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.
Egocentric Action Recognition by Video Attention and Temporal Context
In this challenge, action recognition is posed as the problem of simultaneously predicting a single `verb' and `noun' class label given an input trimmed video clip.
Topology-aware Differential Privacy for Decentralized Image Classification
In this paper, we design Top-DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems.
Long-Term Cloth-Changing Person Re-identification
Specifically, we consider that under cloth-changes, soft-biometrics such as body shape would be more reliable.
Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention
Departing from existing alternatives, our W3 module models all three facets of video attention jointly.
Ranked #1 on
Action Recognition
on EgoGesture
Domain-Adaptive Few-Shot Learning
Existing few-shot learning (FSL) methods make the implicit assumption that the few target class samples are from the same domain as the source class samples.
Domain Adaptive Ensemble Learning
Each such classifier is an expert to its own domain and a non-expert to others.
Deep Domain-Adversarial Image Generation for Domain Generalisation
This is achieved by having a learning objective formulated to ensure that the generated data can be correctly classified by the label classifier while fooling the domain classifier.
Ranked #72 on
Domain Generalization
on PACS
Incremental Few-Shot Object Detection
To this end we propose OpeN-ended Centre nEt (ONCE), a detector designed for incrementally learning to detect novel class objects with few examples.
AdarGCN: Adaptive Aggregation GCN for Few-Shot Learning
To address this problem, we propose a graph convolutional network (GCN)-based label denoising (LDN) method to remove the irrelevant images.
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo instance given a user's query sketch.
Cross-Modal Retrieval
On-the-Fly Sketch Based Image Retrieval
+2
Fine-Grained Instance-Level Sketch-Based Video Retrieval
Existing sketch-analysis work studies sketches depicting static objects or scenes.
Byzantine-resilient Decentralized Stochastic Gradient Descent
It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes.
Meta-Learning across Meta-Tasks for Few-Shot Learning
In this paper, we argue that the inter-meta-task relationships should be exploited and those tasks are sampled strategically to assist in meta-learning.
Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
Specifically, armed with a set transformer based attention module, we construct each episode with two sub-episodes without class overlap on the seen classes to simulate the domain shift between the seen and unseen classes.
Deep Learning for Person Re-identification: A Survey and Outlook
The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets.
Ranked #1 on
Cross-Modal Person Re-Identification
on RegDB-C
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present.
Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch
Person re-identification (re-ID), which aims to re-identify people across different camera views, has been significantly advanced by deep learning in recent years, particularly with convolutional neural networks (CNNs).
Learning Generalisable Omni-Scale Representations for Person Re-Identification
An effective person re-identification (re-ID) model should learn feature representations that are both discriminative, for distinguishing similar-looking people, and generalisable, for deployment across datasets without any adaptation.
Unsupervised Domain Adaptation
Unsupervised Person Re-Identification
Simple and Effective Stochastic Neural Networks
Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian neural networks, variational information bottleneck (VIB) and noise regularized learning.
Few-Shot Learning with Global Class Representations
In this paper, we propose to tackle the challenging few-shot learning (FSL) problem by learning global class representations using both base and novel class training samples.
Goal-Driven Sequential Data Abstraction
In the former one asks whether a machine can `understand' enough about the meaning of input data to produce a meaningful but more compact abstraction.
Omni-Scale Feature Learning for Person Re-Identification
As an instance-level recognition problem, person re-identification (ReID) relies on discriminative features, which not only capture different spatial scales but also encapsulate an arbitrary combination of multiple scales.
Ranked #2 on
Person Re-Identification
on MSMT17-C
Compressing deep neural networks by matrix product operators
A deep neural network is a parametrization of a multilayer mapping of signals in terms of many alternatively arranged linear and nonlinear transformations.
Differentiable Programming Tensor Networks
Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation (AD).
Strongly Correlated Electrons Quantum Physics
Tree Tensor Networks for Generative Modeling
Matrix product states (MPS), a tensor network designed for one-dimensional quantum systems, has been recently proposed for generative modeling of natural data (such as images) in terms of `Born machine'.
Face-Focused Cross-Stream Network for Deception Detection in Videos
To address the training data scarcity problem, our FFCSN model is trained with both meta learning and adversarial learning.
Zero-Shot Learning with Sparse Attribute Propagation
The standard approach to ZSL requires a set of training images annotated with seen class labels and a semantic descriptor for seen/unseen classes (attribute vector is the most widely used).
Disjoint Label Space Transfer Learning with Common Factorised Space
In this paper, a unified approach is presented to transfer learning that addresses several source and target domain label-space and annotation assumptions with a single model.
Ranked #21 on
Unsupervised Domain Adaptation
on Market to Duke
Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning
Inspired by the fact that an unseen class is not exactly `unseen' if it belongs to the same superclass as a seen class, we propose a novel inductive ZSL model that leverages superclasses as the bridge between seen and unseen classes to narrow the domain gap.
Domain-Invariant Projection Learning for Zero-Shot Recognition
This is made possible by learning a projection between a feature space and a semantic space (e. g. attribute space).
Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
Specifically, we assume that each synthesised data point can belong to any unseen class; and the most likely two class candidates are exploited to learn a robust projection function in a competitive fashion.
SketchyScene: Richly-Annotated Scene Sketches
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.
Deep Factorised Inverse-Sketching
Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included.
Video Summarisation by Classification with Deep Reinforcement Learning
Most existing video summarisation methods are based on either supervised or unsupervised learning.
Person Re-Identification in Identity Regression Space
Most existing person re-identification (re-id) methods are unsuitable for real-world deployment due to two reasons: Unscalability to large population size, and Inadaptability over time.
Learning to Sketch with Shortcut Cycle Consistency
In this paper, we present a novel approach for translating an object photo to a sketch, mimicking the human sketching process.
Sketch-a-Classifier: Sketch-based Photo Classifier Generation
Contemporary deep learning techniques have made image recognition a reasonably reliable technology.
Learning Deep Sketch Abstraction
Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR).
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval
Key to our network design is the embedding of unique characteristics of human sketch, where (i) a two-branch CNN-RNN architecture is adapted to explore the temporal ordering of strokes, and (ii) a novel hashing loss is specifically designed to accommodate both the temporal and abstract traits of sketches.
Multi-Level Factorisation Net for Person Re-Identification
Key to effective person re-identification (Re-ID) is modelling discriminative and view-invariant factors of person appearance at both high and low semantic levels.
Inverse Visual Question Answering: A New Benchmark and VQA Diagnosis Tool
The iVQA task is to generate a question that corresponds to a given image and answer pair.
Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward
Video summarization aims to facilitate large-scale video browsing by producing short, concise summaries that are diverse and representative of original videos.
Ranked #7 on
Unsupervised Video Summarization
on TvSum
Pose-Normalized Image Generation for Person Re-identification
Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations.
The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching
Many vision problems require matching images of object instances across different domains.
Learning to Compare: Relation Network for Few-Shot Learning
Once trained, a RN is able to classify images of new classes by computing relation scores between query images and the few examples of each new class without further updating the network.
Recent Advances in Zero-shot Recognition
With the recent renaissance of deep convolution neural networks, encouraging breakthroughs have been achieved on the supervised recognition tasks, where each class has sufficient training data and fully annotated training data.
iVQA: Inverse Visual Question Answering
The iVQA task is to generate a question that corresponds to a given image and answer pair.
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval
Human sketches are unique in being able to capture both the spatial topology of a visual object, as well as its subtle appearance details.
Ranked #2 on
Sketch-Based Image Retrieval
on Handbags
Multi-scale Deep Learning Architectures for Person Re-identification
Our model is able to learn deep discriminative feature representations at different scales and automatically determine the most suitable scales for matching.
Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation
In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept; 2) how to annotate an image with the optimal number of class labels.
Scalable and Effective Deep CCA via Soft Decorrelation
Specifically, exact decorrelation is replaced by soft decorrelation via a mini-batch based Stochastic Decorrelation Loss (SDL) to be optimised jointly with the other training objectives.
Zero-Shot Fine-Grained Classification by Deep Feature Learning with Semantics
In this paper, to address the two issues, we propose a two-phase framework for recognizing images from unseen fine-grained classes, i. e. zero-shot fine-grained classification.
Learning to Learn: Meta-Critic Networks for Sample Efficient Learning
We propose a novel and flexible approach to meta-learning for learning-to-learn from only a few examples.
Actor-Critic Sequence Training for Image Captioning
Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing.
Bayesian Joint Modelling for Object Localisation in Weakly Labelled Images
We address the problem of localisation of objects as bounding boxes in images and videos with weak labels.
Transferring a Semantic Representation for Person Re-Identification and Search
Learning semantic attributes for person re-identification and description-based person search has gained increasing interest due to attributes' great potential as a pose and view-invariant representation.
Deep Mutual Learning
Model distillation is an effective and widely used technique to transfer knowledge from a teacher to a student network.
Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
Sketch-based image retrieval (SBIR) is challenging due to the inherent domain-gap between sketch and photo.
Ranked #5 on
Sketch-Based Image Retrieval
on Chairs
Bayesian Joint Topic Modelling for Weakly Supervised Object Localisation
(3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning.
Transfer Learning by Ranking for Weakly Supervised Object Annotation
Most existing approaches to training object detectors rely on fully supervised learning, which requires the tedious manual annotation of object location in a training set.
Semantic Autoencoder for Zero-Shot Learning
We show that with this additional reconstruction constraint, the learned projection function from the seen classes is able to generalise better to the new unseen classes.
Equivalence of restricted Boltzmann machines and tensor network states
Conversely, we give sufficient and necessary conditions to determine whether a TNS can be transformed into an RBM of given architectures.
Human-In-The-Loop Person Re-Identification
Current person re-identification (re-id) methods assume that (1) pre-labelled training data are available for every camera pair, (2) the gallery size for re-identification is moderate.
Highly Efficient Regression for Scalable Person Re-Identification
Existing person re-identification models are poor for scaling up to large data required in real-world applications due to: (1) Complexity: They employ complex models for optimal performance resulting in high computational cost for training at a large scale; (2) Inadaptability: Once trained, they are unsuitable for incremental update to incorporate any new data available.
Semantic Regularisation for Recurrent Image Annotation
We propose a simple modification to the design pattern that makes learning more effective and efficient.
Deep Transfer Learning for Person Re-identification
Second, a two-stepped fine-tuning strategy is developed to transfer knowledge from auxiliary datasets.
Learning a Deep Embedding Model for Zero-Shot Learning
In this paper we argue that the key to make deep ZSL models succeed is to choose the right embedding space.
Ranked #16 on
Zero-Shot Action Recognition
on Kinetics
Unsupervised Cross-Dataset Transfer Learning for Person Re-Identification
Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.
Sketch Me That Shoe
We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images.
Ranked #3 on
Sketch-Based Image Retrieval
on Chairs
Learning a Discriminative Null Space for Person Re-identification
Most existing person re-identification (re-id) methods focus on learning the optimal distance metrics across camera views.
Ranked #120 on
Person Re-Identification
on Market-1501
Unsupervised Domain Adaptation for Zero-Shot Learning
Zero-shot learning (ZSL) can be considered as a special case of transfer learning where the source and target domains have different tasks/label spaces and the target domain is unlabelled, providing little guidance for the knowledge transfer.
Partial Person Re-Identification
We address a new partial person re-identification (re-id) problem, where only a partial observation of a person is available for matching across different non-overlapping camera views.
Multi-Scale Learning for Low-Resolution Person Re-Identification
In real world person re-identification (re-id), images of people captured at very different resolutions from different locations need be matched.
