Search Results for author: Josef Kittler

Found 83 papers, 23 papers with code

Generative-based Fusion Mechanism for Multi-Modal Tracking

1 code implementation4 Sep 2023 Zhangyong Tang, Tianyang Xu, XueFeng Zhu, Xiao-Jun Wu, Josef Kittler

In this context, we seek to uncover the potential of harnessing generative techniques to address the critical challenge, information fusion, in multi-modal tracking.

Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding

no code implementations22 Aug 2023 Jiantao Wu, Shentong Mo, Muhammad Awais, Sara Atito, ZhenHua Feng, Josef Kittler

Self-supervised pretraining (SSP) has emerged as a popular technique in machine learning, enabling the extraction of meaningful feature representations without labelled data.

Contrastive Learning Self-Supervised Learning +4

FusionBooster: A Unified Image Fusion Boosting Paradigm

no code implementations10 May 2023 Chunyang Cheng, Tianyang Xu, Xiao-Jun Wu, Hui Li, Xi Li, Josef Kittler

We argue that there is a scope to improve the fusion performance with the help of the FusionBooster, a model specifically designed for the fusion task.

LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images

1 code implementation11 Apr 2023 Hui Li, Tianyang Xu, Xiao-Jun Wu, Jiwen Lu, Josef Kittler

In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model.

Representation Learning

ASiT: Audio Spectrogram vIsion Transformer for General Audio Representation

no code implementations23 Nov 2022 Sara Atito, Muhammad Awais, Wenwu Wang, Mark D Plumbley, Josef Kittler

Vision transformers, which were originally developed for natural language processing, have recently generated significant interest in the computer vision and audio communities due to their flexibility in learning long-range relationships.

Keyword Spotting Speaker Identification

RGBD1K: A Large-scale Dataset and Benchmark for RGB-D Object Tracking

1 code implementation21 Aug 2022 Xue-Feng Zhu, Tianyang Xu, Zhangyong Tang, Zucheng Wu, Haodong Liu, Xiao Yang, Xiao-Jun Wu, Josef Kittler

To demonstrate the benefits of training on a larger RGB-D data set in general, and RGBD1K in particular, we develop a transformer-based RGB-D tracker, named SPT, as a baseline for future visual object tracking studies using the new dataset.

Visual Object Tracking

Doubly Reparameterized Importance Weighted Structure Learning for Scene Graph Generation

no code implementations22 Jun 2022 Daqi Liu, Miroslaw Bober, Josef Kittler

As a structured prediction task, scene graph generation, given an input image, aims to explicitly model objects and their relationships by constructing a visually-grounded scene graph.

Graph Generation Scene Graph Generation +2

DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification

no code implementations16 Jun 2022 Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, Josef Kittler

Image set-based visual classification methods have achieved remarkable performance, via characterising the image set in terms of a non-singular covariance matrix on a symmetric positive definite (SPD) manifold.

GMML is All you Need

1 code implementation30 May 2022 Sara Atito, Muhammad Awais, Josef Kittler

This has motivated the research in self-supervised transformer pretraining, which does not need to decode the semantic information conveyed by labels to link it to the image properties, but rather focuses directly on extracting a concise representation of the image data that reflects the notion of similarity, and is invariant to nuisance factors.

Data Augmentation Self-Learning +1

Importance Weighted Structure Learning for Scene Graph Generation

no code implementations14 May 2022 Daqi Liu, Miroslaw Bober, Josef Kittler

Scene graph generation is a structured prediction task aiming to explicitly model objects and their relationships via constructing a visually-grounded scene graph for an input image.

Graph Generation Scene Graph Generation +2

Constrained Structure Learning for Scene Graph Generation

no code implementations27 Jan 2022 Daqi Liu, Miroslaw Bober, Josef Kittler

As a structured prediction task, scene graph generation aims to build a visually-grounded scene graph to explicitly model objects and their relationships in an input image.

Graph Generation Scene Graph Generation +2

Riemannian Local Mechanism for SPD Neural Networks

1 code implementation25 Jan 2022 Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, Josef Kittler

The Symmetric Positive Definite (SPD) matrices have received wide attention for data representation in many scientific areas.


Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

1 code implementation21 Jan 2022 Zhangyong Tang, Tianyang Xu, Hui Li, Xiao-Jun Wu, XueFeng Zhu, Josef Kittler

The effectiveness of the proposed decision-level fusion strategy owes to a number of innovative contributions, including a dynamic weighting of the RGB and TIR contributions and a linear template update operation.

Visual Object Tracking

Neural Belief Propagation for Scene Graph Generation

no code implementations10 Dec 2021 Daqi Liu, Miroslaw Bober, Josef Kittler

Scene graph generation aims to interpret an input image by explicitly modelling the potential objects and their relationships, which is predominantly solved by the message passing neural network models in previous methods.

Graph Generation Scene Graph Generation

MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning

no code implementations30 Nov 2021 Sara Atito, Muhammad Awais, Ammarah Farooq, ZhenHua Feng, Josef Kittler

In this aspect the proposed SSL frame-work MC-SSL0. 0 is a step towards Multi-Concept Self-Supervised Learning (MC-SSL) that goes beyond modelling single dominant label in an image to effectively utilise the information from all the concepts present in it.

Image Classification Self-Supervised Learning +1

Global Interaction Modelling in Vision Transformer via Super Tokens

no code implementations25 Nov 2021 Ammarah Farooq, Muhammad Awais, Sara Ahmed, Josef Kittler

Hence, most of the learning is independent of the image patches $(N)$ in the higher layers, and the class embedding is learned solely based on the Super tokens $(N/M^2)$ where $M^2$ is the window size.

Image Classification Representation Learning

Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms

no code implementations17 Nov 2021 Norman Poh, Thirimachos Bourlai, Josef Kittler, Lorene Allano, Fernando Alonso-Fernandez, Onkar Ambekar, John Baker, Bernadette Dorizzi, Omolara Fatukasi, Julian Fierrez, Harald Ganster, Javier Ortega-Garcia, Donald Maurer, Albert Ali Salah, Tobias Scheidat, Claus Vielhauer

The cost-sensitive evaluation, on the other hand, investigates how well a fusion algorithm can perform given restricted computation and in the presence of software and hardware failures, resulting in errors such as failure-to-acquire and failure-to-match.


Learning PAC-Bayes Priors for Probabilistic Neural Networks

no code implementations21 Sep 2021 Maria Perez-Ortiz, Omar Rivasplata, Benjamin Guedj, Matthew Gleeson, Jingyu Zhang, John Shawe-Taylor, Miroslaw Bober, Josef Kittler

We experiment on 6 datasets with different strategies and amounts of data to learn data-dependent PAC-Bayes priors, and we compare them in terms of their effect on test performance of the learnt predictors and tightness of their risk certificate.

MOON: Multi-Hash Codes Joint Learning for Cross-Media Retrieval

no code implementations17 Aug 2021 Donglin Zhang, Xiao-Jun Wu, He-Feng Yin, Josef Kittler

To this end, we develop a novel Multiple hash cOdes jOint learNing method (MOON) for cross-media retrieval.


PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion

no code implementations29 Jul 2021 Yu Fu, Tianyang Xu, XiaoJun Wu, Josef Kittler

In this paper, we propose a Patch Pyramid Transformer(PPT) to effectively address the above issues. Specifically, we first design a Patch Transformer to transform the image into a sequence of patches, where transformer encoding is performed for each patch to extract local representations.

Image Classification Image Reconstruction

SiT: Self-supervised vIsion Transformer

2 code implementations8 Apr 2021 Sara Atito, Muhammad Awais, Josef Kittler

We also observed that SiT is good for few shot learning and also showed that it is learning useful representation by simply training a linear classifier on top of the learned features from SiT.

Few-Shot Learning Self-Supervised Learning

RFN-Nest: An end-to-end residual fusion network for infrared and visible images

1 code implementation7 Mar 2021 Hui Li, Xiao-Jun Wu, Josef Kittler

The most difficult part of the design is to choose an appropriate strategy to generate the fused image for a specific task in hand.

Infrared And Visible Image Fusion

NPT-Loss: A Metric Loss with Implicit Mining for Face Recognition

no code implementations5 Mar 2021 Syed Safwan Khalid, Muhammad Awais, Chi-Ho Chan, ZhenHua Feng, Ammarah Farooq, Ali Akbari, Josef Kittler

One key ingredient of DCNN-based FR is the appropriate design of a loss function that ensures discrimination between various identities.

Face Recognition

Differentiable Neural Architecture Learning for Efficient Neural Network Design

1 code implementation3 Mar 2021 Qingbei Guo, Xiao-Jun Wu, Josef Kittler, Zhiquan Feng

To address this computational complexity issue, we introduce a novel \emph{architecture parameterisation} based on scaled sigmoid function, and propose a general \emph{Differentiable Neural Architecture Learning} (DNAL) method to optimize the neural architecture without the need to evaluate candidate neural networks.

Efficient Neural Network Neural Architecture Search

Separable Batch Normalization for Robust Facial Landmark Localization with Cross-protocol Network Training

no code implementations17 Jan 2021 Shuangping Jin, ZhenHua Feng, Wankou Yang, Josef Kittler

Different from the standard BN layer that uses all the training data to calculate a single set of parameters, SepBN considers that the samples of a training dataset may belong to different sub-domains.

Face Alignment

A Flatter Loss for Bias Mitigation in Cross-dataset Facial Age Estimation

no code implementations20 Oct 2020 Ali Akbari, Muhammad Awais, Zhen-Hua Feng, Ammarah Farooq, Josef Kittler

Compared with existing loss functions, the lower gradient of the proposed loss function leads to the convergence of SGD to a better optimum point, and consequently a better generalisation.

Age Estimation Benchmarking

Self-grouping Convolutional Neural Networks

1 code implementation29 Sep 2020 Qingbei Guo, Xiao-Jun Wu, Josef Kittler, Zhiquan Feng

To tackle this issue, we propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN, in which the filters of each convolutional layer group themselves based on the similarity of their importance vectors.

Clustering Domain Adaptation +3

Deep Convolutional Neural Network Ensembles using ECOC

no code implementations7 Sep 2020 Sara Atito Ali Ahmed, Cemre Zor, Berrin Yanikoglu, Muhammad Awais, Josef Kittler

Deep neural networks have enhanced the performance of decision making systems in many applications including image understanding, and further gains can be achieved by constructing ensembles.

Decision Making

AFAT: Adaptive Failure-Aware Tracker for Robust Visual Object Tracking

no code implementations27 May 2020 Tianyang Xu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler

To this end, we propose a failure-aware system, realised by a Quality Prediction Network (QPN), based on convolutional and LSTM modules in the decision stage, enabling online reporting of potential tracking failures.

Benchmarking One-Shot Learning +1

Adaptive Distraction Context Aware Tracking Based on Correlation Filter

no code implementations24 Dec 2019 Fei Feng, Xiao-Jun Wu, Tianyang Xu, Josef Kittler, Xue-Feng Zhu

In the response map obtained for the previous frame by the CF algorithm, we adaptively find the image blocks that are similar to the target and use them as negative samples.

Face Recognition via Locality Constrained Low Rank Representation and Dictionary Learning

1 code implementation6 Dec 2019 He-Feng Yin, Xiao-Jun Wu, Josef Kittler

First, a low-rank representation is introduced to handle the possible contamination of the training as well as test data.

Dictionary Learning Face Recognition +1

An Accelerated Correlation Filter Tracker

no code implementations5 Dec 2019 Tianyang Xu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler

Recent visual object tracking methods have witnessed a continuous improvement in the state-of-the-art with the development of efficient discriminative correlation filters (DCF) and robust deep neural network features.

Benchmarking Visual Object Tracking

Learning a Representation with the Block-Diagonal Structure for Pattern Classification

no code implementations23 Nov 2019 He-Feng Yin, Xiao-Jun Wu, Josef Kittler, Zhen-Hua Feng

To counteract this problem, we propose an approach that learns Representation with Block-Diagonal Structure (RBDS) for robust image recognition.

Benchmarking Classification +2

Dual Encoder-Decoder based Generative Adversarial Networks for Disentangled Facial Representation Learning

no code implementations19 Sep 2019 Cong Hu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler

To be more specific, the encoder-decoder structured generator is used to learn a pose disentangled face representation, and the encoder-decoder structured discriminator is tasked to perform real/fake classification, face reconstruction, determining identity and estimating face pose.

Benchmarking Face Generation +5

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework based Kernel Matrix Representation

2 code implementations16 Sep 2019 Kai-Xuan Chen, Xiao-Jun Wu, Jie-Yi Ren, Rui Wang, Josef Kittler

We consider a family of structural descriptors for visual data, namely covariance descriptors (CovDs) that lie on a non-linear symmetric positive definite (SPD) manifold, a special type of Riemannian manifolds.

Multiple Riemannian Manifold-valued Descriptors based Image Set Classification with Multi-Kernel Metric Learning

no code implementations6 Aug 2019 Rui Wang, Xiao-Jun Wu, Josef Kittler

Specifically, the covariance matrix, linear subspace, and Gaussian distribution are applied for set representation simultaneously.

Classification Emotion Recognition +5

Multi-Task Kernel Null-Space for One-Class Classification

no code implementations22 May 2019 Shervin Rahimzadeh Arashloo, Josef Kittler

The non-linear structure learning method is then extended to a sparse setting where different tasks compete in an output composition mechanism, leading to a sparse non-linear structure among multiple problems.

Classification General Classification +2

Transition Subspace Learning based Least Squares Regression for Image Classification

no code implementations14 May 2019 Zhe Chen, Xiao-Jun Wu, Josef Kittler

Only learning one projection matrix from original samples to the corresponding binary labels is too strict and will consequentlly lose some intrinsic geometric structures of data.

Classification General Classification +2

Low-Rank Discriminative Least Squares Regression for Image Classification

no code implementations19 Mar 2019 Zhe Chen, Xiao-Jun Wu, Josef Kittler

To solve above problems, we propose a low-rank discriminative least squares regression model (LRDLSR) for multi-class image classification.

Classification General Classification +2

Fisher Discriminative Least Squares Regression for Image Classification

1 code implementation19 Mar 2019 Zhe Chen, Xiao-Jun Wu, Josef Kittler

On one hand, the Fisher criterion improves the intra-class compactness of the relaxed labels during relaxation learning.

Classification Face Recognition +3

Non-negative representation based discriminative dictionary learning for face recognition

no code implementations19 Mar 2019 Zhe Chen, Xiao-Jun Wu, Josef Kittler

In this paper, we propose a non-negative representation based discriminative dictionary learning algorithm (NRDL) for multicategory face classification.

Dictionary Learning Face Recognition +1

Visual Semantic Information Pursuit: A Survey

no code implementations13 Mar 2019 Daqi Liu, Miroslaw Bober, Josef Kittler

Since it helps to enhance the accuracy and the consistency of the resulting interpretation, visual context reasoning is often incorporated with visual perception in current deep end-to-end visual semantic information pursuit methods.

Graph Generation object-detection +5

Robust One-Class Kernel Spectral Regression

no code implementations6 Feb 2019 Shervin Rahimzadeh Arashloo, Josef Kittler

This work addresses these shortcomings by studying the effect of regularising the solution of the null-space kernel Fisher methodology in the context of its regression-based formulation (OC-KSR).

General Classification One-Class Classification +1

Discriminative Supervised Hashing for Cross-Modal similarity Search

no code implementations6 Dec 2018 Jun Yu, Xiao-Jun Wu, Josef Kittler

With the advantage of low storage cost and high retrieval efficiency, hashing techniques have recently been an emerging topic in cross-modal similarity search.

Cross-Modal Retrieval Retrieval

Pose Invariant 3D Face Reconstruction

no code implementations13 Nov 2018 Lei Jiang, Xiao-Jun Wu, Josef Kittler

Our method solves the problem of face reconstruction of a single image of a traditional method in a large pose, works on arbitrary Pose and Expressions, greatly improves the accuracy of reconstruction.

3D Face Reconstruction

MDLatLRR: A novel decomposition method for infrared and visible image fusion

2 code implementations6 Nov 2018 Hui Li, Xiao-Jun Wu, Josef Kittler

We develop a novel image fusion framework based on MDLatLRR, which is used to decompose source images into detail parts(salient features) and base parts.

Infrared And Visible Image Fusion

Landmark Weighting for 3DMM Shape Fitting

no code implementations16 Aug 2018 Yu Yanga, Xiao-Jun Wu, Josef Kittler

In this paper we show that landmark weighting is instrumental to improve the accuracy of shape reconstruction and propose a novel 3D Morphable Model Fitting method.

3D Face Reconstruction

Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking

1 code implementation30 Jul 2018 Tianyang Xu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler

The key innovations of the proposed method include adaptive spatial feature selection and temporal consistent constraints, with which the new tracker enables joint spatial-temporal filter learning in a lower dimensional discriminative manifold.

Benchmarking feature selection +2

One-Class Kernel Spectral Regression

no code implementations3 Jul 2018 Shervin Rahimzadeh Arashloo, Josef Kittler

The paper introduces a new efficient nonlinear one-class classifier formulated as the Rayleigh quotient criterion optimisation.

Graph Embedding One-class classifier +1

Client-Specific Anomaly Detection for Face Presentation Attack Detection

no code implementations2 Jul 2018 Shervin Rahimzadeh Arashloo, Josef Kittler

In addition, it is demonstrated that the same set of deep convolutional features used for the recognition purposes is effective for face presentation attack detection in the class-specific one-class anomaly detection paradigm.

Anomaly Detection Decision Making +1

Semi-supervised Hashing for Semi-Paired Cross-View Retrieval

no code implementations19 Jun 2018 Jun Yu, Xiao-Jun Wu, Josef Kittler

Recently, hashing techniques have gained importance in large-scale retrieval tasks because of their retrieval speed.


Riemannian kernel based Nyström method for approximate infinite-dimensional covariance descriptors with application to image set classification

no code implementations16 Jun 2018 Kai-Xuan Chen, Xiao-Jun Wu, Rui Wang, Josef Kittler

We propose a novel framework for representing image sets by approximating infinite-dimensional CovDs in the paradigm of the Nystr\"om method based on a Riemannian kernel.

General Classification

Multiple Manifolds Metric Learning with Application to Image Set Classification

no code implementations30 May 2018 Rui Wang, Xiao-Jun Wu, Kai-Xuan Chen, Josef Kittler

In image set classification, a considerable advance has been made by modeling the original image sets by second order statistics or linear subspace, which typically lie on the Riemannian manifold.

Classification Face Recognition +3

A Simple Riemannian Manifold Network for Image Set Classification

no code implementations27 May 2018 Rui Wang, Xiao-Jun Wu, Josef Kittler

The proposed RieMNet and DRieMNet are evaluated on three tasks: video-based face recognition, set-based object categorization, and set-based cell identification.

Classification Face Recognition +2

L1-(2D)2PCANet: A Deep Learning Network for Face Recognition

no code implementations26 May 2018 YunKun Li, Xiao-Jun Wu, Josef Kittler

In our network, the role of L1-(2D)2PCA is to learn the filters of multiple convolution layers.

Face Recognition

Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

1 code implementation ECCV 2018 Baris Gecer, Binod Bhattarai, Josef Kittler, Tae-Kyun Kim

We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model.

Domain Adaptation Face Generation +3

Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild

no code implementations14 Mar 2018 Zhen-Hua Feng, Patrik Huber, Josef Kittler, Peter JB Hancock, Xiao-Jun Wu, Qijun Zhao, Paul Koppen, Matthias Rätsch

To this end, we organise a competition that provides a new benchmark dataset that contains 2000 2D facial images of 135 subjects as well as their 3D ground truth face scans.

3D Face Reconstruction 3D Reconstruction +1

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

6 code implementations CVPR 2018 Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Huber, Xiao-Jun Wu

We present a new loss function, namely Wing loss, for robust facial landmark localisation with Convolutional Neural Networks (CNNs).

 Ranked #1 on Face Alignment on 300W (NME_inter-pupil (%, Common) metric)

Data Augmentation Face Alignment

Person Re-Identification with Vision and Language

no code implementations3 Oct 2017 Fei Yan, Krystian Mikolajczyk, Josef Kittler

We propose a joint vision and language model based on CCA and CNN architectures to match across the two modalities as well as to enrich visual examples for which there are no language descriptions.

Language Modelling Person Re-Identification

3D Morphable Models as Spatial Transformer Networks

1 code implementation23 Aug 2017 Anil Bas, Patrik Huber, William A. P. Smith, Muhammad Awais, Josef Kittler

In this paper, we show how a 3D Morphable Model (i. e. a statistical model of the 3D shape of a class of objects such as faces) can be used to spatially transform input data as a module (a 3DMM-STN) within a convolutional neural network.

A Unified Tensor-based Active Appearance Face Model

no code implementations30 Dec 2016 Zhen-Hua Feng, Josef Kittler, William Christmas, Xiao-Jun Wu

To deal with this challenge, we present a Unified Tensor-based Active Appearance Model (UT-AAM) for jointly modelling the geometry and texture information of 2D faces.

Face Model

Dictionary Integration using 3D Morphable Face Models for Pose-invariant Collaborative-representation-based Classification

no code implementations1 Nov 2016 Xiaoning Song, Zhen-Hua Feng, Guosheng Hu, Josef Kittler, William Christmas, Xiao-Jun Wu

The paper presents a dictionary integration algorithm using 3D morphable face models (3DMM) for pose-invariant collaborative-representation-based face classification.

Classification General Classification

3D Face Tracking and Texture Fusion in the Wild

no code implementations22 May 2016 Patrik Huber, Philipp Kopp, Matthias Rätsch, William Christmas, Josef Kittler

We present a fully automatic approach to real-time 3D face reconstruction from monocular in-the-wild videos.

3D Face Reconstruction Face Model

Delta divergence: A novel decision cognizant measure of classifier incongruence

no code implementations15 Apr 2016 Josef Kittler, Cemre Zor

In this paper, we postulate the properties that a divergence measure should satisfy and propose a novel divergence measure, referred to as Delta divergence.

A Multiresolution 3D Morphable Face Model and Fitting Framework

1 code implementation1 Feb 2016 Patrik Huber, Guosheng Hu, Rafael Tena, Pouria Mortazavian, Willem P. Koppen, William Christmas, Matthias Rätsch, Josef Kittler

In this paper, we present the Surrey Face Model, a multi-resolution 3D Morphable Model that we make available to the public for non-commercial purposes.

3D Face Reconstruction Face Generation +4

Fitting 3D Morphable Models using Local Features

1 code implementation8 Mar 2015 Patrik Huber, Zhen-Hua Feng, William Christmas, Josef Kittler, Matthias Rätsch

Our approach is unique in that we are the first to use local features to fit a Morphable Model.


Cannot find the paper you are looking for? You can Submit a new open access paper.