Search Results for author: Munawar Hayat

Found 68 papers, 42 papers with code

DiffAugment: Diffusion based Long-Tailed Visual Relationship Recognition

no code implementations1 Jan 2024 Parul Gupta, Tuan Nguyen, Abhinav Dhall, Munawar Hayat, Trung Le, Thanh-Toan Do

The task of Visual Relationship Recognition (VRR) aims to identify relationships between two interacting objects in an image and is particularly challenging due to the widely-spread and highly imbalanced distribution of <subject, relation, object> triplets.

Object Relation

AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset

1 code implementation26 Nov 2023 Zhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat, Abhinav Dhall, Kalin Stefanov

The comprehensive benchmark of the proposed dataset utilizing state-of-the-art deepfake detection and localization methods indicates a significant drop in performance compared to previous datasets.

DeepFake Detection Face Swapping +1

Open-Vocabulary Object Detection via Scene Graph Discovery

no code implementations7 Jul 2023 Hengcan Shi, Munawar Hayat, Jianfei Cai

However, they only use pairs of nouns and individual objects in VL data, while these data usually contain much more information, such as scene graphs, which are also crucial for OV detection.

Graph Generation Object +5

Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization

1 code implementation3 May 2023 Zhixi Cai, Shreya Ghosh, Abhinav Dhall, Tom Gedeon, Kalin Stefanov, Munawar Hayat

The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations.

Binary Classification DeepFake Detection +2

Real-time Trajectory-based Social Group Detection

1 code implementation12 Apr 2023 Simindokht Jahangard, Munawar Hayat, Hamid Rezatofighi

These results demonstrate that our proposed method is suitable for real-time robotic applications.

Graph Clustering Robot Navigation

ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning

no code implementations CVPR 2023 Islam Nassar, Munawar Hayat, Ehsan Abbasnejad, Hamid Rezatofighi, Gholamreza Haffari

Finally, ProtoCon addresses the poor training signal in the initial phase of training (due to fewer confident predictions) by introducing an auxiliary self-supervised loss.

Online Clustering Pseudo Label

Energy-based Self-Training and Normalization for Unsupervised Domain Adaptation

no code implementations ICCV 2023 Samitha Herath, Basura Fernando, Ehsan Abbasnejad, Munawar Hayat, Shahram Khadivi, Mehrtash Harandi, Hamid Rezatofighi, Gholamreza Haffari

EBL can be used to improve the instance selection for a self-training task on the unlabelled target domain, and 2. alignment and normalizing energy scores can learn domain-invariant representations.

Unsupervised Domain Adaptation

Zero-Shot Point Cloud Segmentation by Semantic-Visual Aware Synthesis

1 code implementation ICCV 2023 Yuwei Yang, Munawar Hayat, Zhao Jin, Hongyuan Zhu, Yinjie Lei

Given only the class-level semantic information for unseen objects, we strive to enhance the correspondence, alignment and consistency between the visual and semantic spaces, to synthesise diverse, generic and transferable visual features.

Point Cloud Segmentation Segmentation +2

Context-Aware Alignment and Mutual Masking for 3D-Language Pre-Training

1 code implementation CVPR 2023 Zhao Jin, Munawar Hayat, Yuwei Yang, Yulan Guo, Yinjie Lei

The current approaches for 3D visual reasoning are task-specific, and lack pre-training methods to learn generic representations that can transfer across various tasks.

3D dense captioning Dense Captioning +3

Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation

1 code implementation CVPR 2023 Yuwei Yang, Munawar Hayat, Zhao Jin, Chao Ren, Yinjie Lei

Despite the significant recent progress made on 3D point cloud semantic segmentation, the current methods require training data for all classes at once, and are not suitable for real-life scenarios where new categories are being continuously discovered.

Class-Incremental Semantic Segmentation

MARLIN: Masked Autoencoder for facial video Representation LearnINg

1 code implementation CVPR 2023 Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall, Jianfei Cai, Hamid Rezatofighi, Reza Haffari, Munawar Hayat

This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS).

Action Classification Attribute +9

CLIP model is an Efficient Continual Learner

1 code implementation6 Oct 2022 Vishal Thengane, Salman Khan, Munawar Hayat, Fahad Khan

In this work, we show that a frozen CLIP (Contrastive Language-Image Pretraining) model offers astounding continual learning performance without any fine-tuning (zero-shot evaluation).

Continual Learning Incremental Learning +1

Hybrid Window Attention Based Transformer Architecture for Brain Tumor Segmentation

1 code implementation16 Sep 2022 Himashi Peiris, Munawar Hayat, Zhaolin Chen, Gary Egan, Mehrtash Harandi

As intensities of MRI volumes are inconsistent across institutes, it is essential to extract universal features of multi-modal MRIs to precisely segment brain tumors.

Brain Tumor Segmentation Tumor Segmentation

Concealing Sensitive Samples against Gradient Leakage in Federated Learning

1 code implementation13 Sep 2022 Jing Wu, Munawar Hayat, Mingyi Zhou, Mehrtash Harandi

Federated Learning (FL) is a distributed learning paradigm that enhances users privacy by eliminating the need for clients to share raw, private data with the server.

Federated Learning Stochastic Optimization

'Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation

1 code implementation3 Aug 2022 Shreya Ghosh, Abhinav Dhall, Jarrod Knibbe, Munawar Hayat

Our proposed method reduces the annotation effort to as low as 2. 67%, with minimal impact on performance; indicating the potential of our model enabling gaze estimation 'in-the-wild' setup.

Gaze Estimation

AV-Gaze: A Study on the Effectiveness of Audio Guided Visual Attention Estimation for Non-Profilic Faces

1 code implementation7 Jul 2022 Shreya Ghosh, Abhinav Dhall, Munawar Hayat, Jarrod Knibbe

In challenging real-life conditions such as extreme head-pose, occlusions, and low-resolution images where the visual information fails to estimate visual attention/gaze direction, audio signals could provide important and complementary information.

Transformer Scale Gate for Semantic Segmentation

no code implementations CVPR 2023 Hengcan Shi, Munawar Hayat, Jianfei Cai

Effectively encoding multi-scale contextual information is crucial for accurate semantic segmentation.

feature selection Segmentation +1

NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results

2 code implementations11 May 2022 Yawei Li, Kai Zhang, Radu Timofte, Luc van Gool, Fangyuan Kong, Mingxi Li, Songwei Liu, Zongcai Du, Ding Liu, Chenhui Zhou, Jingyi Chen, Qingrui Han, Zheyuan Li, Yingqi Liu, Xiangyu Chen, Haoming Cai, Yu Qiao, Chao Dong, Long Sun, Jinshan Pan, Yi Zhu, Zhikai Zong, Xiaoxiao Liu, Zheng Hui, Tao Yang, Peiran Ren, Xuansong Xie, Xian-Sheng Hua, Yanbo Wang, Xiaozhong Ji, Chuming Lin, Donghao Luo, Ying Tai, Chengjie Wang, Zhizhong Zhang, Yuan Xie, Shen Cheng, Ziwei Luo, Lei Yu, Zhihong Wen, Qi Wu1, Youwei Li, Haoqiang Fan, Jian Sun, Shuaicheng Liu, Yuanfei Huang, Meiguang Jin, Hua Huang, Jing Liu, Xinjian Zhang, Yan Wang, Lingshun Long, Gen Li, Yuanfan Zhang, Zuowei Cao, Lei Sun, Panaetov Alexander, Yucong Wang, Minjie Cai, Li Wang, Lu Tian, Zheyuan Wang, Hongbing Ma, Jie Liu, Chao Chen, Yidong Cai, Jie Tang, Gangshan Wu, Weiran Wang, Shirui Huang, Honglei Lu, Huan Liu, Keyan Wang, Jun Chen, Shi Chen, Yuchun Miao, Zimo Huang, Lefei Zhang, Mustafa Ayazoğlu, Wei Xiong, Chengyi Xiong, Fei Wang, Hao Li, Ruimian Wen, Zhijing Yang, Wenbin Zou, Weixin Zheng, Tian Ye, Yuncheng Zhang, Xiangzhen Kong, Aditya Arora, Syed Waqas Zamir, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Dandan Gaoand Dengwen Zhouand Qian Ning, Jingzhu Tang, Han Huang, YuFei Wang, Zhangheng Peng, Haobo Li, Wenxue Guan, Shenghua Gong, Xin Li, Jun Liu, Wanjun Wang, Dengwen Zhou, Kun Zeng, Hanjiang Lin, Xinyu Chen, Jinsheng Fang

The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.

Image Super-Resolution

Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization

1 code implementation13 Apr 2022 Zhixi Cai, Kalin Stefanov, Abhinav Dhall, Munawar Hayat

Our baseline method for benchmarking the proposed dataset is a 3DCNN model, termed as Boundary Aware Temporal Forgery Detection (BA-TFD), which is guided via contrastive, boundary matching, and frame classification loss functions.

Benchmarking DeepFake Detection +1

Semantic-Aware Domain Generalized Segmentation

1 code implementation CVPR 2022 Duo Peng, Yinjie Lei, Munawar Hayat, Yulan Guo, Wen Li

In this paper, we address domain generalized semantic segmentation, where a segmentation model is trained to be domain-invariant without using any target domain data.

Domain Generalization Segmentation +1

Transformers in Medical Imaging: A Survey

1 code implementation24 Jan 2022 Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, Huazhu Fu

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators.

Image Classification Image Segmentation +6

Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching

no code implementations18 Jan 2022 Hengcan Shi, Munawar Hayat, Jianfei Cai

To avoid the laborious annotation in conventional referring grounding, unpaired referring grounding is introduced, where the training data only contains a number of images and queries without correspondences.

Image-text matching Referring Expression +1

ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cues

no code implementations CVPR 2022 Hengcan Shi, Munawar Hayat, Yicheng Wu, Jianfei Cai

Firstly, we analyze CLIP for unsupervised open-category proposal generation and design an objectness score based on our empirical analysis on proposal selection.

Object object-detection +2

A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation

1 code implementation26 Nov 2021 Himashi Peiris, Munawar Hayat, Zhaolin Chen, Gary Egan, Mehrtash Harandi

We propose a Transformer architecture for volumetric segmentation, a challenging task that requires keeping a complex balance in encoding local and global spatial cues, and preserving information along all axes of the volume.

Brain Tumor Segmentation Segmentation +2

Restormer: Efficient Transformer for High-Resolution Image Restoration

11 code implementations CVPR 2022 Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang

Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks.

Color Image Denoising Deblurring +7

MTGLS: Multi-Task Gaze Estimation with Limited Supervision

no code implementations23 Oct 2021 Shreya Ghosh, Munawar Hayat, Abhinav Dhall, Jarrod Knibbe

Our proposed framework outperforms the unsupervised state-of-the-art on CAVE (by 6. 43%) and even supervised state-of-the-art methods on Gaze360 (by 6. 59%) datasets.

Gaze Estimation

Automatic Gaze Analysis: A Survey of Deep Learning based Approaches

1 code implementation12 Aug 2021 Shreya Ghosh, Abhinav Dhall, Munawar Hayat, Jarrod Knibbe, Qiang Ji

Eye gaze analysis is an important research problem in the field of Computer Vision and Human-Computer Interaction.

Gaze Estimation

Survey: Image Mixing and Deleting for Data Augmentation

1 code implementation13 Jun 2021 Humza Naveed, Saeed Anwar, Munawar Hayat, Kashif Javed, Ajmal Mian

One such method is augmentation which introduces different types of corruption in the data to prevent the model from overfitting and to memorize patterns present in the data.

Image Augmentation Image Classification +2

Intriguing Properties of Vision Transformers

1 code implementation NeurIPS 2021 Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang

We show and analyze the following intriguing properties of ViT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content.

Few-Shot Learning Semantic Segmentation

On Generating Transferable Targeted Perturbations

3 code implementations ICCV 2021 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

To this end, we propose a new objective function that not only aligns the global distributions of source and target images, but also matches the local neighbourhood structure between the two domains.

Orthogonal Projection Loss

1 code implementation ICCV 2021 Kanchana Ranasinghe, Muzammal Naseer, Munawar Hayat, Salman Khan, Fahad Shahbaz Khan

The CE loss encourages features of a class to have a higher projection score on the true class-vector compared to the negative classes.

Domain Generalization Few-Shot Learning

Multi-Stage Progressive Image Restoration

7 code implementations CVPR 2021 Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao

At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features.

Deblurring Image Deblurring +3

Low Light Image Enhancement via Global and Local Context Modeling

no code implementations4 Jan 2021 Aditya Arora, Muhammad Haris, Syed Waqas Zamir, Munawar Hayat, Fahad Shahbaz Khan, Ling Shao, Ming-Hsuan Yang

These contexts can be crucial towards inferring several image enhancement tasks, e. g., local and global contrast, brightness and color corrections; which requires cues from both local and global spatial extent.

Low-Light Image Enhancement

Transformers in Vision: A Survey

no code implementations4 Jan 2021 Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah

Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems.

Action Recognition Colorization +10

Meta-learning the Learning Trends Shared Across Tasks

no code implementations19 Oct 2020 Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Mubarak Shah

This demonstrates their ability to acquire transferable knowledge, a capability that is central to human learning.


Synthesizing the Unseen for Zero-shot Object Detection

2 code implementations19 Oct 2020 Nasir Hayat, Munawar Hayat, Shafin Rahman, Salman Khan, Syed Waqas Zamir, Fahad Shahbaz Khan

The existing zero-shot detection approaches project visual features to the semantic domain for seen objects, hoping to map unseen objects to their corresponding semantics during inference.

Generalized Zero-Shot Object Detection Object +1

Stylized Adversarial Defense

1 code implementation29 Jul 2020 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

In contrast to existing adversarial training methods that only use class-boundary information (e. g., using a cross-entropy loss), we propose to exploit additional information from the feature space to craft stronger adversaries that are in turn used to learn a robust model.

Adversarial Defense

Self-supervised Knowledge Distillation for Few-shot Learning

1 code implementation17 Jun 2020 Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Mubarak Shah

Our experiments show that, even in the first stage, self-supervision can outperform current state-of-the-art methods, with further gains achieved by our second stage distillation process.

Few-Shot Image Classification Few-Shot Learning +2

A Self-supervised Approach for Adversarial Robustness

2 code implementations CVPR 2020 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems e. g., for classification, segmentation and object detection.

Adversarial Robustness General Classification +3

iTAML: An Incremental Task-Agnostic Meta-learning Approach

1 code implementation CVPR 2020 Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Mubarak Shah

In this paper, we hypothesize this problem can be avoided by learning a set of generalized parameters, that are neither specific to old nor new tasks.

Incremental Learning Meta-Learning

CycleISP: Real Image Restoration via Improved Data Synthesis

8 code implementations CVPR 2020 Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao

This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline.

Ranked #9 on Image Denoising on DND (using extra training data)

Image Denoising Image Restoration

Learning Enriched Features for Real Image Restoration and Enhancement

12 code implementations ECCV 2020 Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao

With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing.

Image Denoising Image Enhancement +2

Towards Robust and Reproducible Active Learning Using Neural Networks

2 code implementations CVPR 2022 Prateek Munjal, Nasir Hayat, Munawar Hayat, Jamshid Sourati, Shadab Khan

Finally, we conclude with a set of recommendations on how to assess the results using a new AL algorithm to ensure results are reproducible and robust under changes in experimental conditions.

Active Learning Classification +1

Random Path Selection for Continual Learning

1 code implementation NeurIPS 2019 Jathushan Rajasegaran, Munawar Hayat, Salman H. Khan, Fahad Shahbaz Khan, Ling Shao

In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity.

Continual Learning Incremental Learning +1

Unsupervised Primitive Discovery for Improved 3D Generative Modeling

no code implementations CVPR 2019 Salman H. Khan, Yulan Guo, Munawar Hayat, Nick Barnes

Using the primitive parts for shapes as attributes, a parameterized 3D representation is modeled in the first stage.

3D Shape Generation

An Adaptive Random Path Selection Approach for Incremental Learning

1 code implementation3 Jun 2019 Jathushan Rajasegaran, Munawar Hayat, Salman Khan, Fahad Shahbaz Khan, Ling Shao, Ming-Hsuan Yang

In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage.

Incremental Learning Knowledge Distillation +1

Learned 3D Shape Representations Using Fused Geometrically Augmented Images: Application to Facial Expression and Action Unit Detection

no code implementations8 Apr 2019 Bilal Taha, Munawar Hayat, Stefano Berretti, Naoufel Werghi

Our approach defines an inverse mapping between different geometric descriptors computed on the mesh surface or its down-sampled version, and the corresponding 2D texture image of the mesh, allowing the construction of fused geometrically augmented images (FGAI).

Action Unit Detection Facial Action Unit Detection +1

Adversarial Defense by Restricting the Hidden Space of Deep Neural Networks

1 code implementation ICCV 2019 Aamir Mustafa, Salman Khan, Munawar Hayat, Roland Goecke, Jianbing Shen, Ling Shao

Deep neural networks are vulnerable to adversarial attacks, which can fool them by adding minuscule perturbations to the input images.

Adversarial Defense

Striking the Right Balance with Uncertainty

no code implementations CVPR 2019 Salman Khan, Munawar Hayat, Waqas Zamir, Jianbing Shen, Ling Shao

Rare classes tend to get a concentrated representation in the classification space which hampers the generalization of learned boundaries to new test examples.

Attribute Classification +4

Image Super-Resolution as a Defense Against Adversarial Attacks

1 code implementation7 Jan 2019 Aamir Mustafa, Salman H. Khan, Munawar Hayat, Jianbing Shen, Ling Shao

The proposed scheme is simple and has the following advantages: (1) it does not require any model training or parameter optimization, (2) it complements other existing defense mechanisms, (3) it is agnostic to the attacked model and attack type and (4) it provides superior performance across all popular attack algorithms.

Adversarial Defense Image Enhancement +2

Adversarial Training of Variational Auto-encoders for High Fidelity Image Generation

1 code implementation27 Apr 2018 Salman H. Khan, Munawar Hayat, Nick Barnes

Our model simultaneously learns to match the data, reconstruction loss and the latent distributions of real and fake images to improve the quality of generated samples.

Image Generation Vocal Bursts Intensity Prediction

Regularization of Deep Neural Networks with Spectral Dropout

no code implementations23 Nov 2017 Salman Khan, Munawar Hayat, Fatih Porikli

We cast the proposed approach in the form of regular Convolutional Neural Network (CNN) weight layers using a decorrelation transform with fixed basis functions.

Scene Categorization With Spectral Features

no code implementations ICCV 2017 Salman H. Khan, Munawar Hayat, Fatih Porikli

To the best of our knowledge, this is the first attempt to use deep learning based spectral features explicitly for image classification task.

Attribute Image Classification

Contractive Rectifier Networks for Nonlinear Maximum Margin Classification

no code implementations ICCV 2015 Senjian An, Munawar Hayat, Salman H. Khan, Mohammed Bennamoun, Farid Boussaid, Ferdous Sohel

The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer.

Classification General Classification

A Spatial Layout and Scale Invariant Feature Representation for Indoor Scene Classification

no code implementations18 Jun 2015 Munawar Hayat, Salman H. Khan, Mohammed Bennamoun, Senjian An

This paper introduces a new learnable feature descriptor called "spatial layout and scale invariant convolutional activations" to deal with these challenges.

General Classification Scene Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.