Search Results for author: Fatih Porikli

Found 137 papers, 31 papers with code

CLNet: A Compact Latent Network for Fast Adjusting Siamese Trackers

1 code implementation ECCV 2020 Xingping Dong, Jianbing Shen, Ling Shao, Fatih Porikli

To make full use of these sequence-specific samples, {we propose a compact latent network to quickly adjust the tracking model to adapt to new scenes.}

PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer

no code implementations16 Jul 2024 Pierre-David Letourneau, Manish Kumar Singh, Hsin-Pai Cheng, Shizhong Han, Yunxiao Shi, Dalton Jones, Matthew Harper Langston, Hong Cai, Fatih Porikli

We present Polynomial Attention Drop-in Replacement (PADRe), a novel and unifying framework designed to replace the conventional self-attention mechanism in transformer models.

Computational Efficiency Image Classification +2

ToSA: Token Selective Attention for Efficient Vision Transformers

no code implementations13 Jun 2024 Manish Kumar Singh, Rajeev Yasarla, Hong Cai, Mingu Lee, Fatih Porikli

In this way, we reduce the quadratic computation and memory costs as fewer tokens participate in self-attention while maintaining the features for all the image patches throughout the network, which allows it to be used for dense prediction tasks.

Depth Prediction Monocular Depth Estimation

FouRA: Fourier Low Rank Adaptation

no code implementations13 Jun 2024 Shubhankar Borse, Shreya Kadambi, Nilesh Prasad Pandey, Kartikeya Bhardwaj, Viswanath Ganapathy, Sweta Priyadarshi, Risheek Garrepalli, Rafael Esteves, Munawar Hayat, Fatih Porikli

While Low-Rank Adaptation (LoRA) has proven beneficial for efficiently fine-tuning large models, LoRA fine-tuned text-to-image diffusion models lack diversity in the generated images, as the model tends to copy data from the observed training samples.

Diversity

EdgeRelight360: Text-Conditioned 360-Degree HDR Image Generation for Real-Time On-Device Video Portrait Relighting

no code implementations15 Apr 2024 Min-Hui Lin, Mahesh Reddy, Guillaume Berger, Michel Sarkis, Fatih Porikli, Ning Bi

In this paper, we present EdgeRelight360, an approach for real-time video portrait relighting on mobile devices, utilizing text-conditioned generation of 360-degree high dynamic range image (HDRI) maps.

Image Generation

FutureDepth: Learning to Predict the Future Improves Video Depth Estimation

no code implementations19 Mar 2024 Rajeev Yasarla, Manish Kumar Singh, Hong Cai, Yunxiao Shi, Jisoo Jeong, Yinhao Zhu, Shizhong Han, Risheek Garrepalli, Fatih Porikli

In this paper, we propose a novel video depth estimation approach, FutureDepth, which enables the model to implicitly leverage multi-frame and motion cues to improve depth estimation by making it learn to predict the future at training.

Decoder Future prediction +1

DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions

no code implementations CVPR 2024 Yunxiao Shi, Manish Kumar Singh, Hong Cai, Fatih Porikli

Leveraging the initial depths and features from this network, we uplift the 2D features to form a 3D point cloud and construct a 3D point transformer to process it, allowing the model to explicitly learn and exploit 3D geometric features.

Depth Completion

PosSAM: Panoptic Open-vocabulary Segment Anything

1 code implementation14 Mar 2024 Vibashan VS, Shubhankar Borse, Hyojin Park, Debasmit Das, Vishal Patel, Munawar Hayat, Fatih Porikli

In this paper, we introduce an open-vocabulary panoptic segmentation model that effectively unifies the strengths of the Segment Anything Model (SAM) with the vision-language CLIP model in an end-to-end framework.

Decoder Open Vocabulary Panoptic Segmentation +3

Neural Mesh Fusion: Unsupervised 3D Planar Surface Understanding

no code implementations26 Feb 2024 Farhad G. Zanjani, Hong Cai, Yinhao Zhu, Leyla Mirvakhabova, Fatih Porikli

This paper presents Neural Mesh Fusion (NMF), an efficient approach for joint optimization of polygon mesh from multi-view image observations and unsupervised 3D planar-surface parsing of the scene.

Neural Rendering

HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation

no code implementations15 Jan 2024 Antoine Mercier, Ramin Nakhli, Mahesh Reddy, Rajeev Yasarla, Hong Cai, Fatih Porikli, Guillaume Berger

Despite the latest remarkable advances in generative modeling, efficient generation of high-quality 3D assets from textual prompts remains a difficult task.

3D Generation Text to 3D

Object-Centric Diffusion for Efficient Video Editing

no code implementations11 Jan 2024 Kumara Kahatapitiya, Adil Karjauv, Davide Abati, Fatih Porikli, Yuki M. Asano, Amirhossein Habibian

By extensive experiments on a wide range of architectures, including the most efficient ones, we demonstrate that delta distillation sets a new state of the art in terms of accuracy vs. efficiency trade-off for semantic segmentation and object detection in videos.

Knowledge Distillation Object +5

Efficient neural supersampling on a novel gaming dataset

no code implementations ICCV 2023 Antoine Mercier, Ruan Erasmus, Yashesh Savani, Manik Dhingra, Fatih Porikli, Guillaume Berger

Real-time rendering for video games has become increasingly challenging due to the need for higher resolutions, framerates and photorealism.

Super-Resolution

DIFT: Dynamic Iterative Field Transforms for Memory Efficient Optical Flow

no code implementations9 Jun 2023 Risheek Garrepalli, Jisoo Jeong, Rajeswaran C Ravindran, Jamie Menjay Lin, Fatih Porikli

Also, we present a novel dynamic coarse-to-fine cost volume processing during various stages of refinement to avoid multiple levels of cost volumes.

Optical Flow Estimation

X-Align++: cross-modal cross-view alignment for Bird's-eye-view segmentation

no code implementations6 Jun 2023 Shubhankar Borse, Senthil Yogamani, Marvin Klingner, Varun Ravi, Hong Cai, Abdulaziz Almuzairee, Fatih Porikli

Bird's-eye-view (BEV) grid is a typical representation of the perception of road components, e. g., drivable area, in autonomous driving.

Autonomous Driving Segmentation

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding

1 code implementation NeurIPS 2023 Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su

Due to their alignment with CLIP embeddings, our learned shape representations can also be integrated with off-the-shelf CLIP-based models for various applications, such as point cloud captioning and point cloud-conditioned image generation.

3D Classification 3D Shape Representation +5

A Review of Deep Learning for Video Captioning

no code implementations22 Apr 2023 Moloud Abdar, Meenakshi Kollati, Swaraja Kuraparthi, Farhad Pourpanah, Daniel McDuff, Mohammad Ghavamzadeh, Shuicheng Yan, Abduallah Mohamed, Abbas Khosravi, Erik Cambria, Fatih Porikli

Video captioning (VC) is a fast-moving, cross-disciplinary area of research that bridges work in the fields of computer vision, natural language processing (NLP), linguistics, and human-computer interaction.

Dense Video Captioning Question Answering +3

DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling

no code implementations CVPR 2023 Jisoo Jeong, Hong Cai, Risheek Garrepalli, Fatih Porikli

We propose a novel data augmentation approach, DistractFlow, for training optical flow estimation models by introducing realistic distractions to the input frames.

Data Augmentation Optical Flow Estimation

X$^3$KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection

no code implementations3 Mar 2023 Marvin Klingner, Shubhankar Borse, Varun Ravi Kumar, Behnaz Rezaei, Venkatraman Narayanan, Senthil Yogamani, Fatih Porikli

Specifically, we propose cross-task distillation from an instance segmentation teacher (X-IS) in the PV feature extraction stage providing supervision without ambiguous error backpropagation through the view transformation.

3D Object Detection Instance Segmentation +3

DejaVu: Conditional Regenerative Learning to Enhance Dense Prediction

no code implementations CVPR 2023 Shubhankar Borse, Debasmit Das, Hyojin Park, Hong Cai, Risheek Garrepalli, Fatih Porikli

Next, we use a conditional regenerator, which takes the redacted image and the dense predictions as inputs, and reconstructs the original image by filling in the missing structural information.

Depth Estimation

TransAdapt: A Transformative Framework for Online Test Time Adaptive Semantic Segmentation

no code implementations24 Feb 2023 Debasmit Das, Shubhankar Borse, Hyojin Park, Kambiz Azarian, Hong Cai, Risheek Garrepalli, Fatih Porikli

Test-time adaptive (TTA) semantic segmentation adapts a source pre-trained image semantic segmentation model to unlabeled batches of target domain test images, different from real-world, where samples arrive one-by-one in an online fashion.

Segmentation Semantic Segmentation +1

Adaptive Siamese Tracking with a Compact Latent Network

no code implementations2 Feb 2023 Xingping Dong, Jianbing Shen, Fatih Porikli, Jiebo Luo, Ling Shao

Under this viewing, we perform an in-depth analysis for them through visual simulations and real tracking examples, and find that the failure cases in some challenging situations can be regarded as the issue of missing decisive samples in offline training.

X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection

no code implementations CVPR 2023 Marvin Klingner, Shubhankar Borse, Varun Ravi Kumar, Behnaz Rezaei, Venkatraman Narayanan, Senthil Yogamani, Fatih Porikli

Specifically, we propose cross-task distillation from an instance segmentation teacher (X-IS) in the PV feature extraction stage providing supervision without ambiguous error backpropagation through the view transformation.

3D Object Detection Instance Segmentation +3

Guidance Through Surrogate: Towards a Generic Diagnostic Attack

no code implementations30 Dec 2022 Muzammal Naseer, Salman Khan, Fatih Porikli, Fahad Shahbaz Khan

Recently, different adversarial training defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well studied adversarial attacks such as PGD.

Adversarial Robustness

X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentation

no code implementations13 Oct 2022 Shubhankar Borse, Marvin Klingner, Varun Ravi Kumar, Hong Cai, Abdulaziz Almuzairee, Senthil Yogamani, Fatih Porikli

Bird's-eye-view (BEV) grid is a common representation for the perception of road components, e. g., drivable area, in autonomous driving.

Autonomous Driving Segmentation

Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild

1 code implementation13 Oct 2022 Kaifeng Zhang, Yang Fu, Shubhankar Borse, Hong Cai, Fatih Porikli, Xiaolong Wang

While 6D object pose estimation has wide applications across computer vision and robotics, it remains far from being solved due to the lack of annotations.

6D Pose Estimation 6D Pose Estimation using RGB +2

Online Adaptive Personalization for Face Anti-spoofing

1 code implementation4 Jul 2022 Davide Belli, Debasmit Das, Bence Major, Fatih Porikli

Face authentication systems require a robust anti-spoofing module as they can be deceived by fabricating spoof images of authorized users.

Face Anti-Spoofing

Learning Implicit Feature Alignment Function for Semantic Segmentation

1 code implementation17 Jun 2022 Hanzhe Hu, Yinbo Chen, Jiarui Xu, Shubhankar Borse, Hong Cai, Fatih Porikli, Xiaolong Wang

As such, IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions.

Segmentation Semantic Segmentation

IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes

no code implementations CVPR 2022 Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, Manmohan Chandraker

Indoor scenes exhibit significant appearance variations due to myriad interactions between arbitrarily diverse object shapes, spatially-changing materials, and complex lighting.

Inverse Rendering

Simple and Efficient Architectures for Semantic Segmentation

1 code implementation16 Jun 2022 Dushyant Mehta, Andrii Skliar, Haitam Ben Yahia, Shubhankar Borse, Fatih Porikli, Amirhossein Habibian, Tijmen Blankevoort

Though the state-of-the architectures for semantic segmentation, such as HRNet, demonstrate impressive accuracy, the complexity arising from their salient design choices hinders a range of model acceleration tools, and further they make use of operations that are inefficient on current hardware.

Decoder Image Classification +2

Imposing Consistency for Optical Flow Estimation

no code implementations CVPR 2022 Jisoo Jeong, Jamie Menjay Lin, Fatih Porikli, Nojun Kwak

Imposing consistency through proxy tasks has been shown to enhance data-driven learning and enable self-supervision in various tasks.

Optical Flow Estimation Self-Supervised Learning

SALISA: Saliency-based Input Sampling for Efficient Video Object Detection

no code implementations5 Apr 2022 Babak Ehteshami Bejnordi, Amirhossein Habibian, Fatih Porikli, Amir Ghodrati

In this paper, we propose SALISA, a novel non-uniform SALiency-based Input SAmpling technique for video object detection that allows for heavy down-sampling of unimportant background regions while preserving the fine-grained details of a high-resolution image.

Object object-detection +1

Delta Distillation for Efficient Video Processing

1 code implementation17 Mar 2022 Amirhossein Habibian, Haitam Ben Yahia, Davide Abati, Efstratios Gavves, Fatih Porikli

By extensive experiments on a wide range of architectures, including the most efficient ones, we demonstrate that delta distillation sets a new state of the art in terms of accuracy vs. efficiency trade-off for semantic segmentation and object detection in videos.

Knowledge Distillation object-detection +4

Consistency and Diversity induced Human Motion Segmentation

no code implementations10 Feb 2022 Tao Zhou, Huazhu Fu, Chen Gong, Ling Shao, Fatih Porikli, Haibin Ling, Jianbing Shen

Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance.

Diversity Motion Segmentation +2

A personalized benchmark for face anti-spoofing

1 code implementation WACV 2022 Davide Belli, Debasmit Das, Bence Major, Fatih Porikli

In real-world scenarios, however, face authentication systems often have an initial enrollment step where a few live images of the user are recorded and stored for identification purposes.

Face Anti-Spoofing

Modality-Agnostic Topology Aware Localization

no code implementations NeurIPS 2021 Farhad Ghazvinian Zanjani, Ilia Karmanov, Hanno Ackermann, Daniel Dijkman, Simone Merlin, Max Welling, Fatih Porikli

This work presents a data-driven approach for the indoor localization of an observer on a 2D topological map of the environment.

Indoor Localization

Distribution Estimation to Automate Transformation Policies for Self-Supervision

no code implementations24 Nov 2021 Seunghan Yang, Debasmit Das, Simyung Chang, Sungrack Yun, Fatih Porikli

However, it is observed that image transformations already present in the dataset might be less effective in learning such self-supervised representations.

Generative Adversarial Network Self-Supervised Learning

Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation

no code implementations11 Nov 2021 John Yang, Yash Bhalgat, Simyung Chang, Fatih Porikli, Nojun Kwak

While hand pose estimation is a critical component of most interactive extended reality and gesture recognition systems, contemporary approaches are not optimized for computational and memory efficiency.

3D Hand Pose Estimation Gesture Recognition

HS3: Learning with Proper Task Complexity in Hierarchically Supervised Semantic Segmentation

no code implementations3 Nov 2021 Shubhankar Borse, Hong Cai, Yizhe Zhang, Fatih Porikli

While deeply supervised networks are common in recent literature, they typically impose the same learning objective on all transitional layers despite their varying representation powers.

Ranked #4 on Semantic Segmentation on Cityscapes test (using extra training data)

Segmentation Semantic Segmentation

X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation

no code implementations24 Oct 2021 Hong Cai, Janarbek Matai, Shubhankar Borse, Yizhe Zhang, Amin Ansari, Fatih Porikli

In order to enable such knowledge distillation across two different visual tasks, we introduce a small, trainable network that translates the predicted depth map to a semantic segmentation map, which can then be supervised by the teacher network.

Knowledge Distillation Monocular Depth Estimation +2

Perceptual Consistency in Video Segmentation

no code implementations24 Oct 2021 Yizhe Zhang, Shubhankar Borse, Hong Cai, Ying Wang, Ning Bi, Xiaoyun Jiang, Fatih Porikli

More specifically, by measuring the perceptual consistency between the predicted segmentation and the available ground truth on a nearby frame and combining it with the segmentation confidence, we can accurately assess the classification correctness on each pixel.

Segmentation Semantic Segmentation +2

AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentation

1 code implementation24 Oct 2021 Yizhe Zhang, Shubhankar Borse, Hong Cai, Fatih Porikli

Since inconsistency mainly arises from the model's uncertainty in its output, we propose an adaptation scheme where the model learns from its own segmentation decisions as it streams a video, which allows producing more confident and temporally consistent labeling for similarly-looking pixels across frames.

Optical Flow Estimation Segmentation +4

A Survey on Deep Learning Technique for Video Segmentation

1 code implementation2 Jul 2021 Tianfei Zhou, Fatih Porikli, David Crandall, Luc van Gool, Wenguan Wang

Video segmentation -- partitioning video frames into multiple segments or objects -- plays a critical role in a broad range of practical applications, from enhancing visual effects in movie, to understanding scenes in autonomous driving, to creating virtual background in video conferencing.

Autonomous Driving Segmentation +3

On Improving Adversarial Transferability of Vision Transformers

3 code implementations ICLR 2022 Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Fahad Shahbaz Khan, Fatih Porikli

(ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT.

Adversarial Attack

Data-driven Weight Initialization with Sylvester Solvers

no code implementations2 May 2021 Debasmit Das, Yash Bhalgat, Fatih Porikli

The initialization is cast as an optimization problem where we minimize a combination of encoding and decoding losses of the input activations, which is further constrained by a user-defined latent code.

InverseForm: A Loss Function for Structured Boundary-Aware Segmentation

1 code implementation CVPR 2021 Shubhankar Borse, Ying Wang, Yizhe Zhang, Fatih Porikli

We present a novel boundary-aware loss term for semantic segmentation using an inverse-transformation network, which efficiently learns the degree of parametric transformations between estimated and target boundaries.

Ranked #5 on Semantic Segmentation on Cityscapes test (using extra training data)

Segmentation Semantic Segmentation

On Generating Transferable Targeted Perturbations

3 code implementations ICCV 2021 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

To this end, we propose a new objective function that not only aligns the global distributions of source and target images, but also matches the local neighbourhood structure between the two domains.

Cascade Weight Shedding in Deep Neural Networks: Benefits and Pitfalls for Network Pruning

no code implementations19 Mar 2021 Kambiz Azarian, Fatih Porikli

We report, for the first time, on the cascade weight shedding phenomenon in deep neural networks where in response to pruning a small percentage of a network's weights, a large percentage of the remaining is shed over a few epochs during the ensuing fine-tuning phase.

Network Pruning

Efficient Action Recognition via Dynamic Knowledge Propagation

no code implementations ICCV 2021 HanUl Kim, Mihir Jain, Jun-Tae Lee, Sungrack Yun, Fatih Porikli

Efficient action recognition has become crucial to extend the success of action recognition to many real-world applications.

Action Recognition

Stylized Adversarial Defense

1 code implementation29 Jul 2020 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

In contrast to existing adversarial training methods that only use class-boundary information (e. g., using a cross-entropy loss), we propose to exploit additional information from the feature space to craft stronger adversaries that are in turn used to learn a robust model.

Adversarial Defense

Towards Purely Unsupervised Disentanglement of Appearance and Shape for Person Images Generation

no code implementations26 Jul 2020 Hongtao Yang, Tong Zhang, Wenbing Huang, Xuming He, Fatih Porikli

To be clear, in this paper, we refer unsupervised learning as learning without task-specific human annotations, pairs or any form of weak supervision.)

Disentanglement

A Self-supervised Approach for Adversarial Robustness

2 code implementations CVPR 2020 Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Fatih Porikli

Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems e. g., for classification, segmentation and object detection.

Adversarial Robustness General Classification +3

Identity Enhanced Residual Image Denoising

1 code implementation26 Apr 2020 Saeed Anwar, Cong Phuoc Huynh, Fatih Porikli

We propose to learn a fully-convolutional network model that consists of a Chain of Identity Mapping Modules and residual on the residual architecture for image denoising.

Image Denoising

Image Segmentation Using Deep Learning: A Survey

2 code implementations15 Jan 2020 Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, Demetri Terzopoulos

Image segmentation is a key topic in image processing and computer vision with applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among many others.

Decoder Image Compression +4

Sparse Coding on Cascaded Residuals

no code implementations7 Nov 2019 Tong Zhang, Fatih Porikli

The residual at a layer is computed by the difference between the aggregated reconstructions of the previous layers and the downsampled original image at that layer.

Computational Efficiency Denoising +2

Component Attention Guided Face Super-Resolution Network: CAGFace

1 code implementation19 Oct 2019 Ratheesh Kalarot, Tao Li, Fatih Porikli

To make the best use of the underlying structure of faces, the collective information through face datasets and the intermediate estimates during the upsampling process, here we introduce a fully convolutional multi-stage neural network for 4$\times$ super-resolution for face images.

Super-Resolution

Sketch-Specific Data Augmentation for Freehand Sketch Recognition

no code implementations14 Oct 2019 Ying Zheng, Hongxun Yao, Xiaoshuai Sun, Shengping Zhang, Sicheng Zhao, Fatih Porikli

Conventional methods for this task often rely on the availability of the temporal order of sketch strokes, additional cues acquired from different modalities and supervised augmentation of sketch datasets with real images, which also limit the applicability and feasibility of these methods in real scenarios.

Data Augmentation Retrieval +2

Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation

no code implementations6 Oct 2019 Liyuan Pan, Yuchao Dai, Miaomiao Liu, Fatih Porikli, Quan Pan

Under our model, these three tasks are naturally connected and expressed as the parameter estimation of 3D scene structure and camera motion (structure and motion for the dynamic scenes).

Deblurring Scene Flow Estimation +1

Deep Ancient Roman Republican Coin Classification via Feature Fusion and Attention

1 code implementation26 Aug 2019 Hafeez Anwar, Saeed Anwar, Sebastian Zambanini, Fatih Porikli

We perform the classification of ancient Roman Republican coins via recognizing their reverse motifs where various objects, faces, scenes, animals, and buildings are minted along with legends.

Classification General Classification

Cross-Domain Transferability of Adversarial Perturbations

2 code implementations NeurIPS 2019 Muzammal Naseer, Salman H. Khan, Harris Khan, Fahad Shahbaz Khan, Fatih Porikli

To this end, we propose a framework capable of launching highly transferable attacks that crafts adversarial patterns to mislead networks trained on wholly different domains.

Identity-preserving Face Recovery from Stylized Portraits

no code implementations7 Apr 2019 Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz

We develop an Identity-preserving Face Recovery from Portraits (IFRP) method that utilizes a Style Removal network (SRN) and a Discriminative Network (DN).

Recovering Faces from Portraits with Auxiliary Facial Attributes

no code implementations7 Apr 2019 Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz

%Our method can recover high-quality photorealistic faces from unaligned portraits while preserving the identity of the face images as well as it can reconstruct a photorealistic face image with a desired set of attributes.

Attribute

Task-generalizable Adversarial Attack based on Perceptual Metric

1 code implementation22 Nov 2018 Muzammal Naseer, Salman H. Khan, Shafin Rahman, Fatih Porikli

Deep neural networks (DNNs) can be easily fooled by adding human imperceptible perturbations to the images.

Adversarial Attack object-detection +2

Face Super-resolution Guided by Facial Component Heatmaps

no code implementations ECCV 2018 Xin Yu, Basura Fernando, Bernard Ghanem, Fatih Porikli, Richard Hartley

State-of-the-art face super-resolution methods use deep convolutional neural networks to learn a mapping between low-resolution (LR) facial patterns and their corresponding high-resolution (HR) counterparts by exploring local information.

Face Hallucination Hallucination +1

Deep Underwater Image Enhancement

2 code implementations10 Jul 2018 Saeed Anwar, Chongyi Li, Fatih Porikli

In an underwater scene, wavelength-dependent light absorption and scattering degrade the visibility of images, causing low contrast and distorted color casts.

Image Enhancement

Local Gradients Smoothing: Defense against localized adversarial attacks

5 code implementations3 Jul 2018 Muzammal Naseer, Salman H. Khan, Fatih Porikli

Deep neural networks (DNNs) have shown vulnerability to adversarial attacks, i. e., carefully perturbed inputs designed to mislead the network at inference time.

Adversarial Attack

A Deeper Look at Power Normalizations

no code implementations CVPR 2018 Piotr Koniusz, Hongguang Zhang, Fatih Porikli

In this paper, we reconsider these operators in the deep learning setup by introducing a novel layer that implements PN for non-linear pooling of feature maps.

Material Classification Scene Recognition

Super-Resolving Very Low-Resolution Face Images With Supplementary Attributes

no code implementations CVPR 2018 Xin Yu, Basura Fernando, Richard Hartley, Fatih Porikli

An LR input contains low-frequency facial components of its HR version while its residual face image defined as the difference between the HR ground-truth and interpolated LR images contains the missing high-frequency facial details.

Attribute Face Hallucination +2

Feature Affinity based Pseudo Labeling for Semi-supervised Person Re-identification

no code implementations16 May 2018 Guodong Ding, Shanshan Zhang, Salman Khan, Zhenmin Tang, Jian Zhang, Fatih Porikli

Our approach measures the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks.

Data Augmentation Representation Learning +1

Video Representation Learning Using Discriminative Pooling

no code implementations CVPR 2018 Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould

In an attempt to tackle this problem, we propose discriminative pooling, based on the notion that among the deep features generated on all short clips, there is at least one that characterizes the action.

Action Recognition In Videos Multiple Instance Learning +2

A Cascaded Convolutional Neural Network for Single Image Dehazing

no code implementations21 Mar 2018 Chongyi Li, Jichang Guo, Fatih Porikli, Huazhu Fu, Yanwei Pang

Different from previous learning-based methods, we propose a flexible cascaded CNN for single hazy image restoration, which considers the medium transmission and global atmospheric light jointly by two task-driven subnetworks.

Image Dehazing Image Restoration +1

Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Concepts

1 code implementation16 Mar 2018 Shafin Rahman, Salman Khan, Fatih Porikli

We hypothesize that this setting is ill-suited for real-world applications where unseen objects appear only as a part of a complex scene, warranting both the `recognition' and `localization' of an unseen category.

Clustering Novel Concepts +3

Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey

no code implementations9 Mar 2018 Muzammal Naseer, Salman H. Khan, Fatih Porikli

With the availability of low-cost and compact 2. 5/3D visual sensing devices, computer vision community is experiencing a growing interest in visual scene understanding of indoor environments.

3D Reconstruction object-detection +6

Face Destylization

no code implementations5 Feb 2018 Fatemeh Shiri, Xin Yu, Fatih Porikli, Piotr Koniusz

To enforce the destylized faces to be similar to authentic face images, we employ a discriminative network, which consists of convolutional and fully connected layers.

Style Transfer

Museum Exhibit Identification Challenge for Domain Adaptation and Beyond

no code implementations4 Feb 2018 Piotr Koniusz, Yusuf Tas, Hongguang Zhang, Mehrtash Harandi, Fatih Porikli, Rui Zhang

To achieve robust baselines, we build on a recent approach that aligns per-class scatter matrices of the source and target CNN streams [15].

Domain Adaptation Few-Shot Learning

Identity-preserving Face Recovery from Portraits

no code implementations8 Jan 2018 Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz

In this paper, we present a new Identity-preserving Face Recovery from Portraits (IFRP) to recover latent photorealistic faces from unaligned stylized portraits.

Chaining Identity Mapping Modules for Image Denoising

no code implementations8 Dec 2017 Saeed Anwar, Cong Phouc Huynh, Fatih Porikli

We propose to learn a fully-convolutional network model that consists of a Chain of Identity Mapping Modules (CIMM) for image denoising.

Image Denoising

DR-Net: Transmission Steered Single Image Dehazing Network with Weakly Supervised Refinement

no code implementations2 Dec 2017 Chongyi Li, Jichang Guo, Fatih Porikli, Chunle Guo, Huzhu Fu, Xi Li

Despite the recent progress in image dehazing, several problems remain largely unsolved such as robustness for varying scenes, the visual quality of reconstructed images, and effectiveness and flexibility for applications.

Image Dehazing Single Image Dehazing +1

Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps

no code implementations27 Nov 2017 Liyuan Pan, Yuchao Dai, Miaomiao Liu, Fatih Porikli

In this paper, we propose to tackle the problem of depth map completion by jointly exploiting the blurry color image sequences and the sparse depth map measurements, and present an energy minimization based formulation to simultaneously complete the depth maps, estimate the scene flow and deblur the color images.

Regularization of Deep Neural Networks with Spectral Dropout

no code implementations23 Nov 2017 Salman Khan, Munawar Hayat, Fatih Porikli

We cast the proposed approach in the form of regular Convolutional Neural Network (CNN) weight layers using a decorrelation transform with fixed basis functions.

Let Features Decide for Themselves: Feature Mask Network for Person Re-identification

no code implementations20 Nov 2017 Guodong Ding, Salman Khan, Zhenmin Tang, Fatih Porikli

Person re-identification aims at establishing the identity of a pedestrian from a gallery that contains images of multiple people obtained from a multi-camera system.

Person Re-Identification Retrieval

Multipartite Pooling for Deep Convolutional Neural Networks

no code implementations20 Oct 2017 Arash Shahriari, Fatih Porikli

We propose a novel pooling strategy that learns how to adaptively rank deep convolutional features for selecting more informative representations.

Scene Categorization With Spectral Features

no code implementations ICCV 2017 Salman H. Khan, Munawar Hayat, Fatih Porikli

To the best of our knowledge, this is the first attempt to use deep learning based spectral features explicitly for image classification task.

Attribute Image Classification

Deep Edge-Aware Saliency Detection

no code implementations15 Aug 2017 Jing Zhang, Yuchao Dai, Fatih Porikli, Mingyi He

There has been profound progress in visual saliency thanks to the deep learning architectures, however, there still exist three major challenges that hinder the detection performance for scenes with complex compositions, multiple salient objects, and salient objects of diverse scales.

Descriptive Saliency Detection

Joint Discriminative Bayesian Dictionary and Classifier Learning

no code implementations CVPR 2017 Naveed Akhtar, Ajmal Mian, Fatih Porikli

To further encourage discrimination in the dictionary, our model uses separate (sets of) Bernoulli distributions to represent data from different classes.

Action Recognition Temporal Action Localization

A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning

no code implementations27 Jun 2017 Shafin Rahman, Salman H. Khan, Fatih Porikli

Then, it learns how to combine these directions to obtain the principal direction for each unseen class such that the CAPD of the test image is aligned with the semantic embedding of the true class, and opposite to the other classes.

Generalized Zero-Shot Learning One-Shot Learning

Integrated Deep and Shallow Networks for Salient Object Detection

no code implementations2 Jun 2017 Jing Zhang, Bo Li, Yuchao Dai, Fatih Porikli, Mingyi He

Then the results from deep FCNN and RBD are concatenated to feed into a shallow network to map the concatenated feature maps to saliency maps.

Object object-detection +3

Quadruplet Network with One-Shot Learning for Fast Visual Object Tracking

no code implementations19 May 2017 Xingping Dong, Jianbing Shen, Dongming Wu, Kan Guo, Xiaogang Jin, Fatih Porikli

In this paper, we propose a new quadruplet deep network to examine the potential connections among the training instances, aiming to achieve a more powerful representation.

One-Shot Learning Visual Object Tracking

Simultaneous Stereo Video Deblurring and Scene Flow Estimation

no code implementations CVPR 2017 Liyuan Pan, Yuchao Dai, Miaomiao Liu, Fatih Porikli

Unlike the existing approach [31] which used a pre-computed scene flow, we propose a single framework to jointly estimate the scene flow and deblur the image, where the motion cues from scene flow estimation and blur information could reinforce each other, and produce superior results than the conventional scene flow estimation or stereo deblurring methods.

Deblurring Scene Flow Estimation

Action Representation Using Classifier Decision Boundaries

no code implementations6 Apr 2017 Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould

Applying multiple instance learning in an SVM setup, we use the parameters of this separating hyperplane as a descriptor for the video.

Action Recognition Multiple Instance Learning +1

Selective Video Object Cutout

no code implementations28 Feb 2017 Wenguan Wang, Jianbing Shen, Fatih Porikli

Conventional video segmentation approaches rely heavily on appearance models.

Computational Efficiency Object +3

Super-Trajectory for Video Segmentation

no code implementations ICCV 2017 Wenguan Wang, Jianbing Shen, Jianwen Xie, Fatih Porikli

We introduce a novel semi-supervised video segmentation approach based on an efficient video representation, called as "super-trajectory".

Clustering Segmentation +2

Ordered Pooling of Optical Flow Sequences for Action Recognition

no code implementations12 Jan 2017 Jue Wang, Anoop Cherian, Fatih Porikli

Training of Convolutional Neural Networks (CNNs) on long video sequences is computationally expensive due to the substantial memory requirements and the massive number of parameters that deep architectures demand.

Action Recognition Optical Flow Estimation +1

Learning an Invariant Hilbert Space for Domain Adaptation

no code implementations CVPR 2017 Samitha Herath, Mehrtash Harandi, Fatih Porikli

This paper introduces a learning scheme to construct a Hilbert space (i. e., a vector space along its inner product) to address both unsupervised and semi-supervised domain adaptation problems.

Domain Adaptation Riemannian optimization +1

Domain Adaptation by Mixture of Alignments of Second- or Higher-Order Scatter Tensors

no code implementations CVPR 2017 Piotr Koniusz, Yusuf Tas, Fatih Porikli

In this paper, we propose an approach to the domain adaptation, dubbed Second- or Higher-order Transfer of Knowledge (So-HoT), based on the mixture of alignments of second- or higher-order scatter statistics between the source and target domains.

Domain Adaptation

Saliency Integration: An Arbitrator Model

no code implementations4 Aug 2016 Yingyue Xu, Xiaopeng Hong, Fatih Porikli, Xin Liu, Jie Chen, Guoying Zhao

Previous offline integration methods usually face two challenges: 1. if most of the candidate saliency models misjudge the saliency on an image, the integration result will lean heavily on those inferior candidate models; 2. an unawareness of the ground truth saliency labels brings difficulty in estimating the expertise of each candidate model.

Learning deep structured network for weakly supervised change detection

no code implementations7 Jun 2016 Salman H. Khan, Xuming He, Fatih Porikli, Mohammed Bennamoun, Ferdous Sohel, Roberto Togneri

We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework.

Change Detection

Going Deeper into Action Recognition: A Survey

no code implementations16 May 2016 Samitha Herath, Mehrtash Harandi, Fatih Porikli

Understanding human actions in visual data is tied to advances in complementary research areas including object recognition, human dynamics, domain adaptation and semantic segmentation.

Action Analysis Action Recognition +6

Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals

no code implementations CVPR 2016 Gao Zhu, Fatih Porikli, Hongdong Li

Our method generates a small number of "high-quality" proposals by a novel instance-specific objectness measure and evaluates them against the object model that can be adopted from an existing tracking-by-detection approach as a core tracker.

Object valid

Pushing the Limits of Deep CNNs for Pedestrian Detection

no code implementations15 Mar 2016 Qichang Hu, Peng Wang, Chunhua Shen, Anton Van Den Hengel, Fatih Porikli

In this work, we show that by re-using the convolutional feature maps (CFMs) of a deep convolutional neural network (DCNN) model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms.

Occlusion Handling Optical Flow Estimation +1

Class-Specific Image Deblurring

no code implementations ICCV 2015 Saeed Anwar, Cong Phuoc Huynh, Fatih Porikli

In image deblurring, a fundamental problem is that the blur kernel suppresses a number of spatial frequencies that are difficult to recover reliably.

Deblurring Image Deblurring

Linearization to Nonlinear Learning for Visual Tracking

no code implementations ICCV 2015 Bo Ma, Hongwei Hu, Jianbing Shen, Yuping Zhang, Fatih Porikli

Building on the theory of globally linear approximations to nonlinear functions, we introduce an elegant method that jointly learns a nonlinear classifier and a visual dictionary for tracking objects in a semi-supervised sparse coding fashion.

Descriptive Dictionary Learning +1

Compositional Memory for Visual Question Answering

no code implementations18 Nov 2015 Aiwen Jiang, Fang Wang, Fatih Porikli, Yi Li

We then feed the episodes to a standard question answering module together with the contextual visual information and linguistic information.

Question Answering Visual Question Answering

When VLAD met Hilbert

no code implementations CVPR 2016 Mehrtash Harandi, Mathieu Salzmann, Fatih Porikli

Vectors of Locally Aggregated Descriptors (VLAD) have emerged as powerful image/video representations that compete with or even outperform state-of-the-art approaches on many challenging visual recognition tasks.

General Classification

Tracking Randomly Moving Objects on Edge Box Proposals

no code implementations29 Jul 2015 Gao Zhu, Fatih Porikli, Hongdong Li

Our method generates a small number of "high-quality" proposals by a novel instance-specific objectness measure and evaluates them against the object model that can be adopted from an existing tracking-by-detection approach as a core tracker.

Object valid

More About VLAD: A Leap From Euclidean to Riemannian Manifolds

no code implementations CVPR 2015 Masoud Faraki, Mehrtash T. Harandi, Fatih Porikli

This paper takes a step forward in image and video coding by extending the well-known Vector of Locally Aggregated Descriptors (VLAD) onto an extensive space of curved Riemannian manifolds.

Classification Face Recognition +2

Saliency-Aware Geodesic Video Object Segmentation

1 code implementation CVPR 2015 Wenguan Wang, Jianbing Shen, Fatih Porikli

Building on the observation that foreground areas are surrounded by the regions with high spatiotemporal edge values, geodesic distance provides an initial estimation for foreground and background.

Ranked #5 on Video Salient Object Detection on DAVSOD-Difficult20 (using extra training data)

Object Segmentation +3

DeepTrack: Learning Discriminative Feature Representations Online for Robust Visual Tracking

no code implementations28 Feb 2015 Hanxi Li, Yi Li, Fatih Porikli

In this work, we present an efficient and very robust tracking algorithm using a single Convolutional Neural Network (CNN) for learning effective feature representations of the target object, in a purely online manner.

Visual Tracking

Efficient Upsampling of Natural Images

no code implementations28 Feb 2015 Chinmay Hegde, Oncel Tuzel, Fatih Porikli

1) For the edge layer, we use a nonparametric approach by constructing a dictionary of patches from a given image, and synthesize edge regions in a higher-resolution version of the image.

Super-Resolving Noisy Images

no code implementations CVPR 2014 Abhishek Singh, Fatih Porikli, Narendra Ahuja

We then show that by taking a convex combination of orientation and frequency selective bands of the noisy and the denoised HR images, we can obtain a desired HR image where (i) some of the textural signal lost in the denoising step is effectively recovered in the HR domain, and (ii) additional textures can be easily synthesized by appropriately constraining the parameters of the convex combination.

Denoising Super-Resolution

Cannot find the paper you are looking for? You can Submit a new open access paper.