Search Results for author: Hengshuang Zhao

Found 51 papers, 32 papers with code

Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning

1 code implementation CVPR 2023 Xiaoyang Wu, Xin Wen, Xihui Liu, Hengshuang Zhao

As a pioneering work, PointContrast conducts unsupervised 3D representation learning via leveraging contrastive learning over raw RGB-D frames and proves its effectiveness on various downstream tasks.

Ranked #2 on Semantic Segmentation on ScanNet (using extra training data)

Contrastive Learning Data Augmentation +3

Influencer Backdoor Attack on Semantic Segmentation

no code implementations21 Mar 2023 Haoheng Lan, Jindong Gu, Philip Torr, Hengshuang Zhao

In this work, we explore backdoor attacks on segmentation models to misclassify all pixels of a victim class by injecting a specific trigger on non-victim pixels during inferences, which is dubbed Influencer Backdoor Attack (IBA).

Backdoor Attack Semantic Segmentation

Open-vocabulary Panoptic Segmentation with Embedding Modulation

no code implementations20 Mar 2023 Xi Chen, Shuang Li, Ser-Nam Lim, Antonio Torralba, Hengshuang Zhao

Open-vocabulary image segmentation is attracting increasing attention due to its critical applications in the real world.

Image Segmentation Open Vocabulary Panoptic Segmentation +1

GeoSpark: Sparking up Point Cloud Segmentation with Geometry Clue

no code implementations14 Mar 2023 Zhening Huang, Xiaoyang Wu, Hengshuang Zhao, Lei Zhu, Shujun Wang, Georgios Hadjidemetriou, Ioannis Brilakis

For feature aggregation, it improves feature modeling by allowing the network to learn from both local points and neighboring geometry partitions, resulting in an enlarged data-tailored receptive field.

Point Cloud Segmentation

PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer

no code implementations7 Feb 2023 Zitong Yu, Yuming Shen, Jingang Shi, Hengshuang Zhao, Yawen Cui, Jiehua Zhang, Philip Torr, Guoying Zhao

As key modules in PhysFormer, the temporal difference transformers first enhance the quasi-periodic rPPG features with temporal difference guided global attention, and then refine the local spatio-temporal representation against interference.

Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners

no code implementations15 Dec 2022 Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik Learned-Miller, Chuang Gan

To address the MTL challenge, we propose Mod-Squad, a new model that is Modularized into groups of experts (a 'Squad').

Multi-Task Learning

$BT^2$: Backward-compatible Training with Basis Transformation

1 code implementation8 Nov 2022 Yifei Zhou, Zilu Li, Abhinav Shrivastava, Hengshuang Zhao, Antonio Torralba, Taipeng Tian, Ser-Nam Lim

In this way, the new representation can be directly compared with the old representation, in principle avoiding the need for any backfilling.


Point Transformer V2: Grouped Vector Attention and Partition-based Pooling

2 code implementations11 Oct 2022 Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao

In this work, we analyze the limitations of the Point Transformer and propose our powerful and efficient Point Transformer V2 model with novel designs that overcome the limitations of previous work.

3D Point Cloud Classification 3D Semantic Segmentation +3

SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness

no code implementations25 Jul 2022 Jindong Gu, Hengshuang Zhao, Volker Tresp, Philip Torr

Since SegPGD can create more effective adversarial examples, the adversarial training with our SegPGD can boost the robustness of segmentation models.

Adversarial Attack Semantic Segmentation

DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation

1 code implementation20 Jul 2022 Xin Lai, Zhuotao Tian, Xiaogang Xu, Yingcong Chen, Shu Liu, Hengshuang Zhao, LiWei Wang, Jiaya Jia

Unsupervised domain adaptation in semantic segmentation has been raised to alleviate the reliance on expensive pixel-wise annotations.

Semantic Segmentation Unsupervised Domain Adaptation

Universal Adaptive Data Augmentation

no code implementations14 Jul 2022 Xiaogang Xu, Hengshuang Zhao

Different from existing methods, UADA would adaptively update DA's parameters according to the target model's gradient information during training: given a pre-defined set of DA operations, we randomly decide types and magnitudes of DA operations for every data batch during training, and adaptively update DA's parameters along the gradient direction of the loss concerning DA's parameters.

Data Augmentation Image Classification +3

FocalClick: Towards Practical Interactive Image Segmentation

1 code implementation CVPR 2022 Xi Chen, Zhiyan Zhao, Yilei Zhang, Manni Duan, Donglian Qi, Hengshuang Zhao

To make the model work with preexisting masks, we formulate a sub-task termed Interactive Mask Correction, and propose Progressive Merge as the solution.

 Ranked #1 on Interactive Segmentation on GrabCut (using extra training data)

Image Segmentation Interactive Segmentation +1

Stratified Transformer for 3D Point Cloud Segmentation

4 code implementations CVPR 2022 Xin Lai, Jianhui Liu, Li Jiang, LiWei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, Jiaya Jia

In this paper, we propose Stratified Transformer that is able to capture long-range contexts and demonstrates strong generalization ability and high performance.

Point Cloud Segmentation Semantic Segmentation

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

1 code implementation CVPR 2022 Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, Philip H. S. Torr

Referring image segmentation is a fundamental vision-language task that aims to segment out an object referred to by a natural language expression from an image.

Image Segmentation Referring Expression +2

PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer

1 code implementation CVPR 2022 Zitong Yu, Yuming Shen, Jingang Shi, Hengshuang Zhao, Philip Torr, Guoying Zhao

Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e. g., remote healthcare and affective computing).

Adversarial Examples on Segmentation Models Can be Easy to Transfer

no code implementations22 Nov 2021 Jindong Gu, Hengshuang Zhao, Volker Tresp, Philip Torr

The high transferability achieved by our method shows that, in contrast to the observations in previous work, adversarial examples on a segmentation model can be easy to transfer to other segmentation models.

Adversarial Robustness Classification +3

Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision

1 code implementation17 Aug 2021 Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Yukang Chen, Lu Qi, LiWei Wang, Zeming Li, Jian Sun, Jiaya Jia

In particular, Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly.

Panoptic Segmentation Weakly-supervised panoptic segmentation

Open-World Entity Segmentation

2 code implementations29 Jul 2021 Lu Qi, Jason Kuen, Yi Wang, Jiuxiang Gu, Hengshuang Zhao, Zhe Lin, Philip Torr, Jiaya Jia

By removing the need of class label prediction, the models trained for such task can focus more on improving segmentation quality.

Image Manipulation Image Segmentation +1

Do Different Tracking Tasks Require Different Appearance Models?

1 code implementation NeurIPS 2021 Zhongdao Wang, Hengshuang Zhao, Ya-Li Li, Shengjin Wang, Philip H. S. Torr, Luca Bertinetto

We show how most tracking tasks can be solved within this framework, and that the same appearance model can be successfully used to obtain results that are competitive against specialised methods for most of the tasks considered.

Multi-Object Tracking Multi-Object Tracking and Segmentation +10

Dual-Cross Central Difference Network for Face Anti-Spoofing

1 code implementation4 May 2021 Zitong Yu, Yunxiao Qin, Hengshuang Zhao, Xiaobai Li, Guoying Zhao

In this paper, we propose two Cross Central Difference Convolutions (C-CDC), which exploit the difference of the center and surround sparse local features from the horizontal/vertical and diagonal directions, respectively.

Face Anti-Spoofing Face Recognition

Distilling Knowledge via Knowledge Review

5 code implementations CVPR 2021 Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia

Knowledge distillation transfers knowledge from the teacher network to the student one, with the goal of greatly improving the performance of the student network.

Instance Segmentation Knowledge Distillation +3

Bidirectional Projection Network for Cross Dimension Scene Understanding

1 code implementation CVPR 2021 WenBo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia, Tien-Tsin Wong

Via the \emph{BPM}, complementary 2D and 3D information can interact with each other in multiple architectural levels, such that advantages in these two visual domains can be combined for better scene recognition.

2D Semantic Segmentation 3D Semantic Segmentation +3

PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

2 code implementations CVPR 2021 Mutian Xu, Runyu Ding, Hengshuang Zhao, Xiaojuan Qi

The key of PAConv is to construct the convolution kernel by dynamically assembling basic weight matrices stored in Weight Bank, where the coefficients of these weight matrices are self-adaptively learned from point positions through ScoreNet.

3D Point Cloud Classification Point Cloud Classification +1

General Adversarial Defense via Pixel Level and Feature Level Distribution Alignment

no code implementations1 Jan 2021 Xiaogang Xu, Hengshuang Zhao, Philip Torr, Jiaya Jia

Specifically, compared with previous methods, we propose a more efficient pixel-level training constraint to weaken the hardness of aligning adversarial samples to clean samples, which can thus obviously enhance the robustness on adversarial samples.

Adversarial Defense Image Classification +3

Point Transformer

17 code implementations ICCV 2021 Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, Vladlen Koltun

For example, on the challenging S3DIS dataset for large-scale semantic scene segmentation, the Point Transformer attains an mIoU of 70. 4% on Area 5, outperforming the strongest prior model by 3. 3 absolute percentage points and crossing the 70% mIoU threshold for the first time.

3D Part Segmentation 3D Point Cloud Classification +6

Fully Convolutional Networks for Panoptic Segmentation

6 code implementations CVPR 2021 Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, LiWei Wang, Zeming Li, Jian Sun, Jiaya Jia

In this paper, we present a conceptually simple, strong, and efficient framework for panoptic segmentation, called Panoptic FCN.

 Ranked #1 on Panoptic Segmentation on Mapillary val (PQth metric)

Panoptic Segmentation

Generalized Few-shot Semantic Segmentation

1 code implementation CVPR 2022 Zhuotao Tian, Xin Lai, Li Jiang, Shu Liu, Michelle Shu, Hengshuang Zhao, Jiaya Jia

Then, since context is essential for semantic segmentation, we propose the Context-Aware Prototype Learning (CAPL) that significantly improves performance by 1) leveraging the co-occurrence prior knowledge from support samples, and 2) dynamically enriching contextual information to the classifier, conditioned on the content of each query image.

Generalized Few-Shot Semantic Segmentation Semantic Segmentation

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

3 code implementations4 Aug 2020 Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang, Ruiyu Li, Jiaya Jia

It consists of novel designs of (1) a training-free prior mask generation method that not only retains generalization power but also improves model performance and (2) Feature Enrichment Module (FEM) that overcomes spatial inconsistency by adaptively enriching query features with support features and prior masks.

Few-Shot Semantic Segmentation Semantic Segmentation

Exploring Self-attention for Image Recognition

1 code implementation CVPR 2020 Hengshuang Zhao, Jiaya Jia, Vladlen Koltun

Recent work has shown that self-attention can serve as a basic building block for image recognition models.

Dynamic Divide-and-Conquer Adversarial Training for Robust Semantic Segmentation

1 code implementation ICCV 2021 Xiaogang Xu, Hengshuang Zhao, Jiaya Jia

Adversarial training is promising for improving robustness of deep neural networks towards adversarial perturbations, especially on the classification task.

Semantic Segmentation

GridMask Data Augmentation

7 code implementations13 Jan 2020 Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia

Then we show limitation of existing information dropping algorithms and propose our structured method, which is simple and yet very effective.

Data Augmentation object-detection +4

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

no code implementations ICCV 2019 Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia

To incorporate point features in the edge branch, we establish a hierarchical graph framework, where the graph is initialized from a coarse layer and gradually enriched along the point decoding process.

Scene Labeling Semantic Segmentation

Region Refinement Network for Salient Object Detection

no code implementations27 Jun 2019 Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Jiaze Wang, Ruiyu Li, Xiaoyong Shen, Jiaya Jia

Albeit intensively studied, false prediction and unclear boundaries are still major issues of salient object detection.

object-detection Portrait Segmentation +4

PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing

1 code implementation CVPR 2019 Hengshuang Zhao, Li Jiang, Chi-Wing Fu, Jiaya Jia

Unlike previous work, we densely connect each point with every other in a local neighborhood, aiming to specify feature of each point based on the local region characteristics for better representing the region.

Ranked #17 on Semantic Segmentation on S3DIS Area5 (oAcc metric)

3D Point Cloud Classification General Classification +2

PSANet: Point-wise Spatial Attention Network for Scene Parsing

4 code implementations ECCV 2018 Hengshuang Zhao, Yi Zhang, Shu Liu, Jianping Shi, Chen Change Loy, Dahua Lin, Jiaya Jia

We notice information flow in convolutional neural networks is restricted inside local neighborhood regions due to the physical design of convolutional filters, which limits the overall understanding of complex scenes.

Scene Parsing Semantic Segmentation

Compositing-aware Image Search

no code implementations ECCV 2018 Hengshuang Zhao, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Brian Price, Jiaya Jia

We present a new image search technique that, given a background image, returns compatible foreground objects for image compositing tasks.

Image Retrieval

Automatic Real-time Background Cut for Portrait Videos

no code implementations28 Apr 2017 Xiaoyong Shen, RuiXing Wang, Hengshuang Zhao, Jiaya Jia

A spatial-temporal refinement network is developed to further refine the segmentation errors in each frame and ensure temporal coherence in the segmentation map.

Semantic Segmentation Video Segmentation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.