Search Results for author: Pheng-Ann Heng

Found 184 papers, 95 papers with code

Overcoming Support Dilution for Robust Few-shot Semantic Segmentation

no code implementations23 Jan 2025 Wailing Tang, Biqi Yang, Pheng-Ann Heng, Yun-hui Liu, Chi-Wing Fu

Few-shot Semantic Segmentation (FSS) is a challenging task that utilizes limited support images to segment associated unseen objects in query images.

Few-Shot Semantic Segmentation Segmentation +1

Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step

1 code implementation23 Jan 2025 Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, Pheng-Ann Heng

We hope our study provides unique insights and paves a new path for integrating CoT reasoning with autoregressive image generation.

Image Generation

The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?

no code implementations20 Jan 2025 Yiyi Zhang, Xingyu Chen, Kexin Chen, Yuyang Du, Xilin Dang, Pheng-Ann Heng

By incorporating an innovative balanced seed in the data generation process, our framework systematically considers both legitimate and illegitimate requests.

Data Augmentation Question Answering +1

Creating Virtual Environments with 3D Gaussian Splatting: A Comparative Study

no code implementations16 Jan 2025 Shi Qiu, Binzhu Xie, Qixuan Liu, Pheng-Ann Heng

3D Gaussian Splatting (3DGS) has recently emerged as an innovative and efficient 3D representation technique.

3DGS

A Survey on Inference Optimization Techniques for Mixture of Experts Models

1 code implementation18 Dec 2024 Jiacheng Liu, Peng Tang, Wenfeng Wang, Yuhang Ren, Xiaofeng Hou, Pheng-Ann Heng, Minyi Guo, Chao Li

This survey provides both a structured overview of existing solutions and identifies key challenges and promising research directions in MoE inference optimization.

Computational Efficiency Distributed Computing +5

MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments

no code implementations12 Dec 2024 Yuqi Tong, Yue Qiu, Ruiyang Li, Shi Qiu, Pheng-Ann Heng

We present MS2Mesh-XR, a novel multi-modal sketch-to-mesh generation pipeline that enables users to create realistic 3D objects in extended reality (XR) environments using hand-drawn sketches assisted by voice inputs.

Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects

no code implementations9 Dec 2024 Shi Qiu, Binzhu Xie, Qixuan Liu, Pheng-Ann Heng

3D Gaussian Splatting (3DGS) has attracted significant attention for its potential to revolutionize 3D representation, rendering, and interaction.

3DGS

Point Cloud Understanding via Attention-Driven Contrastive Learning

no code implementations22 Nov 2024 Yi Wang, Jiaze Wang, Ziyu Guo, Renrui Zhang, Donghao Zhou, Guangyong Chen, Anfeng Liu, Pheng-Ann Heng

Recently Transformer-based models have advanced point cloud understanding by leveraging self-attention mechanisms, however, these methods often overlook latent information in less prominent regions, leading to increased sensitivity to perturbations and limited global comprehension.

Contrastive Learning Few-Shot Learning

HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference

no code implementations3 Nov 2024 Peng Tang, Jiacheng Liu, Xiaofeng Hou, YiFei PU, Jing Wang, Pheng-Ann Heng, Chao Li, Minyi Guo

We present HOBBIT, a mixed precision expert offloading system to enable flexible and efficient MoE inference.

DEL-Ranking: Ranking-Correction Denoising Framework for Elucidating Molecular Affinities in DNA-Encoded Libraries

no code implementations19 Oct 2024 Hanqun Cao, Mutian He, Ning Ma, Chang-Yu Hsieh, Chunbin Gu, Pheng-Ann Heng

DNA-encoded library (DEL) screening has revolutionized the detection of protein-ligand interactions through read counts, enabling rapid exploration of vast chemical spaces.

Denoising Zero-shot Generalization

MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models

no code implementations17 Oct 2024 Donghao Zhou, Jiancheng Huang, Jinbin Bai, Jiaze Wang, Hao Chen, Guangyong Chen, Xiaowei Hu, Pheng-Ann Heng

Recent text-to-image models generate high-quality images from text prompts but lack precise control over specific components within visual concepts.

Image Generation

PCF-Lift: Panoptic Lifting by Probabilistic Contrastive Fusion

1 code implementation14 Oct 2024 Runsong Zhu, Shi Qiu, Qianyi Wu, Ka-Hei Hui, Pheng-Ann Heng, Chi-Wing Fu

Panoptic lifting is an effective technique to address the 3D panoptic segmentation task by unprojecting 2D panoptic segmentations from multi-views to 3D scene.

3D Panoptic Segmentation Panoptic Segmentation +1

Neural P$^3$M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs

no code implementations26 Sep 2024 Yusong Wang, Chaoran Cheng, Shaoning Li, Yuxuan Ren, Bin Shao, Ge Liu, Pheng-Ann Heng, Nanning Zheng

Geometric graph neural networks (GNNs) have emerged as powerful tools for modeling molecular geometry.

Adapting Segment Anything Model for Unseen Object Instance Segmentation

no code implementations23 Sep 2024 Rui Cao, Chuanxin Song, Biqi Yang, Jiangliu Wang, Pheng-Ann Heng, Yun-hui Liu

Unseen Object Instance Segmentation (UOIS) is crucial for autonomous robots operating in unstructured environments.

Decoder Segmentation +2

Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models

1 code implementation3 Sep 2024 Jiaqi Xu, Mengyang Wu, Xiaowei Hu, Chi-Wing Fu, Qi Dou, Pheng-Ann Heng

For clearness enhancement, we use real-world data, utilizing a dual-step strategy with pseudo-labels assessed by vision-language models and weather prompt learning.

Image Restoration Language Modeling +1

SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners

1 code implementation29 Aug 2024 Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Chengzhuo Tong, Peng Gao, Chunyuan Li, Pheng-Ann Heng

We introduce SAM2Point, a preliminary exploration adapting Segment Anything Model 2 (SAM 2) for zero-shot and promptable 3D segmentation.

Segmentation

Surgical Workflow Recognition and Blocking Effectiveness Detection in Laparoscopic Liver Resections with Pringle Maneuver

1 code implementation20 Aug 2024 Diandian Guo, Weixin Si, Zhixi Li, Jialun Pei, Pheng-Ann Heng

Pringle maneuver (PM) in laparoscopic liver resection aims to reduce blood loss and provide a clear surgical view by intermittently blocking blood inflow of the liver, whereas prolonged PM may cause ischemic injury.

Blocking

G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors

1 code implementation18 Aug 2024 Haoxin Yang, Xuemiao Xu, Cheng Xu, Huaidong Zhang, Jing Qin, Yi Wang, Pheng-Ann Heng, Shengfeng He

This paper introduces G\textsuperscript{2}Face, which leverages both generative and geometric priors to enhance identity manipulation, achieving high-quality reversible face anonymization without compromising data utility.

Decoder Face Anonymization +1

Decoupling Feature Representations of Ego and Other Modalities for Incomplete Multi-modal Brain Tumor Segmentation

1 code implementation16 Aug 2024 Kaixiang Yang, Wenqi Shan, Xudong Li, Xuan Wang, Xikai Yang, Xi Wang, Pheng-Ann Heng, Qiang Li, Zhiwei Wang

Multi-modal brain tumor segmentation typically involves four magnetic resonance imaging (MRI) modalities, while incomplete modalities significantly degrade performance.

Brain Tumor Segmentation Tumor Segmentation

Improving AlphaFlow for Efficient Protein Ensembles Generation

no code implementations8 Jul 2024 Shaoning Li, Mingyu Li, Yusong Wang, Xinheng He, Nanning Zheng, Jian Zhang, Pheng-Ann Heng

Investigating conformational landscapes of proteins is a crucial way to understand their biological functions and properties.

Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation

1 code implementation7 Jul 2024 Juzheng Miao, Cheng Chen, Keli Zhang, Jie Chuai, Quanzheng Li, Pheng-Ann Heng

To harness the power of foundation models for application in SSL, we propose a cross prompting consistency method with segment anything model (CPC-SAM) for semi-supervised medical image segmentation.

Decoder Image Segmentation +3

FM-OSD: Foundation Model-Enabled One-Shot Detection of Anatomical Landmarks

1 code implementation7 Jul 2024 Juzheng Miao, Cheng Chen, Keli Zhang, Jie Chuai, Quanzheng Li, Pheng-Ann Heng

By using solely a single template image, our method demonstrates significant superiority over strong state-of-the-art one-shot landmark detection methods.

Anatomical Landmark Detection

A Narrative Review of Image Processing Techniques Related to Prostate Ultrasound

no code implementations30 Jun 2024 Haiqiao Wang, Hong Wu, Zhuoyuan Wang, Peiyan Yue, Dong Ni, Pheng-Ann Heng, Yi Wang

In consequence, this survey provides a \textcolor{blue}{narrative } analysis of this field, outlining the evolution of image processing methods in the context of TRUS image analysis and meanwhile highlighting their relevant contributions.

Image Registration Prognosis +1

Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting

1 code implementation28 Jun 2024 Wei Li, Jingyang Zhang, Pheng-Ann Heng, Lixu Gu

Generalist segmentation models are increasingly favored for diverse tasks involving various objects from different image sources.

Denoising Incremental Learning +2

Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation

1 code implementation26 Jun 2024 Dunyuan Xu, Xi Wang, Jingyang Zhang, Pheng-Ann Heng

To achieve this, we create the orientational gradient alignment to ensure memorizability on previous sites, and arbitrary gradient alignment to enhance generalizability on unseen sites.

Continual Learning Domain Generalization +3

Depth-Driven Geometric Prompt Learning for Laparoscopic Liver Landmark Detection

1 code implementation25 Jun 2024 Jialun Pei, Ruize Cui, Yaoqian Li, Weixin Si, Jing Qin, Pheng-Ann Heng

Liver anatomical landmarks, e. g., ridge and ligament, serve as important markers for 2D-3D alignment, which can significantly enhance the spatial perception of surgeons for precise surgery.

Benchmarking

Epicardium Prompt-guided Real-time Cardiac Ultrasound Frame-to-volume Registration

1 code implementation20 Jun 2024 Long Lei, Jun Zhou, Jialun Pei, Baoliang Zhao, Yueming Jin, Yuen-Chun Jeremy Teoh, Jing Qin, Pheng-Ann Heng

A comprehensive guidance view for cardiac interventional surgery can be provided by the real-time fusion of the intraoperative 2D images and preoperative 3D volume based on the ultrasound frame-to-volume registration.

Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Rooms

no code implementations14 Apr 2024 Diandian Guo, Manxi Lin, Jialun Pei, He Tang, Yueming Jin, Pheng-Ann Heng

A comprehensive understanding of surgical scenes allows for monitoring of the surgical process, reducing the occurrence of accidents and enhancing efficiency for medical professionals.

Graph Generation Scene Graph Generation

Noise Level Adaptive Diffusion Model for Robust Reconstruction of Accelerated MRI

1 code implementation8 Mar 2024 Shoujin Huang, GuanXiong Luo, Xi Wang, Ziran Chen, Yuwan Wang, Huaishui Yang, Pheng-Ann Heng, Lingyan Zhang, Mengye Lyu

In general, diffusion model-based MRI reconstruction methods incrementally remove artificially added noise while imposing data consistency to reconstruct the underlying images.

Denoising MRI Reconstruction

LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic Surgery

1 code implementation26 Feb 2024 Yuyang Du, Kexin Chen, Yue Zhan, Chang Han Low, Tao You, Mobarakol Islam, Ziyu Guo, Yueming Jin, Guangyong Chen, Pheng-Ann Heng

We further design an adaptive weight assignment approach that balances the generalization ability of the LLM and the domain expertise of the old CL model.

Continual Learning Language Modelling +4

S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR

1 code implementation22 Feb 2024 Jialun Pei, Diandian Guo, Jingyang Zhang, Manxi Lin, Yueming Jin, Pheng-Ann Heng

In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S^2Former-OR, aimed to complementally leverage multi-view 2D scenes and 3D point clouds for SGG in an end-to-end manner.

Graph Generation object-detection +3

Multi-scale Spatio-temporal Transformer-based Imbalanced Longitudinal Learning for Glaucoma Forecasting from Irregular Time Series Images

no code implementations21 Feb 2024 Xikai Yang, Jian Wu, Xi Wang, Yuchen Yuan, Ning Li Wang, Pheng-Ann Heng

Extensive experiments on the Sequential fundus Images for Glaucoma Forecast (SIGF) dataset demonstrate the superiority of the proposed MST-former method, achieving an AUC of 98. 6% for glaucoma forecasting.

Disease Prediction Irregular Time Series +1

Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning

1 code implementation4 Feb 2024 Lanqing Li, Hai Zhang, Xinyu Zhang, Shatong Zhu, Yang Yu, Junqiao Zhao, Pheng-Ann Heng

As demonstrations, we propose a supervised and a self-supervised implementation of $I(Z; M)$, and empirically show that the corresponding optimization algorithms exhibit remarkable generalization across a broad spectrum of RL benchmarks, context shift scenarios, data qualities and deep learning architectures.

Meta Reinforcement Learning Offline RL +3

SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation

1 code implementation2 Feb 2024 Yinqiao Wang, Hao Xu, Pheng-Ann Heng, Chi-Wing Fu

Estimating 3D hand mesh from RGB images is a longstanding track, in which occlusion is one of the most challenging problems.

Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading

no code implementations17 Jan 2024 Dunyuan Xu, Xi Wang, Jinyue Cai, Pheng-Ann Heng

Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly.

MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation

1 code implementation16 Sep 2023 Cheng Chen, Juzheng Miao, Dufan Wu, Zhiling Yan, Sekeun Kim, Jiang Hu, Aoxiao Zhong, Zhengliang Liu, Lichao Sun, Xiang Li, Tianming Liu, Pheng-Ann Heng, Quanzheng Li

The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks.

Image Segmentation Medical Image Segmentation +5

Unite-Divide-Unite: Joint Boosting Trunk and Structure for High-accuracy Dichotomous Image Segmentation

1 code implementation26 Jul 2023 Jialun Pei, Zhangjun Zhou, Yueming Jin, He Tang, Pheng-Ann Heng

First, a dual-size input feeds into the shared backbone to produce more holistic and detailed features while keeping the model lightweight.

Decoder Dichotomous Image Segmentation +2

CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation

1 code implementation16 Jul 2023 Jialun Pei, Tao Jiang, He Tang, Nian Liu, Yueming Jin, Deng-Ping Fan, Pheng-Ann Heng

We propose a novel approach for RGB-D salient instance segmentation using a dual-branch cross-modal feature calibration architecture called CalibNet.

Instance Segmentation Semantic Segmentation

3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation

1 code implementation23 Jun 2023 Shizhan Gong, Yuan Zhong, Wenao Ma, Jinpeng Li, Zhao Wang, Jingyang Zhang, Pheng-Ann Heng, Qi Dou

Notably, the original SAM architecture is designed for 2D natural images, therefore would not be able to extract the 3D spatial information from volumetric medical data effectively.

Image Segmentation Medical Image Segmentation +2

Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images

1 code implementation23 Jun 2023 Zhizhong Chai, Luyang Luo, Huangjing Lin, Pheng-Ann Heng, Hao Chen

To tackle this challenge, the literature on object detection has witnessed an increase of weakly-supervised and semi-supervised approaches, yet still lacks a unified framework that leverages various forms of fully-labeled, weakly-labeled, and unlabeled data.

Fracture detection object-detection

Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images

no code implementations30 May 2023 Yanwen Li, Luyang Luo, Huangjing Lin, Pheng-Ann Heng, Hao Chen

To guide the segmentation branch to learn from richer high-resolution features, we propose a feature affinity module and a scale affinity module to enhance the multi-task learning of the dual branches.

Image Segmentation Image Super-Resolution +4

Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning

1 code implementation21 Mar 2023 Yang Yu, Danruo Deng, Furui Liu, Yueming Jin, Qi Dou, Guangyong Chen, Pheng-Ann Heng

Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers).

Outlier Detection

Traj-MAE: Masked Autoencoders for Trajectory Prediction

no code implementations ICCV 2023 Hao Chen, Jiaze Wang, Kun Shao, Furui Liu, Jianye Hao, Chenyong Guan, Guangyong Chen, Pheng-Ann Heng

Specifically, our Traj-MAE employs diverse masking strategies to pre-train the trajectory encoder and map encoder, allowing for the capture of social and temporal information among agents while leveraging the effect of environment from multiple granularities.

Autonomous Driving Prediction +1

PointPatchMix: Point Cloud Mixing with Patch Scoring

no code implementations12 Mar 2023 Yi Wang, Jiaze Wang, Jinpeng Li, Zixu Zhao, Guangyong Chen, Anfeng Liu, Pheng-Ann Heng

With Point-MAE as our baseline, our model surpasses previous methods by a significant margin, achieving 86. 3% accuracy on ScanObjectNN and 94. 1% accuracy on ModelNet40.

Data Augmentation

Uncertainty Estimation by Fisher Information-based Evidential Deep Learning

1 code implementation3 Mar 2023 Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, Pheng-Ann Heng

To address this problem, we propose a novel method, Fisher Information-based Evidential Deep Learning ($\mathcal{I}$-EDL).

Deep Learning Informativeness +1

Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training

no code implementations27 Feb 2023 Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzhi Li, Pheng-Ann Heng

In this paper, we explore how the 2D modality can benefit 3D masked autoencoding, and propose Joint-MAE, a 2D-3D joint MAE framework for self-supervised 3D point cloud pre-training.

Decoder Point Cloud Pre-training +1

On the Pitfall of Mixup for Uncertainty Calibration

1 code implementation CVPR 2023 Deng-Bao Wang, Lanqing Li, Peilin Zhao, Pheng-Ann Heng, Min-Ling Zhang

It has been recently found that models trained with mixup also perform well on uncertainty calibration.

Class-Conditional Sharpness-Aware Minimization for Deep Long-Tailed Recognition

1 code implementation CVPR 2023 Zhipeng Zhou, Lanqing Li, Peilin Zhao, Pheng-Ann Heng, Wei Gong

It's widely acknowledged that deep learning models with flatter minima in its loss landscape tend to generalize better.

Long-tail Learning

RepMode: Learning to Re-parameterize Diverse Experts for Subcellular Structure Prediction

2 code implementations CVPR 2023 Donghao Zhou, Chunbin Gu, Junde Xu, Furui Liu, Qiong Wang, Guangyong Chen, Pheng-Ann Heng

In biological research, fluorescence staining is a key technique to reveal the locations and morphology of subcellular structures.

Video Instance Shadow Detection Under the Sun and Sky

1 code implementation23 Nov 2022 Zhenghao Xing, Tianyu Wang, Xiaowei Hu, Haoran Wu, Chi-Wing Fu, Pheng-Ann Heng

Instance shadow detection, crucial for applications such as photo editing and light direction estimation, has undergone significant advancements in predicting shadow instances, object instances, and their associations.

Contrastive Learning Instance Shadow Detection +3

Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening

no code implementations9 Nov 2022 Kang Li, Lequan Yu, Pheng-Ann Heng

Particularly, we first present a style-oriented replay module to enable structure-realistic and memory-efficient reproduction of past data, and then incorporate the replayed past data to jointly optimize the model with current data to alleviate catastrophic forgetting.

Image Segmentation Incremental Learning +2

Multi-Task Mixture Density Graph Neural Networks for Predicting Cu-based Single-Atom Alloy Catalysts for CO2 Reduction Reaction

no code implementations15 Sep 2022 Chen Liang, Bowen Wang, Shaogang Hao, Guangyong Chen, Pheng-Ann Heng, Xiaolong Zou

Graph neural networks (GNNs) have drawn more and more attention from material scientists and demonstrated a high capacity to establish connections between the structure and properties.

A Survey on Generative Diffusion Model

1 code implementation6 Sep 2022 Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, Stan Z. Li

Deep generative models are a prominent approach for data generation, and have been used to produce high quality samples in various domains.

Dimensionality Reduction model +1

MetaRF: Differentiable Random Forest for Reaction Yield Prediction with a Few Trails

no code implementations22 Aug 2022 Kexin Chen, Guangyong Chen, Junyou Li, Yuansheng Huang, Pheng-Ann Heng

In high-throughput experimentation (HTE) datasets, the average yield of our methodology's top 10 high-yield reactions is relatively close to the results of ideal yield selection.

Dimensionality Reduction Few-Shot Learning +1

Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations

no code implementations20 Jul 2022 Yang Yu, Zixu Zhao, Yueming Jin, Guangyong Chen, Qi Dou, Pheng-Ann Heng

Concretely, for trusty representation learning, we propose to incorporate pseudo labels to instruct the pair selection, obtaining more reliable representation pairs for pixel contrast.

Pseudo Label Representation Learning +2

Instance Shadow Detection with A Single-Stage Detector

2 code implementations11 Jul 2022 Tianyu Wang, Xiaowei Hu, Pheng-Ann Heng, Chi-Wing Fu

This paper formulates a new problem, instance shadow detection, which aims to detect shadow instance and the associated object instance that cast each shadow in the input image.

Instance Shadow Detection Object +2

ORF-Net: Deep Omni-supervised Rib Fracture Detection from Chest CT Scans

no code implementations5 Jul 2022 Zhizhong Chai, Huangjing Lin, Luyang Luo, Pheng-Ann Heng, Hao Chen

In this paper, we proposed a novel omni-supervised object detection network, which can exploit multiple different forms of annotated data to further improve the detection performance.

Fracture detection Object +1

Dynamic Bank Learning for Semi-supervised Federated Image Diagnosis with Class Imbalance

1 code implementation27 Jun 2022 Meirui Jiang, Hongzheng Yang, Xiaoxiao Li, Quande Liu, Pheng-Ann Heng, Qi Dou

Despite recent progress on semi-supervised federated learning (FL) for medical image diagnosis, the problem of imbalanced class distributions among unlabeled clients is still unsolved for real-world use.

Federated Learning

Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds

no code implementations14 Jun 2022 Runsong Zhu, Di Kang, Ka-Hei Hui, Yue Qian, Xuefei Zhe, Zhen Dong, Linchao Bao, Pheng-Ann Heng, Chi-Wing Fu

To guide the network quickly fit the coarse shape, we propose to utilize the signed supervision in regions that are obviously outside the object and can be easily determined, resulting in our semi-signed supervision.

3D geometry Surface Reconstruction

Robust Medical Image Classification from Noisy Labeled Data with Global and Local Representation Guided Co-training

no code implementations10 May 2022 Cheng Xue, Lequan Yu, Pengfei Chen, Qi Dou, Pheng-Ann Heng

In this paper, we propose a novel collaborative training paradigm with global and local representation learning for robust medical image classification from noisy-labeled data to combat the lack of high quality annotated medical data.

Image Classification Medical Image Analysis +2

Acknowledging the Unknown for Multi-label Learning with Single Positive Labels

1 code implementation30 Mar 2022 Donghao Zhou, Pengfei Chen, Qiong Wang, Guangyong Chen, Pheng-Ann Heng

Due to the difficulty of collecting exhaustive multi-label annotations, multi-label datasets often contain partial labels.

Multi-Label Learning Weakly-supervised Learning

Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation

1 code implementation29 Mar 2022 Yueming Jin, Yang Yu, Cheng Chen, Zixu Zhao, Pheng-Ann Heng, Danail Stoyanov

Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre.

Contrastive Learning Relation +1

Pseudo Bias-Balanced Learning for Debiased Chest X-ray Classification

1 code implementation18 Mar 2022 Luyang Luo, Dunyuan Xu, Hao Chen, Tien-Tsin Wong, Pheng-Ann Heng

Deep learning models were frequently reported to learn from shortcuts like dataset biases.

TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery

no code implementations17 Feb 2022 Zixu Zhao, Yueming Jin, Pheng-Ann Heng

Specifically, we introduce the prior query that encoded with previous temporal knowledge, to transfer tracking signals to current instances via identity matching.

Segmentation

Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network

1 code implementation8 Nov 2021 Jiacheng Wang, Yueming Jin, Shuntian Cai, Hongzhi Xu, Pheng-Ann Heng, Jing Qin, Liansheng Wang

Compared with existing solutions, which either neglect geometric relationships among targeting objects or capture the relationships by using complicated aggregation schemes, the proposed network is capable of achieving satisfactory accuracy while maintaining real-time performance by taking full advantage of the spatial relations among landmarks.

Multi-Task Learning Relation +1

Hepatic vessel segmentation based on 3D swin-transformer with inductive biased multi-head self-attention

no code implementations5 Nov 2021 Mian Wu, Yinling Qian, Xiangyun Liao, Qiong Wang, Pheng-Ann Heng

In practice, we introduce the voxel-wise embedding rather than patch-wise embedding to locate precise liver vessel voxels, and adopt multi-scale convolutional operators to gain local spatial information.

Medical Image Analysis Segmentation

Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning

1 code implementation NeurIPS 2021 Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, Pheng-Ann Heng

The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones.

Continual Learning

Efficient Global-Local Memory for Real-time Instrument Segmentation of Robotic Surgical Video

1 code implementation28 Sep 2021 Jiacheng Wang, Yueming Jin, Liansheng Wang, Shuntian Cai, Pheng-Ann Heng, Jing Qin

On the other hand, we develop an active global memory to gather the global semantic correlation in long temporal range to current one, in which we gather the most informative frames derived from model uncertainty and frame similarity.

Optical Flow Estimation Segmentation

Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling

1 code implementation19 Sep 2021 Cheng Chen, Quande Liu, Yueming Jin, Qi Dou, Pheng-Ann Heng

We present a novel denoised pseudo-labeling method for this problem, which effectively makes use of the source model and unlabeled target data to promote model self-adaptation from pseudo labels.

Denoising Image Segmentation +2

SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

1 code implementation30 Aug 2021 Jiaqi Xu, Bin Li, Bo Lu, Yun-hui Liu, Qi Dou, Pheng-Ann Heng

Ten learning-based surgical tasks are built in the platform, which are common in the real autonomous surgical execution.

Reinforcement Learning (RL)

Accurate Grid Keypoint Learning for Efficient Video Prediction

1 code implementation28 Jul 2021 Xiaojie Gao, Yueming Jin, Qi Dou, Chi-Wing Fu, Pheng-Ann Heng

Video prediction methods generally consume substantial computing resources in training and deployment, among which keypoint-based approaches show promising improvement in efficiency by simplifying dense image prediction to light keypoint prediction.

Prediction Video Prediction

Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching

1 code implementation16 Jun 2021 Quande Liu, Hongzheng Yang, Qi Dou, Pheng-Ann Heng

This paper studies a practical yet challenging FL problem, named \textit{Federated Semi-supervised Learning} (FSSL), which aims to learn a federated model by jointly utilizing the data from both labeled and unlabeled clients (i. e., hospitals).

Federated Learning Image Classification +2

Deep Semi-supervised Metric Learning with Dual Alignment for Cervical Cancer Cell Detection

no code implementations7 Apr 2021 Zhizhong Chai, Luyang Luo, Huangjing Lin, Hao Chen, Anjia Han, Pheng-Ann Heng

Specifically, our model learns a metric space and conducts dual alignment of semantic features on both the proposal level and the prototype levels.

Cell Detection Metric Learning +2

Temporal Memory Relation Network for Workflow Recognition from Surgical Video

1 code implementation30 Mar 2021 Yueming Jin, Yonghao Long, Cheng Chen, Zixu Zhao, Qi Dou, Pheng-Ann Heng

In this paper, we propose a novel end-to-end temporal memory relation network (TMRNet) for relating long-range and multi-scale temporal patterns to augment the present features.

Relation Relation Network

One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video

no code implementations24 Mar 2021 Zixu Zhao, Yueming Jin, Bo Lu, Chi-Fai Ng, Qi Dou, Yun-hui Liu, Pheng-Ann Heng

To greatly increase the label efficiency, we explore a new problem, i. e., adaptive instrument segmentation, which is to effectively adapt one source model to new robotic surgical videos from multiple target domains, only given the annotated instruments in the first frame.

General Knowledge Meta-Learning

Future Frame Prediction for Robot-assisted Surgery

no code implementations18 Mar 2021 Xiaojie Gao, Yueming Jin, Zixu Zhao, Qi Dou, Pheng-Ann Heng

Predicting future frames for robotic surgical video is an interesting, important yet extremely challenging problem, given that the operative tasks may have complex dynamics.

Future prediction Optical Flow Estimation +1

Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer

1 code implementation17 Mar 2021 Xiaojie Gao, Yueming Jin, Yonghao Long, Qi Dou, Pheng-Ann Heng

In this paper, we introduce, for the first time in surgical workflow analysis, Transformer to reconsider the ignored complementary effects of spatial and temporal features for accurate surgical phase recognition.

Surgical phase recognition

Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment

no code implementations6 Mar 2021 Xueying Shi, Yueming Jin, Qi Dou, Jing Qin, Pheng-Ann Heng

In this paper, we propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i. e., both kinematic and visual data, from simulator to real robot.

Gesture Recognition Surgical Gesture Recognition +1

Deep Texture-Aware Features for Camouflaged Object Detection

no code implementations5 Feb 2021 Jingjing Ren, Xiaowei Hu, Lei Zhu, Xuemiao Xu, Yangyang Xu, Weiming Wang, Zijun Deng, Pheng-Ann Heng

Camouflaged object detection is a challenging task that aims to identify objects having similar texture to the surroundings.

Object object-detection +1

Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation

1 code implementation7 Jan 2021 Kang Li, Shujun Wang, Lequan Yu, Pheng-Ann Heng

In this way, the dual teacher models would transfer acquired inter- and intra-domain knowledge to the student model for further integration and exploitation.

Cardiac Segmentation Domain Adaptation +3

C3-SemiSeg: Contrastive Semi-Supervised Segmentation via Cross-Set Learning and Dynamic Class-Balancing

no code implementations ICCV 2021 Yanning Zhou, Hang Xu, Wei zhang, Bin Gao, Pheng-Ann Heng

The semi-supervised semantic segmentation methods utilize the unlabeled data to increase the feature discriminative ability to alleviate the burden of the annotated data.

Contrastive Learning Data Augmentation +1

Noise against noise: stochastic label noise helps combat inherent label noise

no code implementations ICLR 2021 Pengfei Chen, Guangyong Chen, Junjie Ye, Jingwei Zhao, Pheng-Ann Heng

The noise in stochastic gradient descent (SGD) provides a crucial implicit regularization effect, previously studied in optimization by analyzing the dynamics of parameter updates.

Learning with noisy labels

Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise

1 code implementation10 Dec 2020 Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, Pheng-Ann Heng

In this work, we present a theoretical hypothesis testing and prove that noise in real-world dataset is unlikely to be CCN, which confirms that label noise should depend on the instance and justifies the urgent need to go beyond the CCN assumption. The theoretical results motivate us to study the more general and practical-relevant instance-dependent noise (IDN).

Image Classification

Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels

1 code implementation8 Dec 2020 Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, Pheng-Ann Heng

For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping.

Learning with noisy labels Model Selection +1

DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets

1 code implementation13 Oct 2020 Shujun Wang, Lequan Yu, Kang Li, Xin Yang, Chi-Wing Fu, Pheng-Ann Heng

Our DoFE framework dynamically enriches the image features with additional domain prior knowledge learned from multi-source domains to make the semantic features more discriminative.

Domain Generalization Image Segmentation +2

Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation

no code implementations4 Oct 2020 Kang Li, Lequan Yu, Shujun Wang, Pheng-Ann Heng

Considering multi-modality data with the same anatomic structures are widely available in clinic routine, in this paper, we aim to exploit the prior knowledge (e. g., shape priors) learned from one modality (aka., assistant modality) to improve the segmentation performance on another modality (aka., target modality) to make up annotation scarcity.

Cardiac Segmentation Image Segmentation +3

Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation

1 code implementation21 Jul 2020 Yanning Zhou, Hao Chen, Huangjing Lin, Pheng-Ann Heng

The teacher's self-ensemble predictions from $K$-time augmented samples are used to construct the reliable pseudo-labels for optimizing the student.

Instance Segmentation Knowledge Distillation +2

Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization

no code implementations ECCV 2020 Shujun Wang, Lequan Yu, Caizi Li, Chi-Wing Fu, Pheng-Ann Heng

To this end, we present a new domain generalization framework that learns how to generalize across domains simultaneously from extrinsic relationship supervision and intrinsic self-supervision for images from multi-source domains.

Anomaly Detection Domain Generalization +3

Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video

1 code implementation6 Jul 2020 Zixu Zhao, Yueming Jin, Xiaojie Gao, Qi Dou, Pheng-Ann Heng

Considering the fast instrument motion, we further introduce a flow compensator to estimate intermediate motion within continuous frames, with a novel cycle learning strategy.

Segmentation

Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains

1 code implementation4 Jul 2020 Quande Liu, Qi Dou, Pheng-Ann Heng

We present a novel shape-aware meta-learning scheme to improve the model generalization in prostate MRI segmentation.

Domain Generalization Meta-Learning +1

Deep Mining External Imperfect Data for Chest X-ray Disease Screening

no code implementations6 Jun 2020 Luyang Luo, Lequan Yu, Hao Chen, Quande Liu, Xi Wang, Jiaqi Xu, Pheng-Ann Heng

Recent researches have demonstrated that performance bottleneck exists in joint training on different CXR datasets, and few made efforts to address the obstacle.

General Classification Missing Labels +1

LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition

1 code implementation21 Apr 2020 Xueying Shi, Yueming Jin, Qi Dou, Pheng-Ann Heng

Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency (LRTD) among continuous frames.

Active Learning

A Rotation-Invariant Framework for Deep Point Cloud Analysis

1 code implementation16 Mar 2020 Xianzhi Li, Ruihui Li, Guangyong Chen, Chi-Wing Fu, Daniel Cohen-Or, Pheng-Ann Heng

Recently, many deep neural networks were designed to process 3D point clouds, but a common drawback is that rotation invariance is not ensured, leading to poor generalization to arbitrary orientations.

Point Cloud Generation Retrieval

PointAugment: an Auto-Augmentation Framework for Point Cloud Classification

2 code implementations CVPR 2020 Ruihui Li, Xianzhi Li, Pheng-Ann Heng, Chi-Wing Fu

We present PointAugment, a new auto-augmentation framework that automatically optimizes and augments point cloud samples to enrich the data diversity when we train a classification network.

3D Point Cloud Data Augmentation Classification +4

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

1 code implementation22 Feb 2020 Cheng Chen, Qi Dou, Yueming Jin, Hao Chen, Jing Qin, Pheng-Ann Heng

We tackle this challenge and propose a novel multimodal segmentation framework which is robust to the absence of imaging modalities.

Brain Tumor Segmentation Disentanglement +3

Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search

no code implementations20 Feb 2020 Xiaojie Gao, Yueming Jin, Qi Dou, Pheng-Ann Heng

Automatic surgical gesture recognition is fundamental for improving intelligence in robot-assisted surgery, such as conducting complicated tasks of surgery surveillance and skill evaluation.

Action Segmentation Gesture Recognition +4

Instance Shadow Detection

3 code implementations CVPR 2020 Tianyu Wang, Xiao-Wei Hu, Qiong Wang, Pheng-Ann Heng, Chi-Wing Fu

Then, we pair up the predicted shadow and object instances, and match them with the predicted shadow-object associations to generate the final results.

Instance Shadow Detection Object +1

CANet: Cross-disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading

1 code implementation4 Nov 2019 Xiaomeng Li, Xiao-Wei Hu, Lequan Yu, Lei Zhu, Chi-Wing Fu, Pheng-Ann Heng

In this paper, we present a novel cross-disease attention network (CANet) to jointly grade DR and DME by exploring the internal relationship between the diseases with only image-level supervision.

CGC-Net: Cell Graph Convolutional Network for Grading of Colorectal Cancer Histology Images

1 code implementation3 Sep 2019 Yanning Zhou, Simon Graham, Navid Alemi Koohbanani, Muhammad Shaban, Pheng-Ann Heng, Nasir Rajpoot

Furthermore, to deal with redundancy in the graph, we propose a sampling technique that removes nodes in areas of dense nuclear activity.

Joint Segmentation and Landmark Localization of Fetal Femur in Ultrasound Volumes

no code implementations31 Aug 2019 Xu Wang, Xin Yang, Haoran Dou, Shengli Li, Pheng-Ann Heng, Dong Ni

In this paper, we propose an effective framework for simultaneous segmentation and landmark localization in prenatal ultrasound volumes.

Segmentation

IRNet: Instance Relation Network for Overlapping Cervical Cell Segmentation

1 code implementation19 Aug 2019 Yanning Zhou, Hao Chen, Jiaqi Xu, Qi Dou, Pheng-Ann Heng

In this paper, we propose a novel Instance Relation Network (IRNet) for robust overlapping cell segmentation by exploring instance relation interaction.

Cell Segmentation Instance Segmentation +5

Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening

no code implementations26 Jul 2019 Xi Wang, Hao Chen, Luyang Luo, An-ran Ran, Poemen P. Chan, Clement C. Tham, Carol Y. Cheung, Pheng-Ann Heng

Besides, the proposed multi-task learning network is capable of exploring the structure and function relationship from the OCT image and visual field measurement simultaneously, which contributes to classification performance boosting.

Multi-Task Learning regression

Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video

1 code implementation18 Jul 2019 Yueming Jin, Keyun Cheng, Qi Dou, Pheng-Ann Heng

In this paper, we propose a novel framework to leverage instrument motion information, by incorporating a derived temporal prior to an attention pyramid network for accurate segmentation.

Decoder Segmentation

Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation

8 code implementations16 Jul 2019 Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, Pheng-Ann Heng

We design a novel uncertainty-aware scheme to enable the student model to gradually learn from the meaningful and reliable targets by exploiting the uncertainty information.

Image Segmentation Left Atrium Segmentation +3

494