Search Results for author: Jianfeng Zhang

Found 44 papers, 16 papers with code

MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts

no code implementations29 May 2024 Renchunzi Xie, Ambroise Odonnat, Vasilii Feofanov, Weijian Deng, Jianfeng Zhang, Bo An

Leveraging the models' outputs, specifically the logits, is a common approach to estimating the test accuracy of a pre-trained neural network on out-of-distribution (OOD) samples without requiring access to the corresponding ground truth labels.

SEGAN: semi-supervised learning approach for missing data imputation

no code implementations21 May 2024 Xiaohua Pan, Weifeng Wu, Peiran Liu, Zhen Li, Peng Lu, Peijian Cao, Jianfeng Zhang, Xianfei Qiu, Yangyang Wu

In addition, the SE-GAN model introduces a missing hint matrix to allow the discriminator to more effectively distinguish between known data and data filled by the generator.

Magic-Boost: Boost 3D Generation with Mutli-View Conditioned Diffusion

no code implementations9 Apr 2024 Fan Yang, Jianfeng Zhang, Yichun Shi, Bowen Chen, Chenxu Zhang, Huichao Zhang, Xiaofeng Yang, Jiashi Feng, Guosheng Lin

Benefiting from the rapid development of 2D diffusion models, 3D content creation has made significant progress recently.

3D Generation

An edge detection-based deep learning approach for tear meniscus height measurement

no code implementations23 Mar 2024 Kesheng Wang, Kunhui Xu, Xiaoyu Chen, Chunlei He, Jianfeng Zhang, Dexing Kong, Qi Dai, Shoujun Huang

For improved segmentation of the pupil and tear meniscus areas, the convolutional neural network Inceptionv3 was first implemented as an image quality assessment model, effectively identifying higher-quality images with an accuracy of 98. 224%.

Edge Detection Image Quality Assessment

Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift

no code implementations17 Jan 2024 Renchunzi Xie, Ambroise Odonnat, Vasilii Feofanov, Ievgen Redko, Jianfeng Zhang, Bo An

Our key idea is that the model should be adjusted with a higher magnitude of gradients when it does not generalize to the test dataset with a distribution shift.

AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

no code implementations29 Nov 2023 Jianfeng Zhang, Xuanmeng Zhang, Huichao Zhang, Jun Hao Liew, Chenxu Zhang, Yi Yang, Jiashi Feng

We study the problem of creating high-fidelity and animatable 3D avatars from only textual descriptions.

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

2 code implementations27 Nov 2023 Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, Mike Zheng Shou

Existing animation works typically employ the frame-warping technique to animate the reference image towards the target motion.

Image Animation

ViT-Lens: Towards Omni-modal Representations

1 code implementation27 Nov 2023 Weixian Lei, Yixiao Ge, Kun Yi, Jianfeng Zhang, Difei Gao, Dylan Sun, Yuying Ge, Ying Shan, Mike Zheng Shou

In this paper, we present ViT-Lens-2 that facilitates efficient omni-modal representation learning by perceiving novel modalities with a pretrained ViT and aligning them to a pre-defined space.

EEG Image Generation +2

Continual Learning via Manifold Expansion Replay

no code implementations12 Oct 2023 Zihao Xu, Xuan Tang, Yufei Shi, Jianfeng Zhang, Jian Yang, Mingsong Chen, Xian Wei

To address this problem, we propose a novel replay strategy called Manifold Expansion Replay (MaER).

Continual Learning Management

Automatic nodule identification and differentiation in ultrasound videos to facilitate per-nodule examination

no code implementations10 Oct 2023 Siyuan Jiang, Yan Ding, Yuling Wang, Lei Xu, Wenli Dai, Wanru Chang, Jianfeng Zhang, Jie Yu, Jianqiao Zhou, Chunquan Zhang, Ping Liang, Dexing Kong

Ultrasound is a vital diagnostic technique in health screening, with the advantages of non-invasive, cost-effective, and radiation free, and therefore is widely applied in the diagnosis of nodules.

GETAvatar: Generative Textured Meshes for Animatable Human Avatars

no code implementations ICCV 2023 Xuanmeng Zhang, Jianfeng Zhang, Rohan Chacko, Hongyi Xu, Guoxian Song, Yi Yang, Jiashi Feng

We study the problem of 3D-aware full-body human generation, aiming at creating animatable human avatars with high-quality textures and geometries.

Image Generation

MagicAvatar: Multimodal Avatar Generation and Animation

no code implementations28 Aug 2023 Jianfeng Zhang, Hanshu Yan, Zhongcong Xu, Jiashi Feng, Jun Hao Liew

This report presents MagicAvatar, a framework for multimodal video generation and animation of human avatars.

Video Generation

MagicEdit: High-Fidelity and Temporally Coherent Video Editing

no code implementations28 Aug 2023 Jun Hao Liew, Hanshu Yan, Jianfeng Zhang, Zhongcong Xu, Jiashi Feng

In this report, we present MagicEdit, a surprisingly simple yet effective solution to the text-guided video editing task.

Translation Video Editing

ViT-Lens: Initiating Omni-Modal Exploration through 3D Insights

1 code implementation20 Aug 2023 Weixian Lei, Yixiao Ge, Jianfeng Zhang, Dylan Sun, Kun Yi, Ying Shan, Mike Zheng Shou

A well-trained lens with a ViT backbone has the potential to serve as one of these foundation models, supervising the learning of subsequent modalities.

3D Classification Question Answering +4

Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction like Radiologists

no code implementations20 Jul 2023 Jianpeng Zhang, Xianghua Ye, Jianfeng Zhang, Yuxing Tang, Minfeng Xu, Jianfei Guo, Xin Chen, Zaiyi Liu, Jingren Zhou, Le Lu, Ling Zhang

In this paper, we propose a radiologist-inspired method to simulate the diagnostic process of radiologists, which is composed of context parsing and prototype recalling modules.

Decision Making

Contrastive Shapelet Learning for Unsupervised Multivariate Time Series Representation Learning

1 code implementation30 May 2023 Zhiyu Liang, Jianfeng Zhang, Chen Liang, Hongzhi Wang, Zheng Liang, Lujia Pan

Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels.

Anomaly Detection Data Augmentation +2

Group Equivariant BEV for 3D Object Detection

no code implementations26 Apr 2023 Hongwei Liu, Jian Yang, Jianfeng Zhang, Dongheng Shao, Jielong Guo, Shaobo Li, Xuan Tang, Xian Wei

Experimental results demonstrate that GeqBevNet can extract more rotational equivariant features in the 3D object detection of the actual road scene and improve the performance of object orientation prediction.

3D Object Detection Object +2

Democratic Policy Decisions with Decentralized Promises Contingent on Vote Outcome

no code implementations17 Apr 2023 Ali Lazrak, Jianfeng Zhang

We study pre-vote interactions in a committee that enacts a welfare-improving reform through voting.

Decision Making

OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis

no code implementations CVPR 2023 Hongyi Xu, Guoxian Song, Zihang Jiang, Jianfeng Zhang, Yichun Shi, Jing Liu, WanChun Ma, Jiashi Feng, Linjie Luo

We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D heads with compelling dynamic details under full disentangled control over camera poses, facial expressions, head shapes, articulated neck and jaw poses.

AgileGAN3D: Few-Shot 3D Portrait Stylization by Augmented Transfer Learning

no code implementations24 Mar 2023 Guoxian Song, Hongyi Xu, Jing Liu, Tiancheng Zhi, Yichun Shi, Jianfeng Zhang, Zihang Jiang, Jiashi Feng, Shen Sang, Linjie Luo

Capitalizing on the recent advancement of 3D-aware GAN models, we perform \emph{guided transfer learning} on a pretrained 3D GAN generator to produce multi-view-consistent stylized renderings.

Transfer Learning

Inducing Neural Collapse in Deep Long-tailed Learning

1 code implementation24 Feb 2023 Xuantong Liu, Jianfeng Zhang, Tianyang Hu, He Cao, Lujia Pan, Yuan YAO

One of the reasons is that the learned representations (i. e. features) from the imbalanced datasets are less effective than those from balanced datasets.

PV3D: A 3D Generative Model for Portrait Video Generation

no code implementations13 Dec 2022 Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Wenqing Zhang, Song Bai, Jiashi Feng, Mike Zheng Shou

While some prior works have applied such image GANs to unconditional 2D portrait video generation and static 3D portrait synthesis, there are few works successfully extending GANs for generating 3D-aware portrait videos.

Video Generation

Med-Query: Steerable Parsing of 9-DoF Medical Anatomies with Query Embedding

1 code implementation5 Dec 2022 Heng Guo, Jianfeng Zhang, Ke Yan, Le Lu, Minfeng Xu

For rib parsing, CT scans have been annotated at the rib instance-level for quantitative evaluation, similarly for spine vertebrae and abdominal organs.

Anatomy Computed Tomography (CT) +5

AvatarGen: A 3D Generative Model for Animatable Human Avatars

1 code implementation26 Nov 2022 Jianfeng Zhang, Zihang Jiang, Dingdong Yang, Hongyi Xu, Yichun Shi, Guoxian Song, Zhongcong Xu, Xinchao Wang, Jiashi Feng

Specifically, we decompose the generative 3D human synthesis into pose-guided mapping and canonical representation with predefined human pose and shape, such that the canonical representation can be explicitly driven to different poses and shapes with the guidance of a 3D parametric human model SMPL.

A New Probabilistic V-Net Model with Hierarchical Spatial Feature Transform for Efficient Abdominal Multi-Organ Segmentation

no code implementations2 Aug 2022 Minfeng Xu, Heng Guo, Jianfeng Zhang, Ke Yan, Le Lu

Accurate and robust abdominal multi-organ segmentation from CT imaging of different modalities is a challenging task due to complex inter- and intra-organ shape and appearance variations among abdominal organs.

Decoder Organ Segmentation +1

AvatarGen: a 3D Generative Model for Animatable Human Avatars

1 code implementation1 Aug 2022 Jianfeng Zhang, Zihang Jiang, Dingdong Yang, Hongyi Xu, Yichun Shi, Guoxian Song, Zhongcong Xu, Xinchao Wang, Jiashi Feng

Unsupervised generation of clothed virtual humans with various appearance and animatable poses is important for creating 3D human avatars and other AR/VR applications.

3D Human Reconstruction

PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

1 code implementation CVPR 2022 Kehong Gong, Bingbing Li, Jianfeng Zhang, Tao Wang, Jing Huang, Michael Bi Mi, Jiashi Feng, Xinchao Wang

Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions like consistency loss to guide the learning, which, inevitably, leads to inferior results in real-world scenarios with unseen poses.

3D Human Pose Estimation Hallucination

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

no code implementations8 Dec 2021 Mingfei Chen, Jianfeng Zhang, Xiangyu Xu, Lijuan Liu, Yujun Cai, Jiashi Feng, Shuicheng Yan

Meanwhile, for achieving higher rendering efficiency, we introduce a progressive rendering pipeline through geometry guidance, which leverages the geometric feature volume and the predicted density values to progressively reduce the number of sampling points and speed up the rendering process.

Direct Multi-view Multi-person 3D Pose Estimation

2 code implementations NeurIPS 2021 Tao Wang, Jianfeng Zhang, Yujun Cai, Shuicheng Yan, Jiashi Feng

Instead of estimating 3D joint locations from costly volumetric representation or reconstructing the per-person 3D pose from multiple detected 2D poses as in previous methods, MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks.

Ranked #3 on 3D Multi-Person Pose Estimation on Panoptic (using extra training data)

3D Multi-Person Pose Estimation 3D Pose Estimation

Knothe-Rosenblatt transport for Unsupervised Domain Adaptation

no code implementations6 Oct 2021 Aladin Virmaux, Illyyne Saffar, Jianfeng Zhang, Balázs Kégl

Knothe-Rosenblatt Domain Adaptation (KRDA) is based on the Knothe-Rosenblatt transport: we exploit autoregressive density estimation algorithms to accurately model the different sources by an autoregressive model using a mixture of Gaussians.

Density Estimation Unsupervised Domain Adaptation

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

1 code implementation CVPR 2021 Kehong Gong, Jianfeng Zhang, Jiashi Feng

To address this problem, we present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity and thus improve generalization of the trained 2D-to-3D pose estimator.

 Ranked #1 on Monocular 3D Human Pose Estimation on Human3.6M (Use Video Sequence metric)

Data Augmentation Monocular 3D Human Pose Estimation +1

Body Meshes as Points

1 code implementation CVPR 2021 Jianfeng Zhang, Dongdong Yu, Jun Hao Liew, Xuecheng Nie, Jiashi Feng

In this work, we present a single-stage model, Body Meshes as Points (BMP), to simplify the pipeline and lift both efficiency and performance.

3D Human Shape Estimation 3D Multi-Person Pose Estimation +1

Mean Field Games Master Equations with Non-separable Hamiltonians and Displacement Monotonicity

no code implementations29 Jan 2021 Wilfrid Gangbo, Alpár R. Mészáros, Chenchen Mou, Jianfeng Zhang

In this manuscript, we propose a structural condition on non-separable Hamiltonians, which we term displacement monotonicity condition, to study second order mean field games master equations.

Analysis of PDEs Optimization and Control Probability 35R15, 49N80, 49Q22, 60H30, 91A16, 93E20

Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation

no code implementations NeurIPS 2020 Jianfeng Zhang, Xuecheng Nie, Jiashi Feng

In this work, we propose a novel framework, Inference Stage Optimization (ISO), for improving the generalizability of 3D pose models when source and target data come from different pose distributions.

Ranked #121 on 3D Human Pose Estimation on 3DPW (PA-MPJPE metric)

3D Human Pose Estimation Self-Supervised Learning

Hierarchical Graph Pooling with Structure Learning

3 code implementations14 Nov 2019 Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, Can Wang

HGP-SL incorporates graph pooling and structure learning into a unified module to generate hierarchical representations of graphs.

Graph Classification Representation Learning

Single-Stage Multi-Person Pose Machines

1 code implementation ICCV 2019 Xuecheng Nie, Jianfeng Zhang, Shuicheng Yan, Jiashi Feng

Based on SPR, we develop the SPM model that can directly predict structured poses for multiple persons in a single stage, and thus offer a more compact pipeline and attractive efficiency advantage over two-stage methods.

3D Pose Estimation Keypoint Detection +1

Predicting Path Failure In Time-Evolving Graphs

2 code implementations10 May 2019 Jia Li, Zhichao Han, Hong Cheng, Jiao Su, Pengyun Wang, Jianfeng Zhang, Lujia Pan

Through experiments on a real-world telecommunication network and a traffic network in California, we demonstrate the superiority of LRGCN to other competing methods in path failure prediction, and prove the effectiveness of SAPE on path representation.

Interactive Binary Image Segmentation with Edge Preservation

no code implementations10 Sep 2018 Jianfeng Zhang, Liezhuo Zhang, Yuankai Teng, Xiao-Ping Zhang, Song Wang, Lili Ju

Binary image segmentation plays an important role in computer vision and has been widely used in many applications such as image and video editing, object extraction, and photo composition.

Image Segmentation Interactive Segmentation +4

Learning for Disparity Estimation through Feature Constancy

2 code implementations CVPR 2018 Zhengfa Liang, Yiliu Feng, Yulan Guo, Hengzhu Liu, Wei Chen, Linbo Qiao, Li Zhou, Jianfeng Zhang

The second part performs matching cost calculation, matching cost aggregation and disparity calculation to estimate the initial disparity using shared features.

Disparity Estimation Stereo Matching +1

Cannot find the paper you are looking for? You can Submit a new open access paper.