Search Results for author: Jianke Zhu

Found 52 papers, 27 papers with code

Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation

1 code implementation18 Apr 2024 Song Wang, Jiawei Yu, Wentong Li, Wenyu Liu, Xiaolu Liu, Junbo Chen, Jianke Zhu

Furthermore, the voxels in the boundary region are more challenging to differentiate than those in the interior.

Autonomous Vehicles

MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction

1 code implementation1 Apr 2024 Xiaolu Liu, Song Wang, Wentong Li, Ruizi Yang, Junbo Chen, Jianke Zhu

Currently, high-definition (HD) map construction leans towards a lightweight online generation tendency, which aims to preserve timely and reliable road scene information.

HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes

no code implementations29 Mar 2024 Zhuopeng Li, Yilin Zhang, Chenming Wu, Jianke Zhu, Liangjun Zhang

The rapid growth of 3D Gaussian Splatting (3DGS) has revolutionized neural rendering, enabling real-time production of high-quality renderings.

Autonomous Driving Neural Rendering

OccFiner: Offboard Occupancy Refinement with Hybrid Propagation

no code implementations13 Mar 2024 Hao Shi, Song Wang, Jiaming Zhang, Xiaoting Yin, Zhongdao Wang, Zhijian Zhao, Guangming Wang, Jianke Zhu, Kailun Yang, Kaiwei Wang

Vision-based occupancy prediction, also known as 3D Semantic Scene Completion (SSC), presents a significant challenge in computer vision.

3D Semantic Scene Completion

Traj-LIO: A Resilient Multi-LiDAR Multi-IMU State Estimator Through Sparse Gaussian Process

1 code implementation14 Feb 2024 Xin Zheng, Jianke Zhu

Nowadays, sensor suits have been equipped with redundant LiDARs and IMUs to mitigate the risks associated with sensor failure.

Osprey: Pixel Understanding with Visual Instruction Tuning

2 code implementations15 Dec 2023 Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, Jianke Zhu

In this paper, we propose Osprey, a mask-text instruction tuning approach, to extend MLLMs by incorporating fine-grained mask regions into language instruction, aiming at achieving pixel-wise visual understanding.

Language Modelling

DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes

no code implementations28 Nov 2023 Zhuopeng Li, Chenming Wu, Liangjun Zhang, Jianke Zhu

Despite the recent success of Neural Radiance Field (NeRF), it is still challenging to render large-scale driving scenes with long trajectories, particularly when the rendering quality and efficiency are in high demand.

Autonomous Driving Depth Estimation +1

Semantic-Preserved Point-based Human Avatar

1 code implementation20 Nov 2023 Lixiang Lin, Jianke Zhu

To enable realistic experience in AR/VR and digital entertainment, we present the first point-based human avatar model that embodies the entirety expressive range of digital humans.

Virtual Try-on

Label-efficient Segmentation via Affinity Propagation

1 code implementation NeurIPS 2023 Wentong Li, Yuqian Yuan, Song Wang, Wenyu Liu, Dongqi Tang, Jian Liu, Jianke Zhu, Lei Zhang

In this work, we formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels.

Box-supervised Instance Segmentation Segmentation +2

Traj-LO: In Defense of LiDAR-Only Odometry Using an Effective Continuous-Time Trajectory

1 code implementation25 Sep 2023 Xin Zheng, Jianke Zhu

Therefore, our proposed Traj-LO approach tries to recover the spatial-temporal consistent movement of LiDAR by tightly coupling the geometric information from LiDAR points and kinematic constraints from trajectory smoothness.

Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport

1 code implementation ICCV 2023 Wentong Li, Yuqian Yuan, Song Wang, Jianke Zhu, Jianshu Li, Jian Liu, Lei Zhang

Weakly-supervised image segmentation has recently attracted increasing research attentions, aiming to avoid the expensive pixel-wise labeling.

Image Segmentation Panoptic Segmentation

Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images

1 code implementation12 Jul 2023 Jinwei Ren, Jianke Zhu

Most of the existing methods extract features from color images to estimate the root-aligned hand meshes, which neglect the crucial depth and scale information in the real world.

FastMESH: Fast Surface Reconstruction by Hexagonal Mesh-based Neural Rendering

no code implementations29 May 2023 Yisu Zhang, Jianke Zhu, Lixiang Lin

Despite the promising results of multi-view reconstruction, the recent neural rendering-based methods, such as implicit surface rendering (IDR) and volume rendering (NeuS), not only incur a heavy computational burden on training but also have the difficulties in disentangling the geometric and appearance.

Neural Rendering Novel View Synthesis +1

BundleRecon: Ray Bundle-Based 3D Neural Reconstruction

no code implementations12 May 2023 Weikun Zhang, Jianke Zhu

With the growing popularity of neural rendering, there has been an increasing number of neural implicit multi-view reconstruction methods.

Neural Rendering

Multi-View Stereo Representation Revisit: Region-Aware MVSNet

no code implementations26 Apr 2023 Yisu Zhang, Jianke Zhu, Lixiang Lin

Comparing to the conventional deep learning-based multi-view stereo methods, our proposed RA-MVSNet approach obtains more complete reconstruction results by taking advantage of signed distance supervision.

3D Reconstruction Point Clouds

LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation

1 code implementation CVPR 2023 Song Wang, Wentong Li, Wenyu Liu, Xiaolu Liu, Jianke Zhu

To mitigate the defects caused by lacking semantic cues in LiDAR data, we present an online Camera-to-LiDAR distillation scheme to facilitate the semantic learning from image to point cloud.

Autonomous Driving

Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation

no code implementations3 Feb 2023 Chuan Zhang, Daoxin Zhang, Ruixiu Zhang, Jiawei Li, Jianke Zhu

To tackle this problem, we propose a multimodal relevance estimation network to capture the relevant semantics among modalities in multimodal emotions.

Contrastive Learning

Multi-View Stereo Representation Revist: Region-Aware MVSNet

no code implementations CVPR 2023 Yisu Zhang, Jianke Zhu, Lixiang Lin

It is essential to the textureless regions and surface boundary that cannot be properly reconstructed. To address this issue, we suggest to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces.

Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution

2 code implementations3 Dec 2022 Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Risheng Yu, Xiansheng Hua, Lei Zhang

In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention.

Box-supervised Instance Segmentation Segmentation

FastHuman: Reconstructing High-Quality Clothed Human in Minutes

no code implementations26 Nov 2022 Lixiang Lin, Songyou Peng, Qijun Gan, Jianke Zhu

We propose an approach for optimizing high-quality clothed human body shapes in minutes, using multi-view posed images.

Neural Rendering Vocal Bursts Intensity Prediction

Box-supervised Instance Segmentation with Level Set Evolution

1 code implementation19 Jul 2022 Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xiansheng Hua, Lei Zhang

A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance.

Box-supervised Instance Segmentation Segmentation

Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters

2 code implementations4 Jul 2022 Wenyu Liu, Wentong Li, Jianke Zhu, Miaomiao Cui, Xuansong Xie, Lei Zhang

With DIAL-Filters, we design both unsupervised and supervised frameworks for nighttime driving-scene segmentation, which can be trained in an end-to-end manner.

Autonomous Driving Scene Segmentation +1

Warped Convolutional Networks: Bridge Homography to sl(3) algebra by Group Convolution

no code implementations23 Jun 2022 Xinrui Zhan, Yang Li, Wenyu Liu, Jianke Zhu

In this paper, we propose Warped Convolution Networks (WCN) to effectively learn and represent the homography by SL(3) group and sl(3) algebra with group convolution.

Homography Estimation Object Tracking

ECTLO: Effective Continuous-time Odometry Using Range Image for LiDAR with Small FoV

1 code implementation17 Jun 2022 Xin Zheng, Jianke Zhu

Prism-based LiDARs are more compact and cheaper than the conventional mechanical multi-line spinning LiDARs, which have become increasingly popular in robotics, recently.

Autonomous Driving

Multiview Textured Mesh Recovery by Differentiable Rendering

1 code implementation25 May 2022 Lixiang Lin, Jianke Zhu, Yisu Zhang

Although having achieved the promising results on shape and color recovery through self-supervision, the multi-layer perceptrons-based methods usually suffer from heavy computational cost on learning the deep implicit surface representation.

Inverse Rendering

READ: Large-Scale Neural Scene Rendering for Autonomous Driving

1 code implementation11 May 2022 Zhuopeng Li, Lu Li, Zeyu Ma, Ping Zhang, Junbo Chen, Jianke Zhu

In this paper, a large-scale neural rendering method is proposed to synthesize the autonomous driving scene~(READ), which makes it possible to synthesize large-scale driving scenarios on a PC through a variety of sampling schemes.

3D Scene Reconstruction Autonomous Driving +4

End-to-end Weakly-supervised Single-stage Multiple 3D Hand Mesh Reconstruction from a Single RGB Image

1 code implementation18 Apr 2022 Jinwei Ren, Jianke Zhu, Jialiang Zhang

In this paper, we consider the challenging task of simultaneously locating and recovering multiple hands from a single 2D image.

Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions

1 code implementation15 Dec 2021 Wenyu Liu, Gaofeng Ren, Runsheng Yu, Shi Guo, Jianke Zhu, Lei Zhang

Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions.

Image Enhancement object-detection +1

Homography Decomposition Networks for Planar Object Tracking

3 code implementations15 Dec 2021 Xinrui Zhan, Yueran Liu, Jianke Zhu, Yang Li

Planar object tracking plays an important role in AI applications, such as robotics, visual servoing, and visual SLAM.

Object Object Tracking

Deep Level Set for Box-supervised Instance Segmentation in Aerial Images

no code implementations7 Dec 2021 Wentong Li, Yijie Chen, Wenyu Liu, Jianke Zhu

Instead of learning the pairwise affinity, the level set method with the carefully designed energy functions treats the object segmentation as curve evolution, which is able to accurately recover the object's boundaries and prevent the interference from the indistinguishable background and similar objects.

Box-supervised Instance Segmentation Segmentation +1

Topology-Preserved Human Reconstruction with Details

1 code implementation11 Jun 2021 Lixiang Lin, Jianke Zhu

It is challenging to directly estimate the human geometry from a single image due to the high diversity and complexity of body shapes with the various clothing styles.

Translational Symmetry-Aware Facade Parsing for 3D Building Reconstruction

no code implementations2 Jun 2021 Hantang Liu, Wentong Li, Jianke Zhu

Although enjoying the merits of promising results on the semantic parsing, deep learning methods cannot directly make use of the architectural rules, which play an important role for man-made structures.

Semantic Parsing Semantic Segmentation

Oriented RepPoints for Aerial Object Detection

2 code implementations CVPR 2022 Wentong Li, Yijie Chen, Kaixuan Hu, Jianke Zhu

In contrast to the generic object, aerial targets are often non-axis aligned with arbitrary orientations having the cluttered surroundings.

Object Object Detection In Aerial Images +2

Efficient LiDAR Odometry for Autonomous Driving

no code implementations22 Apr 2021 Xin Zheng, Jianke Zhu

LiDAR odometry plays an important role in self-localization and mapping for autonomous navigation, which is usually treated as a scan registration problem.

Autonomous Driving Autonomous Navigation

Horizontal-to-Vertical Video Conversion

1 code implementation11 Jan 2021 Tun Zhu, Daoxin Zhang, Yao Hu, Tianran Wang, XiaoLong Jiang, Jianke Zhu, Jiawei Li

Alongside the prevalence of mobile videos, the general public leans towards consuming vertical videos on hand-held devices.

Boundary Detection Multi-Object Tracking

Weakly-Supervised Multi-Face 3D Reconstruction

1 code implementation6 Jan 2021 Jialiang Zhang, Lixiang Lin, Jianke Zhu, Steven C. H. Hoi

3D face reconstruction plays a very important role in many real-world multimedia applications, including digital entertainment, social media, affection analysis, and person identification.

3D Face Reconstruction 3D Reconstruction +3

DeVLBert: Learning Deconfounded Visio-Linguistic Representations

1 code implementation16 Aug 2020 Shengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang, Zhou Zhao, Jianke Zhu, Jin Yu, Hongxia Yang, Fei Wu

In this paper, we propose to investigate the problem of out-of-domain visio-linguistic pretraining, where the pretraining data distribution differs from that of downstream data on which the pretrained model will be fine-tuned.

Image Retrieval Question Answering +2

Attribute-aware Pedestrian Detection in a Crowd

1 code implementation21 Oct 2019 Jialiang Zhang, Lixiang Lin, Yang Li, Yun-chen Chen, Jianke Zhu, Yao Hu, Steven C. H. Hoi

To tackle this critical problem, we propose an attribute-aware pedestrian detector to explicitly model people's semantic attributes in a high-level feature detection fashion.

Attribute Pedestrian Detection

Manifold Adversarial Learning

no code implementations16 Jul 2018 Shufei Zhang, Kai-Zhu Huang, Jianke Zhu, Yang Liu

All the existing adversarial training methods consider only how the worst perturbed examples (i. e., adversarial examples) could affect the model output.

Single-Shot Bidirectional Pyramid Networks for High-Quality Object Detection

no code implementations22 Mar 2018 Xiongwei Wu, Daoxin Zhang, Jianke Zhu, Steven C. H. Hoi

Recent years have witnessed many exciting achievements for object detection using deep learning techniques.

Object object-detection +2

Robust Estimation of Similarity Transformation for Visual Object Tracking

2 code implementations14 Dec 2017 Yang Li, Jianke Zhu, Steven C. H. Hoi, Wenjie Song, Zhefeng Wang, Hantang Liu

In order to efficiently search in such a large 4-DoF space in real-time, we formulate the problem into two 2-DoF sub-problems and apply an efficient Block Coordinates Descent solver to optimize the estimation result.

Object Visual Object Tracking

Feature Agglomeration Networks for Single Stage Face Detection

no code implementations3 Dec 2017 Jialiang Zhang, Xiongwei Wu, Jianke Zhu, Steven C. H. Hoi

In this paper, we propose a novel simple yet effective framework of "Feature Agglomeration Networks" (FANet) to build a new single stage face detector, which not only achieves state-of-the-art performance but also runs efficiently.

Face Detection

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

no code implementations16 Nov 2017 Dan Ma, Bin Liu, Zhao Kang, Jiayu Zhou, Jianke Zhu, Zenglin Xu

Generating high fidelity identity-preserving faces with different facial attributes has a wide range of applications.

Image Generation

Scalable Image Retrieval by Sparse Product Quantization

no code implementations15 Mar 2016 Qingqun Ning, Jianke Zhu, Zhiyuan Zhong, Steven C. H. Hoi, Chun Chen

Unlike the existing approaches, in this paper, we propose a novel approach called Sparse Product Quantization (SPQ) to encoding the high-dimensional feature vectors into sparse representation.

Content-Based Image Retrieval Quantization +1

Reliable Patch Trackers: Robust Visual Tracking by Exploiting Reliable Patches

no code implementations CVPR 2015 Yang Li, Jianke Zhu, Steven C. H. Hoi

Most modern trackers typically employ a bounding box given in the first frame to track visual objects, where their tracking results are often sensitive to the initialization.

Clustering Visual Tracking

Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering

no code implementations NeurIPS 2009 Lei Wu, Rong Jin, Steven C. Hoi, Jianke Zhu, Nenghai Yu

Learning distance functions with side information plays a key role in many machine learning and data mining applications.

Clustering

Adaptive Regularization for Transductive Support Vector Machine

no code implementations NeurIPS 2009 Zenglin Xu, Rong Jin, Jianke Zhu, Irwin King, Michael Lyu, Zhirong Yang

In this framework, SVM and TSVM can be regarded as a learning machine without regularization and one with full regularization from the unlabeled data, respectively.

Efficient Convex Relaxation for Transductive Support Vector Machine

no code implementations NeurIPS 2007 Zenglin Xu, Rong Jin, Jianke Zhu, Irwin King, Michael Lyu

We consider the problem of Support Vector Machine transduction, which involves a combinatorial problem with exponential computational complexity in the number of unlabeled examples.

Cannot find the paper you are looking for? You can Submit a new open access paper.