Structure-Aware Sparse-View X-ray 3D Reconstruction

18 Nov 2023  ·  Yuanhao Cai, Jiahao Wang, Alan Yuille, Zongwei Zhou, Angtian Wang ·

X-ray, known for its ability to reveal internal structures of objects, is expected to provide richer information for 3D reconstruction than visible light. Yet, existing neural radiance fields (NeRF) algorithms overlook this important nature of X-ray, leading to their limitations in capturing structural contents of imaged objects. In this paper, we propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF), for sparse-view X-ray 3D reconstruction. Firstly, we design a Line Segment-based Transformer (Lineformer) as the backbone of SAX-NeRF. Linefomer captures internal structures of objects in 3D space by modeling the dependencies within each line segment of an X-ray. Secondly, we present a Masked Local-Global (MLG) ray sampling strategy to extract contextual and geometric information in 2D projection. Plus, we collect a larger-scale dataset X3D covering wider X-ray applications. Experiments on X3D show that SAX-NeRF surpasses previous NeRF-based methods by 12.56 and 2.49 dB on novel view synthesis and CT reconstruction. Code, models, and data are released at

PDF Abstract


Introduced in the Paper:

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Low-Dose X-Ray Ct Reconstruction X3D SAX-NeRF PSNR 37.25 # 1
SSIM 0.9753 # 1
Low-Dose X-Ray Ct Reconstruction X3D FDK PSNR 25.12 # 9
SSIM 0.6422 # 9
Low-Dose X-Ray Ct Reconstruction X3D ASD-POCS PSNR 32.32 # 6
SSIM 0.9400 # 4
Low-Dose X-Ray Ct Reconstruction X3D SART PSNR 32.33 # 5
SSIM 0.9342 # 7
Novel View Synthesis X3D SAX-NeRF PSNR 51.37 # 1
SSIM 0.9994 # 1