MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures

Neural Radiance Fields (NeRFs) have demonstrated amazing ability to synthesize images of 3D scenes from novel views. However, they rely upon specialized volumetric rendering algorithms based on ray marching that are mismatched to the capabilities of widely deployed graphics hardware. This paper introduces a new NeRF representation based on textured polygons that can synthesize novel images efficiently with standard rendering pipelines. The NeRF is represented as a set of polygons with textures representing binary opacities and feature vectors. Traditional rendering of the polygons with a z-buffer yields an image with features at every pixel, which are interpreted by a small, view-dependent MLP running in a fragment shader to produce a final pixel color. This approach enables NeRFs to be rendered with the traditional polygon rasterization pipeline, which provides massive pixel-level parallelism, achieving interactive frame rates on a wide range of compute platforms, including mobile phones.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Novel View Synthesis LLFF SNeRG PSNR 25.63 # 12
SSIM 0.818 # 8
LPIPS 0.183 # 2
Novel View Synthesis LLFF MobileNeRF PSNR 25.91 # 10
SSIM 0.825 # 7
LPIPS 0.183 # 2
Novel View Synthesis LLFF NeRF PSNR 26.5 # 7
SSIM 0.811 # 9
LPIPS 0.25 # 1
Novel View Synthesis LLFF JAXNeRF PSNR 26.92 # 2
SSIM 0.831 # 6
LPIPS 0.173 # 4
Novel View Synthesis Mip-NeRF 360 NeRF PSNR 0.2146 # 3
SSIM 0.458 # 3
LPIPS 0.515 # 1
Novel View Synthesis Mip-NeRF 360 MobileNeRF PSNR 21.95 # 2
SSIM 0.47 # 2
LPIPS 0.47 # 2
Novel View Synthesis Mip-NeRF 360 NeRF++ PSNR 22.76 # 1
SSIM 0.548 # 1
LPIPS 0.427 # 3
Novel View Synthesis NeRF NeRF PSNR 31 # 7
SSIM 0.947 # 7
LPIPS 0.081 # 1
Novel View Synthesis NeRF MobileNeRF PSNR 30.9 # 8
SSIM 0.947 # 7
LPIPS 0.062 # 2
Novel View Synthesis NeRF SNeRG PSNR 30.38 # 9
SSIM 0.95 # 6
LPIPS 0.05 # 4
Novel View Synthesis NeRF JAXNeRF PSNR 31.65 # 6
SSIM 0.952 # 5
LPIPS 0.051 # 3

Methods