Plenoptic Condensation A Novel Approach to Generalized Scene Reconstruction

Quidient Tech Note

Brevin Tilmon Alex DeJournett John Leffingwell Scott Ackerson
Quidient, Columbia, MD
arXiv Data
Teaser figure

PCon reconstructs the global car hood surface patch more than twice as accurately as the SOTA methods.
We compare PCon to two state-of-the-art generalized scene reconstruction (GSR) methods: NeRO and RT-Splatting. The top row shows real (GT) or rendered (GSR) images. The middle row shows virtually illuminated mesh surface normals. The bottom row shows the mesh illuminated by a virtual stripe projector with a manual BSDF that makes the dents easily perceived by humans.
PCon's local damage profile error is 35µm (0.035 mm)

Abstract

We present a novel Generalized Scene Reconstruction (GSR) approach called Plenoptic Condensation (PCon). PCon uses a multi-stage reconstruction pipeline, initially converting images into “soupy” scene elements with low (representational) power, then adaptively condensing the “soup” into “structured” elements of higher power capable of efficiently representing, for example, sharp edges and smooth reflective surfaces. PCon scene models called Reality Models™ (Relms) enable spatially varying representational power, which is essential for high-fidelity rendering, measurement, and scene understanding. We showcase several in-the-wild PCon reconstructions captured with consumer phone cameras. In one case called “Damaged Fiat,” PCon is benchmarked against two state-of-the-art (SOTA) GSR methods: NeRO and RT-Splatting. Referring to Figure 1 below, PCon reconstructs the global car hood surface patch more than twice as accurately as the SOTA methods. But more importantly, the local damage profile error for PCon is 35µm (0.035 mm), whereas the two other SOTA methods are essentially unable to measure the damage at all.