An Adaptive Screen-Space Meshing Approach for Normal Integration

Oct 1, 2024·
Moritz Heep
Moritz Heep
,
Eduard Zell
· 2 min read
Abstract
Reconstructing surfaces from normals is a key component of photometric stereo. This work introduces an adaptive surface triangulation in the image domain and afterwards performs the normal integration on a triangle mesh. Our key insight is that surface curvature can be computed from normals. Based on the curvature, we identify flat areas and aggregate pixels into triangles. Compared to pixel grids, our triangle meshes adapt locally to surface details and yield much sparser representations. We derive a mesh-based formulation of the normal integration problem. Here, the sparser representations yield major runtime benefits.
Type
Publication
Computer Vision - ECCV 2024

Motivation

Increasing the resolution of the normal map improves the accuracy of fine structures but increases computational complexity. In smooth, featureless regions, this added complexity yields little additional information.

Output of our Adaptive Screen-Space Meshign Approach at different quality settings.
Our screen-space remeshing pipeline decimates smooth, featureless areas efficiently before the normal integration while preserving high-frequency details. Depicted results illustrate high, mid and low-resolution triangulations.
We introduce an adaptive screen-space meshing approach to reduce complexity before integration and give a full derivation of the normal integration on general triangle meshes.

By focusing on fine details and removing redundant information, we can avoid the quadratic growth of variables with increasing geometric resolution in pixel-based methods.

Overview

To create a feature-adaptive mesh before the actual surface integration, we must reliably predict detailed surface areas and adjust the density of the triangle mesh accordingly. We extract the surface curvature from the normal maps and use it to calculate an optimal edge length to keep the expected deviation between the triangle mesh and the underlying smooth surface below a user defined threshold.

Results

before after

Pixel-Wise

Ours

Integration results for pixel-based integration and mesh-based integration with 20k pixels/vertices respectively. Our adaptive vertex placement preserves high-frequency details much better than the regular pixel grid.

Citation

@InProceedings{adaptive_screen_meshing,
    author="Heep, Moritz and Zell, Eduard",
    title="An Adaptive Screen-Space Meshing Approach for Normal Integration",
    booktitle="Computer Vision -- ECCV 2024",
    year="2025",
    publisher="Springer Nature Switzerland",
    pages="445--461",
    isbn="978-3-031-72920-1",
    doi="10.1007/978-3-031-72920-1_25"
}
Normal Integration
Moritz Heep
Authors
Moritz Heep
PhD Student