Image Pre-Segmentation from Shadow Masks

Motivation

While deep learning methods dominate image segmentation, they struggle with similar colors, complex textures, and require millions of annotated images. We propose a training-free approach that leverages shadow masks from photometric stereo data. Our key insight: changing light positions create shadow-to-light transitions that reveal object boundaries independent of colour or texture.

Overview

Our two-stage pipeline transforms shadow information into image segments:

1. Contour Detection: We detect object boundaries by analyzing shadow-to-light transitions across multiple lighting conditions, combining shadow masks with albedo and surface normal gradients for robustness.

2. Delaunay Segmentation: Detected contours are converted into a Delaunay triangulation, then progressively coarsened using a modified minimum spanning tree algorithm. Edge weights encode contour strength, preserving strong boundaries while merging weak ones.

The method allows resegmentation in real-time and requires no training data.

Results

Our analytical approach achieves competitive results compared to state-of-the-art methods:

• Strong performance on textured objects and similar-colored regions where traditional methods fail
• Complementary inputs: shadows detect boundaries, albedo/normals add robustness
• Interactive control: real-time adjustment of segmentation granularity via a single user parameter

Limitations: Best suited for scenes with multiple objects or well-separated parts.

Shadow information provides a powerful, underutilized signal for image segmentation—achieving results close to deep learning methods without any training data.

@InProceedings{Heep_2025_VMV,
    booktitle = {Vision, Modeling, and Visualization},
    title     = {{Image Pre-Segmentation from Shadow Masks}},
    author    = {Heep, Moritz and Parakkat, Amal Dev and Zell, Eduard},
    year      = {2025},
    publisher = {The Eurographics Association},
    doi       = {10.2312/vmv.20251239}
}