A Semi-Automatic Annotation Methodology for Photogrammetric Images of Heritage Buildings

Heritage Building Information Modelling (H-BIM) plays an increasing role in the Cultural Heritage sector. However, a key challenge persists in the Scan-to-BIM, the process of transforming point cloud data into usable models. Recent advancements in machine learning have enhanced the Scan-to-BIM, particularly by enabling more efficient 3D point cloud processing through semantic segmentation. In addition to methods based on the direct segmentation of 3D point clouds, there are also indirect approaches that rely on the intermediate segmentation of images representative of the 3D scene. However, their development remains limited due to the need for large datasets, which are currently unavailable for images of historical buildings and whose creation requires labour-intensive manual operations. This study introduces a semi-automated annotation technique to reduce per-pixel image annotation time by projecting manually assigned labels from 3D point clouds onto 2D images. The generated images can then support the training of image-based semantic segmentation models, which can then be integrated into multi-view or projection-based strategies for transferring the results back into 3D space. When compared to manual annotation and existing semi-automatic tools, our procedure, applied to selected case studies, yielded significant quantitative improvements in evaluation metrics such as Global Accuracy and Intersection over Union.