A multiview approach for the semantic segmentation of heritage building point clouds

Computer-aided digitization has become a powerful tool for enhancing documentation and preservation of cultural heritage as demonstrated by the recent emergence of Heritage Building Information Modeling (H-BIM). However, the reconstruction of as-built models still poses significant challenges, particularly in managing the large-scale data resulting from acquisition campaigns. To address these issues, this thesis proposes a novel point cloud semantic segmentation procedure based on a deep learning multiview approach. Firstly, the proposed approach employs a deep convolutional neural network to extract semantic information from multiple images resulting from a photogrammetric survey. Subsequently, the extracted semantic information is projected onto the 3D related photogrammetric point cloud by means of the intrinsic and extrinsic camera parameters. The method is validated and assessed through a series of tests using an image-point based dataset composed by five heritage scenes, specifically designed to train and test the proposed procedure. Overall, the results are still at an early stage in terms of predicting unseen scenarios, but the procedure demonstrate promising advancements in terms of performance and reliability if it is properly trained through datasets with a greater generalization capability.