Augmented vision: Seeing beyond field of view and occlusions via uncalibrated visual transfer from multiple viewpoints

We present an approach for merging into a single super-image a set of uncalibrated images of a general 3D scene taken from multiple viewpoints. To this aim, the content of either image is augmented with visual information taken from the others, while maintaining projective coherence. The approach extends the usual mosaicing techniques to image collections with 3D parallax, and operates like a virtual sensor provided with an enlarged field of view and the capability of seeing through visual occlusions in an “X-ray” fashion. Fundamental matrices are used to transfer visual information through the vertexes of an image graph. A dense stereo paradigm is employed to achieve photorealism by partitioning image pairs into corresponding regions. Results in oriented projective geometry are then exploited to both detect and handle occlusions by assessing the visibility properties of each transferred point.