Spatial and temporal surface interpolation in structure from motion