Flexible patches for recovering surfaces from binocular disparity

Observers are faced with the difficult task of inferring surfaces from retinal information. The problem is that the same retinal images are projected by infinitely many 3D structures. However, Lappin and Craft (2000) showed that second-order spatial derivatives taken locally along two spatial dimensions of smooth surface regions (i.e., 2D spatial manifolds) are isomorphic to second-order derivatives of image properties (e.g. disparities). That is, there is a one-to-one mapping between surface structures and image properties. In line with this analysis, they found that observers measured these properties very consistently. In this work, we argue that disparity fields specified by a few texture elements sampled from random spatial positions on surfaces are inherently ambiguous. Given this ambiguity, two alternative hypotheses can be formulated: (1) 2D second-order spatial manifolds can interpolate the sparse data (Lappin & Craft, 2000), or (2) simple planar or low-curvature “flexible” patches can fit local image regions. According to the first hypothesis, we expect that human performance on tasks involving the placement of probe dots on smooth surfaces (Exp. 1) or discriminating smooth from randomly depth-jittered surfaces (Exp. 2) should not depend on the magnitude of local curvature of quadratic (second-order) surfaces that are simulated in random-dot stereo displays. On the contrary, we found that human performance systematically decreased with curvature. Moreover, we found that observers more accurately discriminated planar surfaces from curved surfaces (using at-threshold curvature values for individual observers) than from depth-jittered surfaces with the same range of disparities as the curved surfaces (Exp. 3). Thus, given ambiguous retinal information, the results across the three experiments suggest that observers infer surfaces by fitting simple yet “flexible” patches to sparse data.