The ease of seeing hides many complexities. A fundamental problem is the one of fragmentation we are able to perceive objects although they are optically incomplete or fragmented, in particular due to occlusion. A central mechanism that addresses this difficulty in biological and artificial visual systems is contour or curve completion, which has been studied in different ways in the different vision sciences. Without denying the role of top down influences, curve completion is known to be much influence by bottom up, stimulus driven processes, which often override explicit knowledge about the scenea or even a lifetime of visual experience. Want to experience of the strength of curve completion? Move the cursor over the field of circles on the right to introduce some occluders and observe how the perception and interpretation of the scene changes drastically, from circles to vertical waves. The same phenomenon can be demonstrated in reversed. No doubt you see a nice cylindical neck behind the occluder of the ostrich. But you might be surprised when you move you mouse over the image to reveal the true shape. The examples below show some more shape completion examples that conflict the context, and one modal completion where observers report illusory shape.
Recent computational, neurophysiological, and psychophysical studies suggest that completed contours emerge from activation patterns of orientation selective cells in the primary visual cortex, or V1. In this project we suggest modeling these patterns as 3D curves in the mathematical continuous space R^2 — S^1, a.k.a. the unit tangent bundle associated with the image plane R^2, that abstracts V1. Then, we propose that the completed shape may follow physical/biological principles which are conveniently abstracted and analyzed in this space. We implement our theories by numerical and biologically plausible algorithms to show ample experimental results of visually completed curves in natural and synthetic scenes.
Our research into curve completion in the tangent bundle explores various completion principle and implementation techniques. While the technical details must be obtained from the papers, here we show some visual results.
Minimum length in the tangent bundle – Analytic/numerical results
These demos show the result of the completion process based on the least action principle in the tangent bundle. Please move your mouse over each of the images to see the computational completed curve (or curves) for the stimulus presented. In all cases, the inducers (position and orientation) were provided manually.
Minimum length in the tangent bundle – Distributed solution with locally connected parallel networks.
These demos show the progression of the completion process in the biologically-plausible parallel network. Each iteration is a frame and each white point signals an active neuron in the network that computes the completed curve.
Curvature-based completion principles
- MLTB numerical library – Curve completion via minimal length in the tangent bundle
- MLTB network code – parallel network implementation for MLTB
- ETB numerical library – Coming soon…
- G. Ben-Yosef and O.Ben-Shahar, Tangent bundle curve completion with locally connected parallel networks , in Neural Computation, in press..
- G. Ben-Yosef and O.Ben-Shahar, A tangent bundle theory for visual curve compeltion, in IEEE Transactions on Pattern Analysis and Machine Intelegence (TPAMI), 2012.
- G. Ben-Yosef and O.Ben-Shahar, A biologically-inspired theory for non-axiomatic parametric curve completion , in the Proceesinds of the Asian conference on Computer Vision (ACCV), 2010.
- G. Ben-Yosef and O.Ben-Shahar, Minimum length in the tangent bundle as a model for curve completion , in the Proceesinds of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2010.
Who and Where…
This research is a joint work by Guy Ben-Yosef and Ohad Ben-Shahar of the Computer Science Department, Ben-Gurion University of The Negev. Beer Sheva, Israel Different aspects of the work were presented in various conferences, including CVPR 2010, ACCV 2010, the Israel Computer Vision Day 2011, and the Israel Machine Vision Conference (IMVC) 2012.
This work was funded in part by the European Commission in the 7th Framework Programme (CROPS GA no. 246252) and the Israel Science Foundation (ISF grant No. 259/12). We also thank the generous support of the Frankel fund, the Paul Ivanier center for Robotics Research and the Zlotowski Center for Neuroscience at Ben-Gurion University.