How do we understand the world around us? How do we know what our eyes see? Gottfried Wilhelm Leibniz, the 17th-century philosopher and mathematician, believed that the eye performs an interpretive function; the information it conveys to the brain contains meaning. Immanuel Kant, in contrast, believed that the eye only “looks” while the brain applies meaning. Both components, he claimed, were necessary for recognition - sight without thought is blind, while thought without an image is empty.
Neuroscience has shown that roughly half of our brain cortex is dedicated to processing visual signals - more “brain power” than for any of the other senses. But researchers have yet to fully understand this intricate process. How does the brain piece together the signals reaching it from the eye to make sense of our world? Might the principles of human vision be used to design computers and robots capable of performing visually related tasks?These challenges, it turns out, are far from trivial. Every three-year-old child, for instance, can easily distinguish between a cat and a dog, but a computer -though capable of beating the world chess champion and calculating global economic models - is severely challenged when faced with this task.
Prof. Shimon Ullman of the Computer Science and Applied Mathematics Department is studying these and related questions. He and his team are trying to break down the complex tasks of vision into small fragments of information - similar to creating an alphabet dedicated to recognizing faces, a horse-recognition alphabet, etc. They create algorithms - programming recipes - that allow computers to identify different objects. For instance, an alphabet of components characteristic of a fair-skinned person will make it possible to detect such an individual in a crowd; a computer equipped with a horse-characterizing alphabet will single out the image of a horse from its surroundings, etc.
One day in the future, a computer will be able to watch a video that will teach it all that a three-year-old child has learned: how to identify new objects; how different machines and structures, as well as cats, dogs and other living beings, look from different angles and in different sizes and colors. Computers with such capabilities will advance a variety of fields - from the biomedical and electro-optic industries to space and aeronautics.
Prof. Shimon Ullman's research is supported by Mr. Efi Arazi, Israel; the Arthur and Rochelle Belfer Institute of Mathematics and Computer Science; and the M.D. Moross Laboratory for Vision Research and Robotics. He is the incumbent of the Ruth and Samy Cohn Professorial Chair of Computer Sciences.