Weizmann Invention: Visual Summarization

Yeda Research and Development, Ltd., the commercial arm of the Weizmann Institute of Science, recently entered into a license agreement with Adobe Systems Incorporated related to a bidirectional similarity measure to summarize visual data.

 

The bidirectional similarity method developed by Prof. Michal Irani and Drs. Denis Simakov, Yaron Caspi and Eli Shechtman of the Institute’s Computer Science and Applied Mathematics Department can be used on both still images and videos. The method produces a complete and coherent visual summary – a smaller or shorter version of the original that retains the most relevant information. The bidirectionality of the method ensures that the resulting image is visually coherent: In addition to telling the same “story,” it is as visually satisfying as the original. As opposed to cropping or clipping, in which important information can be lost; or scaling down, in which resolution is lost, summarizing manages to maintain relevant information as well as resolution details, despite the decrease in size. 

 

The method is based on eliminating redundant information from the image/video. Video summarization works in a similar way, except that the program exploits redundancy in space-time. Gradual resizing and rechecking ensures that the final result is seamless and coherent.

In addition to summarizing images and videos, the method may have a number of other applications: completing missing parts in images/videos; creating montages out of separate images; photo reshuffling (in which elements in the image/video can be moved around); automatic cropping; image synthesis (in which an image can be expanded, rather than summarized); and image morphing (generating a video sequence that displays a smooth transition from one image to another, possibly unrelated, image).

Prof. Michal Irani's research is supported by the Citi Foundation.