Adobe's Interactive Video-Object Manipulation

Nearly every company that makes some sort of product has a research and development (R&D) division. This applies just as much to software developers as it does to hardware manufacturers. In an ideal world, the time and money spent on R&D makes it way into improvements in existing products and sometimes even into brand new products. As reality has it, though, many concepts and prototypes never make it out of the lab; and more often than not, the general pubic is never aware of what is going on deep inside a company's secret R&D lair.

Adobe has decided to break with the secret approach, however, and show the world some of the ideas coming out of its Advanced Technology Labs  (ATL). One of ATL's Graphics & Imaging "lablets" is located in Seattle, Washington and is staffed by research scientists from Adobe as well as from a number of universities, including the local University of Washington. One project that the lab is working on is called "Video Visualization and Interaction." Essentially, the project is designed to create new ways for users to easily interact with, manipulate, and add annotation to existing objects in videos--in a way, its aim to create a much simpler-to-use and much-quicker way of applying some of the motion graphics approaches that are traditionally relegated to high-end and expensive motion graphics applications, such as Adobe After Effects or Apple's Motion (which is part of Apple's Final Cut Pro suite). One of the project's researchers, Adobe's Dan B. Goldman recently posted a video online to shows a proof-of-concept of how some of these ideas can be implemented.

"Existing approaches to interaction with digital video are complex, and some operations lack the immediacy of interactive feedback. Our research in video annotation, visualization, and interaction harnesses computer vision to aid users in understanding and communicating with digital video. We are developing new approaches for computing the motion of points and objects in a video clip, and interactive systems that utilize this data to visually annotate independently moving objects in the video. We have applied this interface to construct visualizations of a short video clip in a single static image, using the visual language of storyboards. The principal advantage of the storyboard representation over standard representations of video is that it requires only a moment to observe and comprehend but at the same time retains much of the detail of the source video. The layout of the storyboard can be optimized to place the elements in a configuration that maximizes the clarity of presentation. We also explore novel interaction techniques for random video frame access using the natural spatial dimensions of a storyboard representation or an individual video frame."

It appears that Goldman has been working on this project for a number of years now, as this project was also his PhD Thesis in 2007 at the University of Washington. Interestingly, there is no guarantee that the technology displayed in this proof of concept will make its way into a shipping product.