Google Cloud Video Intelligence API Uses Machine Learning To Identify Objects Within Videos

Google Sign
If you’re a regular user of Google Photos, then you know that the app and online version of the service uses machine learning to “put a name to a face”, making it easy to organize or search for photos by typing in a person’s name. Or you could type in the search term “beach”, for example, to bring up some photos from your last trip to the coast.

Google is now expanding those capabilities to videos, with the Cloud Video Intelligence API. This new API, which is currently in Private Beta, employs deep learning and the TensorFlow framework to analyze online videos and pick out pertinent details for tagging purposes.

machine learning next 2

Not only can the API detect objects within a video, like a baseball or baseball bat at a little league game, but it can also identify actions. So, if you were a bystander at a marathon capturing race participants with your smartphone, the Cloud Video Intelligence API might tag a verb such as “running”.

“It can even provide contextual understanding of when those entities appear; for example, searching for ‘Tiger’ would find all precise shots containing tigers across a video collection in Google Cloud Storage,” states Google.

The API of course could be integrated with relative ease into Google’s incredibly popular YouTube video service. However, other companies will also be able to use the API for their own video catalogs to help organize content.

Google’s competition — namely Amazon Web Services and Microsoft Azure — don’t yet have this video tagging feature in their respective arsenals, but it stands to reason that they are also working on similar tagging algorithms.

Top Image Source: Matt Montagne/Flickr