Startup CrunchFish Builds 'Kinect' for Smartphones

Swedish mobile app developer Crunchfish has prototyped a mobile app that lets users control their devices using finger gestures in the air. The app requires a front-facing camera and can replace the touchscreen gesture control with what the company calls "3D control."

Crunchfish says that this 3D control means more than simply clicking, dragging and scrolling without touching the phone. It means objects on the phone can be twisted, turned and manipulated in 3D, similar to using Microsoft Kinect. Tho' let's not get ahead of ourselves. The demo was, so far, limited to mouse-like motions and clicking.

Crunchfish gesture control working in low light.

However, Crunchfish has also separately demonstrated software where the objects move in 3D, so obviously the next step is to link it all together.

Add this 3D demo with the demon video above and you get something really interesting.

Crunchfish claims the technology works in low-light and that no extra hardware is needed, so if it gets past the demo stage, it's possible that users savvy enough to root their phones may be able to install it themselves. Then they would also be on the hook to develop their own apps for it. Crunchfish says that their demonstrations at MWC lead to lots of interest from handset developers, so here's hoping that it will one-day materialize as an option for phones and tablets, complete with video games ready to be played.

The company is best known for its mobile apps, including the Hearway, an audio navigation app, and Yapa, a polling application for mobile devices.

Here's the full press release:

The Solution can replace touch screen interaction with movement detection only. The device will need a camera giving the user the opportunity to click, drag and drop and scroll by doing finger movements in front of the camera.

Touch screens has been around for as long as we can remember so Crunchfish decided it was time for the next step in user interaction. Crunchfish has prototyped its latest cell phone innovation making it possible for users to control their user interface on mobile phones using finger gestures in the air.

Sure, now you do not have to clean your screen from fatty spots from your fingers. But the real advantage is 3D control. What the heck does that mean?

Since the controller responds to your fingers' position in real space and are not confined to a flat 2D screen anymore, the user can control graphical objects in 3 dimensions. The applications are endless, from chemical molecule manipulations to interaction with 3D avatar characters.

And the really cool thing is that this comes with no additional hardware need - the solution uses the existing front cam. A prototype has been showcased in Crunchfish's research lab.

The next step of the minority report is to integrate the gesture control with Crunchfish's existing 3D rendering engine for cell phones. Handset manufacturers can enable new games and UI interaction with 3rd part developers.

You guessed it: It also work without any additional hardware.