Samsung keen to patent Google Glass-like invention that lets users interact with 3D images in mid-air

Samsung has applied to patent a Google Glass-style device that is light years ahead of Google's current technology and enables users to interact with 3D images in mid-air to dial phone numbers, send SMS text messages and even play a virtual piano keyboard.

Google Glass has not proved to be as successful as the internet giant previously hoped it would, with Google discontinuing sales of the device worldwide in January and announcing a need to rethink its strategy. For now, the gadget is regarded by many in the industry to be a rather expensive toy that does not solve any problems and does not really have any clear use.

However, Samsung has not given up on the dream of consumers one day wearing augmented reality glasses that work with their smartphones and, to that end, has been filing numerous patents relating to its own smart glasses project in September.

Various patents relating to components in Samsung's smart glasses project have been approved by the US Patent and Trademark Office, but in particular, one patent application filed with the World Intellectual Property Organization points at uses for smart glasses that are far beyond the scope of what Google imagined.

Interacting with 3D images in mid-air

The concept of Google Glass as it stands is to have a pair of smart glasses that work together as a snazzy accessory with your smartphone, similar to how the Apple Watch works.

However, where the Apple Watch has a physical watch face screen that users can tap on to answer calls, check their calendar or access apps, Google Glass interactions rely on the user either tapping a button on the right frame arm of the device or using specific voice commands.

Instead, the patent application for Samsung's smart glasses describes using two cameras in the glasses to project three-dimensional images in mid-air in order to create a virtual input interface, with the aim being that users then have a visual way to interact with apps and smartphone functions, similar to the way they would interact with their phones.

But Samsung's engineers have also gone a step further to visualise how a user could project the augmented reality objects onto physical real-life objects and use them to interact with the smart glasses' menus or apps.

For example, in order to make a phone call or send a text message, the user can project the phone's key pad on to his or her hand, held 10cm away from the glasses, and then use the fingers of another hand to press on the projected numbers on the hand to trigger the smartphone to make a call.

Using physical objects to interact with projected apps

Similarly, the user could also use this technique to type a quick SMS text message to a contact and flick through the phone's dictionary for the correct word or character.

Even better, the user could project a virtual piano keyboard from the glasses on to a desk and proceed to play virtual keys, or use a stylus, pen or finger to touch a blank note pad, on to which an app, such as a word processor with a QWERTY keyboard or a mobile game is projected. In the case of a game, speakers in the smart glasses would broadcast any sounds to the user's ears.

The technology being used is known as SixthSense. It was developed by Pranav Mistry while he was studying for a PhD at MIT in 2009 and he showcased it in a TEDIndia talk in the same year. Today, Mistry works for Samsung and he presented the very first version of the Samsung Gear VR headset in 2013.

"A current input method of smart glasses is limited. A user basically controls the smart glasses by using a voice command. However, it is difficult for the user to control the smart glasses by using only a voice command if a text input is required. Thus a wearable system that provides various input interaction methods is required," Samsung's engineers write in the patent application.

Can Samsung's ideas for smart glasses succeed where Google has so far failed? Perhaps creating an interactive experience with a visual aspect that users can touch would be good, but just imagine how strange you would look tapping on stuff in mid-air that no one else can see?

Perhaps it would be best to just get out your smartphone or tablet and interact with these devices since, you know, they already exist.

UPDATE: This article has been updated to reflect that Samsung has applied for the patent, rather than having been granted the patent.