Facebook adds AI to its mobile app to transform your videos into works of art

Facebook has unveiled brand new deep learning software called Caffe2go that will allow smartphone users to apply and transfer artistic styles for live video streams on iOS and Android. Although deep learning usually requires "the content to be sent off to data centres for processing on big-computer servers", the new software can "capture, analyse and process pixels in real time" from "the palm of your hand", Facebook notes.

Demonstrating the new technology at the Web Summit technology conference in Lisbon, Portugal, on Tuesday (8 November), Facebook CTO Mike Schroepfer said this "is the exciting future of AI".

"This is one application of AI on the device, it's one of the the first," Schroepfer said, CNBC reports. "But the real breakthrough here is being able to train and build models on a big server... and deploy them directly to your pocket so you can can run them in real time wherever you are."

With the AI framework baked into Facebook's mobile app, the social media giant condensed the size of the AI software, allowing it to run "various deep neural networks with high efficiency" and to apply different styles to videos as you're taking them.

Facebook adds that having an "industrial-strength" deep learning platform on its mobile app also allows for other interesting opportunities.

"We can create gesture-based controls, where the computer can see where you're pointing and activate different styles or commands," the company said in a statement. "We can recognise facial expressions and perform related actions, like putting a "yay" filter over your selfie when you smile. With Caffe2Go, AI has opened the door to new ways for people to express themselves."

As interest in the field of AI continues to grow globally, Facebook says its latest technology is part of its ongoing efforts to turn "the latest research breakthroughs into tools, platforms and infrastructure" that allows anyone at Facebook to implement AI in their own projects at scale.

"When our research succeeds in teaching computers all [these] abilities... these will add up to something like what we call common sense," Facebook said. "And when computers have common sense they can interact with us in better, more natural ways, from surfacing the most relevant information for us and assisting us with tasks to enabling whole new ways for people to connect."

Facebook said it is rolling Caffe2go out via Facebook's stack and will open-source parts of the AI framework over the next few months.