Google plans to analyse food photos to count calories

Google is developing software capable of analysing still photos of food and counting the calories it contains, according to reports.

Google research scientist Kevin Murphy unveiled the Im2Calories project at the recent Rework Deep Learning Summit in Boston, Popular Science reports, describing how the new form of artificial intelligence could transform the health and fitness applications found on smartphones.

By using "sophisticated deep-learning algorithms" and "the depth of each pixel in an image" Im2Calories is able to determine both the type and portion size of the food.

"If it only works 30% of the time, it's enough that people will start using it, we'll collect data, and it'll get better over time," Murphy said.

"To me it's obvious that people really want this and this is really useful. OK fine, maybe we get the calories off by 20%. It doesn't matter. We're going to average over a week or a month or a year.

"We can start to potentially join this information from multiple people and start to do population level statistics. I have colleagues in epidemiology and public health, and they really want this stuff."

Im2Calories is still in the research and development phase but plans are already underway to test the technology in other situations.

One possible application could be in analysing traffic patterns in order to help drivers find parking spaces.

"If we can do this for food, that's just the killer app," Murphy said. "Suppose we did street scene analysis. We don't want to just say there are cars in this intersection. That's boring.

"We want to do things like localise cars, count the cars, get attributes of the cars, which way are they facing. Then we can do things like traffic scene analysis, predict where the most likely parking spot is.

"And since this is all learned from data, the technology is the same, you just change the data."