Pizza shop advertisement caught secretly tracking customers using face scanning tech

In Spielberg's 2002 sci-fi flick Minority Report, one seminal scene depicts our hero having to undergo a black-market eye transplant to avoid being caught by a biometric recognition system. Now, in the reality that is 2017, it's not just evil conspiracies using such tech – but pizza shops.

That's the world we now live in, where advertisements track your face to sell you food. This week, images circulated showing one marketing board from Peppe's Pizza in Norway which had crashed and was left displaying computer code of a real-time facial scanning system.

It appears the image was first uploaded to Facebook by a Redditer using the name Forsaken75 and later uploaded to linustechtips.com and shared on Twitter.

The initial poster said when he approached the screen it started to record his basic traits including his gender, if he was wearing glasses and if he was smiling.

"The intention behind my original post on Facebook was merely to point out that people may not know that these sort of demographics are being collected about them merely by approaching and looking at an advertisement. The camera was not, at a glance, evident," he said.

"It was merely meant as informational, maybe to point out what we all know or suspect anyway, but just to put it out in the open," he said.

"I believe the only intent behind the data collected is engagement and demographic statistics for better targeted advertisements," he added.

On Reddit, a commenter with marketing experience added: "I've worked with digital signage and that's exactly what is happening. No video or photo is recorded or sent anywhere.

"It's just a facial recognition software running on a PC which logs impressions for the display.

"Set up correctly you'll get decent stats for every message on the screen enabling the content producers to improve the overall effectiveness of the signage. No one at Peppe's has time to spy."

Others were not so sure, claiming such systems may have legal implications.

According to news outlet Dinside, the technology is made by artificial intelligence (AI) firm Kairos, sold via a digital marketing firm ProntoTV. The latter company's marketing manager, Jørn Olsen, confirmed the displayed ads are based on personal traits and facial expressions.

The Kairos website describes the system as having the ability to analyse "millions of pixels per second" and anonymously detect general viewer traits and engagement data from multiple people at the same time. No images or video files are being stored, the website claims.

A description reads: "The facial features in the image are detected, and any other objects [...] are ignored. Detection algorithms are based on a 'learned' face pattern [and] this allows the software to determine the gender and age of anonymous participants."

According to The Guardian, AI-based marketing is increasingly being used by companies around the world. Andy Martin, a business manager at facial recognition firm Axis, previously told the newspaper that the tech is being used to count how many people have viewed advertisements.

"Where I think we'll start to move on and actually see people individually recognised is customer loyalty schemes," he said last year.

"We're already working behind the scenes with some well-known retailers who are interested in spotting their loyal shoppers when they walk through the door," he continued, adding: "It would allow a sales assistant to be alerted and offer some a truly personalised shopping experience."

In May, DiGiorno pizza, the Nestlé brand, conducted an experiment using facial recognition technology to track the emotion reactions to different types of pizza. The test judged levels of joy, sorrow, anger, fear and surprise, Digiday reported at the time.

Social media firms, especially Facebook, are also becoming highly invested in face scanning capabilities to track users, link profiles and tag photographs in real-time.

Okay, so where's that eye doctor?